Data access method and apparatus, and device and computer-readable storage medium

Abstract

A data access method and apparatus, and a device and a computer-readable storage medium, which belong to the technical field of computers. In the present method, for a read request from a client, data segments of data requested by the read request are read on a segment-by-segment basis; each time a data segment is read, the data segment is cached in a cache space, and the data segment is returned to the client; after the data segment is returned to the client, a storage space occupied by the data segment in the cache space can be released without waiting for the completion of reading other data segments; thus, the duration during which a single data segment occupies a small storage space in the cache space is reduced, such that the duration during which the entire data that the client requests to read occupies a large storage space in the cache space can be reduced, and the entire cache space is enabled to provide caching services for more read and write requests, thereby improving the utilization efficiency of the entire cache space. When the cache space is a memory, the utilization efficiency of the memory can be improved.

Description

Data access methods, apparatus, devices and computer-readable storage media

[0001] This application claims priority to Chinese Patent Application No. 202510189374.0, filed on February 19, 2025, entitled “Data Access Method, Apparatus, Device and Computer-Readable Storage Medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of computer technology, and in particular to a data access method, apparatus, device, and computer-readable storage medium. Background Technology

[0003] Current storage systems typically include a storage controller and multiple storage media. Storage media are memory devices used to store data, such as hard disk drives (HDDs) or solid-state drives (SSDs). The storage controller manages these multiple storage media, and the memory within the storage controller provides data caching services for the storage controller.

[0004] A client can access the storage medium through a storage controller to read or write data. Taking reading data as an example, the process of a client accessing the storage medium can be as follows: the client sends a read request to the storage controller, indicating that it wants to read the first data; after receiving the read request, the storage controller first allocates storage space for the first data from its memory, then reads the first data from the storage medium based on the read request, caches all the read first data in the allocated storage space, and transmits the first data in the storage space to the client. After the first data is successfully transmitted to the client, the storage controller releases the storage space occupied by the first data.

[0005] However, during the process of the client accessing the storage medium, the storage space is only released after all the first data has been transferred to the client. This results in a long memory occupation time for a single read request, and the memory cannot provide data caching services for more read requests, thus leading to low memory utilization efficiency of the storage control device. Summary of the Invention

[0006] This application provides a data access method, apparatus, device, and computer-readable storage medium, which can improve the utilization efficiency of the memory of the storage control device during data reading. The technical solution is as follows:

[0007] Firstly, a data access method is provided, which applies a storage control device, which can be a controller for a storage medium, such as an HDD or an SSD.

[0008] The method includes: a storage control device receiving a read request from a client; in response to the read request, the storage control device reading a first data segment of first data from a storage medium, caching the first data segment in a cache space, returning the first data segment to the client, and releasing the storage space occupied by the first data segment in the cache space; the storage control device reading a second data segment of the first data from the storage medium, caching the second data segment in the cache space, returning the second data segment to the client, and releasing the storage space occupied by the second data segment in the cache space. The read request indicates reading the first data, and the first and second data segments are different data segments of the first data. The cache space can be the memory of the storage control device or an on-chip cache on the storage control device.

[0009] This method addresses client read requests by reading data in segments. Each segment is cached in a cache space (potentially the memory of the storage controller or an on-chip cache within the storage controller) and returned to the client. After returning the segment, the cache space is released without waiting for other segments to finish reading. This reduces the time a single segment occupies a small portion of the cache space, thus reducing the time the entire data request occupies a large portion of the cache space. This allows the cache space to serve more read and write requests, improving overall cache space utilization. When the cache space is memory, it also improves memory utilization. When the cache space is an on-chip cache, there's no need to read or write data to the storage controller's memory during read request processing, avoiding the memory wall problem.

[0010] In one possible implementation, the storage control device reading a first data segment of first data from the storage medium includes: sending a first read task to the storage medium and receiving a first data segment returned by the storage medium in response to the first read task; the storage control device reading a second data segment of the first data from the storage medium includes: sending a second read task to the storage medium and receiving a second data segment returned by the storage medium in response to the second read task. The first read task indicates reading the first data segment, and the second read task indicates reading the second data segment.

[0011] Based on the above possible implementation methods, the client's read request is split into multiple read tasks targeting the storage medium. By sending multiple read tasks to the storage medium, different data segments can be read from the storage medium.

[0012] In one possible implementation, the data size of the first data segment is less than or equal to the maximum transmission unit of the network between the storage controller and the client, and the data size of the second data segment is less than or equal to the maximum transmission unit.

[0013] Based on the above possible implementation methods, when the storage control device reads the data segment of the first data, it can return the data segment to the client through the network without further splitting the data segment. This allows the process of reading the data segment of the first data from the storage medium to the network transmission to be completely pipelined, thereby improving the processing efficiency of read requests.

[0014] In one possible implementation, the read request carries target address information of the first data, which indicates the target storage location of the first data on the client; the aforementioned return of the first data segment to the client includes: sending a first read response to the client, wherein the first read response carries the first data segment and first sub-target address information, which indicates a portion of the storage location within the target storage location; the aforementioned return of the second data segment to the client includes: sending a second read response to the client, which carries a second data segment and second sub-target address information, which indicates a portion of the storage location within the target storage location.

[0015] In one possible implementation, the network protocol used between the client and the storage controller is any one of NVMe based on ROCE, a network file system based on ROCE, or NVMe based on the Transmission Control Protocol, where NVMe is the Non-Volatile Memory Host Controller Interface Specification and ROCE is Remote Direct Memory Access based on Converged Ethernet.

[0016] These network protocols are defined by relevant technical standards. Based on the possible implementation methods described above, during the data reading process, the storage control device can interact with the client through messages supported by the network protocols defined by the relevant technical standards. This allows the storage control device and the client to interface using the network protocols defined by the relevant technical standards. The general-purpose client is already capable of interfaceing with the network protocols defined by the relevant technical standards, so users can read data through the general-purpose client without modifying it.

[0017] In one possible implementation, the read request carries metadata of the first data. After the storage control device receives the read request from the client, the method further includes: the storage control device determining the storage location of the first data in the storage medium based on the metadata.

[0018] Based on the above possible implementation methods, for each read request from the client, the storage controller only needs to maintain one set of metadata, which will not increase the metadata processing load of the storage controller.

[0019] In one possible implementation, the cache space includes the memory of the storage control device, and the method further includes: before or after reading the first data segment, the storage control device requests storage space from the memory for the first data segment; before or after reading the second data segment, the storage control device requests storage space from the memory for the second data segment.

[0020] Based on the above possible implementation methods, multiple timings for requesting storage space are provided so that memory space can be requested for data segments at different times in different scenarios, thus adapting to various scenarios.

[0021] In one possible implementation, caching the second data segment in the cache space includes: after the storage space occupied by the first data segment is released, using the storage space originally occupied by the first data segment to cache the second data segment.

[0022] Based on the above possible implementation methods, multiple data segments can reuse the same storage space, thereby reducing the storage space occupied in the cache space during data reading and reducing the occupation of the cache space.

[0023] In one possible implementation, caching the first data segment in the cache space includes any of the following: caching the first data segment in the memory of the storage controller; or caching the first data segment in an on-chip cache on the storage controller; or, if the on-chip cache is sufficient to store the first data segment, caching the first data segment in the on-chip cache, and if the on-chip cache is insufficient to store the first data segment, caching the first data segment in memory; or, if the memory is sufficient to store the first data segment, caching the first data segment in memory, and if the memory is insufficient to store the first data segment, caching the first data segment in the on-chip cache. The caching method for the second data segment can refer to the caching method for the first data segment, and will not be repeated here.

[0024] Based on the above possible implementation methods, multiple caching methods for data segments are provided so that different caching methods can be used to cache data segments in different scenarios, thereby maximizing the utilization of cache space.

[0025] In one possible implementation, after the storage control device receives a read request from the client, the method further includes: the storage control device generating first status information and second status information, wherein the first status information indicates the read status and transmission status of the first data segment, and the second status information indicates the read status and transmission status of the second data segment; the read status indicates whether the data segment has been read, and the transmission status indicates whether the data segment has been transmitted to the client; based on this, if the first data segment is read, the storage control device sets the read status of the first data segment to "read" in the first status information; if the first data segment is successfully transmitted, the storage control device sets the transmission status of the first data segment to "transmitted" in the first status information; if the second data segment is read, the storage control device sets the read status of the second data segment to "read" in the second status information; if the second data segment is successfully transmitted, the storage control device sets the transmission status of the second data segment to "transmitted" in the second status information.

[0026] Based on the above possible implementation methods, the state of the data segment during the data reading process is maintained through state information, so as to monitor the processing status of the data segment during the data reading process.

[0027] Secondly, a data access method is provided, applied to the aforementioned storage control device. The method includes: the storage control device receiving a write request from a client; in response to the write request, the storage control device reading a third data segment of second data from the client, caching the third data segment in a cache space, writing the third data segment to the storage medium, and releasing the storage space occupied by the third data segment in the cache space after the third data segment is written; the storage control device reading a fourth data segment of the second data from the client, caching the fourth data segment in the cache space, writing the fourth data segment to the storage medium, and releasing the storage space occupied by the fourth data segment in the cache space after the fourth data segment is written. The write request indicates writing second data, and the third and fourth data segments are different data segments from the second data.

[0028] This method addresses client write requests by segmenting the data to be written into segments from the client's memory. Each segment is read, cached in a cache space (potentially the memory of the storage controller or an on-chip cache), and then written to the storage medium. Once this segment is written, it releases its storage space without waiting for other segments to complete. This reduces the time a single segment occupies a small portion of the cache space, thus reducing the time the entire client write request occupies a large portion of the cache space. This allows the cache space to serve more read and write requests, improving overall cache space utilization. When the cache space is memory, this also improves memory utilization efficiency. When the cache space is an on-chip cache, there's no need to read or write data to memory during write request processing, avoiding the memory wall problem.

[0029] In one possible implementation, the storage control device reading the third data segment of the second data from the client includes: the storage control device sending a third read task to the client and receiving the third data segment returned by the client in response to the third read task; the storage control device reading the fourth data segment of the second data from the client includes: the storage control device sending a fourth read task to the client and receiving the fourth data segment returned by the client in response to the fourth read task. The third read task indicates reading the third data segment, and the fourth read task indicates reading the fourth data segment.

[0030] Based on the above possible implementation methods, the client's write request is split into multiple read tasks for the client. By sending multiple read tasks to the client, different data segments are read from the storage medium.

[0031] In one possible implementation, the write request carries target address information of the second data, which indicates the target storage location of the second data on the storage medium; writing the third data segment to the storage medium includes sending a first write task to the storage medium, wherein the first write task carries the third data segment and first sub-target address information, which indicates a portion of the storage location in the target storage location; writing the fourth data segment to the storage medium includes sending a second write task to the storage medium, wherein the second write task carries the fourth data segment and second sub-target address information, which indicates a portion of the storage location in the target storage location.

[0032] Based on the above possible implementation methods, the client's write request is split into multiple write tasks for the storage medium. By sending multiple write tasks to the client, different data segments are written to the storage medium.

[0033] In one possible implementation, the network protocol used between the client and the storage controller is any one of NVMe based on ROCE, a network file system based on ROCE, or NVMe based on the Transmission Control Protocol, where NVMe is the Non-Volatile Memory Host Controller Interface Specification and ROCE is Remote Direct Memory Access based on Converged Ethernet.

[0034] These network protocols are defined by relevant technical standards. Based on the possible implementation methods described above, during the data writing process, the storage control device can interact with the client through messages supported by the network protocols defined by the relevant technical standards. This allows the storage control device and the client to interface using the network protocols defined by the relevant technical standards. The general-purpose client is already capable of interfaceing with the network protocols defined by the relevant technical standards, so users can read data through the general-purpose client without modifying it.

[0035] In one possible implementation, the data size of the third data segment is less than or equal to the maximum transmission unit of the network between the storage controller and the client, and the data size of the fourth data segment is less than or equal to the maximum transmission unit.

[0036] Based on the above possible implementation methods, when the client returns the data segment of the second data to the storage control device via the network, there is no need to further split the data segment, so that the process of reading the data segment of the second data from the client to the network transmission can be completely pipelined, thereby improving the processing efficiency of write requests.

[0037] In one possible implementation, the above method also includes:

[0038] After the third data segment is written, the storage controller generates metadata for the third data segment; after the fourth data segment is written, the storage controller generates metadata for the fourth data segment.

[0039] Based on the above possible implementation methods, the storage control device generates metadata for different data segments of the second data, so that the metadata of the second data can be generated subsequently based on the metadata of each data segment of the second data.

[0040] In one possible implementation, the cache space includes the memory of the storage control device, and the method further includes:

[0041] Before or after reading the third data segment, the storage control device requests storage space from memory for the third data segment; before or after reading the fourth data segment, the storage control device requests storage space from memory for the fourth data segment.

[0042] Based on the above possible implementation methods, multiple timings for requesting storage space are provided so that memory space can be requested for data segments at different times in different scenarios, thus adapting to various scenarios.

[0043] In one possible implementation, caching the fourth data segment in the cache space includes: after the storage space occupied by the third data segment is released, using the storage space originally occupied by the third data segment to cache the fourth data segment.

[0044] Based on the above possible implementation methods, multiple data segments can reuse the same storage space, thereby reducing the storage space occupied in the cache space during the data writing process and reducing the occupation of the cache space.

[0045] In one possible implementation, caching the third data segment in the cache space includes any of the following: caching the third data segment in the memory of the storage controller; or caching the third data segment in an on-chip cache on the storage controller; or, if the on-chip cache is sufficient to store the third data segment, caching the third data segment in the on-chip cache, and if the on-chip cache is insufficient to store the third data segment, caching the third data segment in memory; or, if the memory is sufficient to store the third data segment, caching the third data segment in memory, and if the memory is insufficient to store the third data segment, caching the third data segment in the on-chip cache. The caching method for the fourth data segment can refer to the caching method for the third data segment, and will not be elaborated further here.

[0046] Based on the above possible implementation methods, multiple caching methods for data segments are provided so that different caching methods can be used to cache data segments in different scenarios, thereby maximizing the utilization of cache space.

[0047] In one possible implementation, after the storage control device receives a write request from the client, the method further includes: the storage control device generating third status information and fourth status information, wherein the third status information indicates the read status and write status of the third data segment, and the fourth status information indicates the read status and write status of the fourth data segment; the read status indicates whether the data segment has been read, and the write status indicates whether the data segment has been written to the storage medium; if the third data segment is read, the storage control device sets the read status of the third data segment to "read" in the third status information; if the third data segment is written, the storage control device sets the write status of the third data segment to "written" in the third status information; if the fourth data segment is read, the storage control device sets the read status of the fourth data segment to "read" in the fourth status information; if the fourth data segment is written, the storage control device sets the write status of the fourth data segment to "written" in the fourth status information.

[0048] Based on the above possible implementation methods, the state of the data segment during the data reading process is maintained through state information, so as to monitor the processing status of the data segment during the data writing process.

[0049] Thirdly, a storage control device is provided for performing the method provided in the first aspect or any alternative to the first aspect, as well as the method provided in the second aspect or any alternative to the second aspect.

[0050] Fourthly, a storage control device is provided, the storage control device including a processor for executing program code, causing the storage control device to perform a method as provided in the first aspect or any alternative of the first aspect, and a method as provided in the second aspect or any alternative of the second aspect.

[0051] Fifthly, a computer-readable storage medium is provided, the storage medium storing at least one piece of program code, the program code being read by a processor to cause a storage control device to perform a method as provided in the first aspect or any alternative of the first aspect, and a method as provided in the second aspect or any alternative of the second aspect.

[0052] In a sixth aspect, a computer program product or computer program is provided, the computer program product or computer program including program code stored in a computer-readable storage medium, a processor in a storage control device reading the program code from the computer-readable storage medium, the processor executing the program code, causing the storage control device to perform the method provided by the first aspect or various optional implementations of the first aspect, as well as the method provided by the second aspect or various optional implementations of the second aspect.

[0053] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0054] Figure 1 is a schematic diagram of a storage system provided in an embodiment of this application;

[0055] Figure 2 is a flowchart of a data access method provided in an embodiment of this application;

[0056] Figure 3 is a flowchart of a data reading method provided in an embodiment of this application;

[0057] Figure 4 is a flowchart of a data reading method that prioritizes the use of on-chip cache according to an embodiment of this application;

[0058] Figure 5 is a flowchart of a data reading method with priority memory usage provided in an embodiment of this application;

[0059] Figure 6 is a schematic diagram of the structure of a message provided in an embodiment of this application;

[0060] Figure 7 is a flowchart of another data access method provided in an embodiment of this application;

[0061] Figure 8 is a flowchart of a data writing method provided in an embodiment of this application;

[0062] Figure 9 is a flowchart of a data writing method that prioritizes the use of on-chip cache according to an embodiment of this application;

[0063] Figure 10 is a flowchart of a data writing method with priority given to memory usage, provided in an embodiment of this application.

[0064] Figure 11 is a schematic diagram of a storage control device provided in an embodiment of this application;

[0065] Figure 12 is a schematic diagram of another storage control device provided in an embodiment of this application;

[0066] Figure 13 is a schematic diagram of the structure of a storage control device provided in an embodiment of this application. Detailed Implementation

[0067] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0068] This application provides a data access method applied to a storage system. Figure 1 is a schematic diagram of a storage system provided in an embodiment of this application. As shown in Figure 1, the storage system includes a client 101, a storage control device 102, and a storage medium 103. The client 101 is communicatively connected to the storage control device 102, and the storage medium 103 is communicatively connected to the storage control device 102, so that the client 101 can access the storage medium 103 through the storage control device 102 to read and write business data on the storage medium 103. The storage control device 102 is, for example, a storage controller or a storage server. The storage medium 103 is, for example, an HDD or an SSD, and is located inside or outside the storage control device.

[0069] Compared to the storage control device 102, the client 101 can have different deployment methods, and the communication connection method between the client 101 and the storage control device 102 is different for different deployment methods.

[0070] Client 101 is computer software or hardware capable of running computer software (e.g., a server). In some embodiments, client 101 is deployed remotely at storage control device 102. The hardware containing client 101 communicates with storage control device 102 via a first network. The network transmission protocol supported by the first network is referred to as the first network transmission protocol. Client 101 and storage control device 102 exchange information using messages supported by the first network transmission protocol. The first network includes IP networks, Ethernet networks, or other types of networks. The first network transmission protocol includes NVMe over ROCE, NFS over ROCE, NVM over TCP, or NFS over TCP / UDP. NVMe is the Non-Volatile Memory Host Controller Interface Specification, ROCE is Remote Direct Memory Access (RDMA) over Converged Ethernet, NFS is a network file system, TCP is the Transmission Control Protocol, and UDP is the User Datagram Protocol. In this embodiment, the type of the first network and the type of the first network transmission protocol are not limited.

[0071] In some embodiments, a software client 101 is deployed inside a storage controller 102. The client 101 and the processor in the storage controller 102 are connected via a bus in the storage controller 102. This bus may be a peripheral component interconnect (PCI) bus or a peripheral component interconnect express (PCIE) bus, etc.

[0072] Compared to the storage control device 102, the storage medium 103 can have different deployment methods, and the communication connection method between the storage medium 103 and the storage control device 102 is different for different deployment methods.

[0073] In some embodiments, the storage medium 103 is deployed outside the storage control device 102. The storage medium 103 is communicatively connected to the storage control device 102 via a second network. The network transmission protocol supported by the second network is referred to as the second network transmission protocol. The storage medium 103 and the storage control device 102 exchange information through messages supported by the second network transmission protocol. The second network may be the same as or different from the first network. The second network may include an IP network, an Ethernet network, or other types of networks. The second network transmission protocol may be the same as or different from the first network transmission protocol. The second network transmission protocol includes NVMe over ROCE, NFS over ROCE, NVMe over TCP, or NFS over TCP / UD, etc. Here, the embodiments of this application do not limit the type of the second network or the type of the second network transmission protocol.

[0074] In some embodiments, the storage medium 103 is deployed inside the storage control device 102, and the storage medium 103 and the processor in the storage control device 102 are connected via a bus in the storage control device 102.

[0075] The storage system includes at least one storage medium 103. In the case of multiple storage media 103 in the storage system, the types of the multiple storage media 103 may be the same or different; for example, some storage media 103 may be HDDs, and others may be SSDs. Here, the embodiments of this application do not limit the type of storage media 103 in the storage system or the number of each type of storage media 103.

[0076] Storage control device 102 is used to manage storage media 103 in the storage system and provide storage services to client 101, including block storage, object storage, file storage, NAS, SAN, NFS, FTP, etc. Here, this embodiment does not limit the type or number of storage services.

[0077] Client 101 is used to write and read business data. For example, client 101 writes business data to storage medium 103 and reads business data from storage medium 103 through storage control device 102. Business data includes block data, object data, file data, network attached storage (NAS) data, storage area network (SAN) data, NFS data, file transfer protocol (FTP) data, etc. Here, this application embodiment does not limit the type of business data.

[0078] During data read / write operations by client 101, storage control device 102 caches the business data involved in the data read / write process in an associated cache space. For example, during data read operations, client 101 requests storage control device 102 to read business data. In response to client 101's read request, storage control device 102 reads the business data from storage medium 103, caches the read business data in the cache space, and then transmits the business data in the cache space to client 101. During data write operations, client 101 requests the business data to be written and sends it to storage control device 102. Storage control device 102 first caches the business data sent by client 101 in the cache space, and then writes the business data in the cache space to storage medium 103. The cache space associated with storage control device 102 includes the memory of storage control device 102 or on-chip cache on storage control device 102, such as central processing unit (CPU) cache, graphics card cache, network card cache, data processing unit (DPU) cache, etc. The storage control device 102 manages the storage medium 103, including managing the storage space of the storage medium and allocating logical addresses to the storage medium 103.

[0079] The memory of the storage control device 102 includes dynamic random access memory (DRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), static random access memory (SRAM), or read-only memory (ROM), etc. In this embodiment, the type of memory is not limited.

[0080] In related technologies, in response to a client's read request, the storage control device allocates memory space for the read request at the granularity of the read request, reads all the requested data from the storage medium, caches all the read data in the allocated memory space, and then transmits the data in that memory space to the client. Only after all the data in that memory space has been successfully transmitted to the client is the memory space released. This results in a single read request occupying memory space for an excessively long time, and the storage control device's memory cannot provide data caching services for more read requests, thus leading to low memory utilization efficiency.

[0081] Based on this, in this application, in response to a client's read request, the storage control device reads each data segment of the data requested by the read request at the data segment granularity. Each time a data segment is successfully transmitted to the client, the storage space occupied by that data segment is released. A data segment is a part of the data requested by the client, and multiple data segments together constitute the data requested by the client. Next, this data reading process will be described with reference to Figure 2. Figure 2 is a flowchart of a data access method provided by an embodiment of this application. This method is applied to the aforementioned storage system and includes the following steps.

[0082] 201. The client sends a first read request to the storage control device, which instructs the reading of the first data.

[0083] The first read request is any read request from the client, and it is a large I / O request. The first data is the business data read by the client in this request. The first read request carries the storage address information of the first data, which indicates the storage location of the first data in the storage medium. The storage address information includes a starting address A and a first target offset. The starting address A is the starting address of the first data in the storage medium, and it is a virtual address. The first target offset is the offset of the first data between the ending address of the storage control device and the starting address A, and this ending address is also a virtual address.

[0084] Optionally, the first read request may also carry target address information for the first data, indicating the target storage location of the first data on the client. The target address information includes a starting address B and a second target offset. The starting address B is the starting address of the first data on the client, i.e., the starting address where the client stores the first data. The second target offset is the offset between the ending address of the first data on the client and the starting address B; this ending address is a virtual address. In some embodiments, the first read request may also omit the target address information for the first data.

[0085] Optionally, the first read request carries metadata for the first data. Metadata is information used to describe the data, providing contextual information about the stored data and helping the system manage and organize it. Metadata itself does not contain the actual data content, but rather describes information such as the data's attributes, structure, and location. The metadata for any given data includes its size, format, storage address on the storage medium, or other data characteristics.

[0086] Based on the metadata of the first data, the subsequent storage control device determines the storage location of the first data in the first storage medium. For example, based on the storage address information of the first data in the metadata, the storage location of the first data in the first storage medium is determined, so that a corresponding storage location is determined for each data segment of the first data in the storage location of the first data, thereby determining the sub-storage address information corresponding to each data segment.

[0087] In related technologies, if the amount of initial data is large, when reading the initial data, the client splits the initial read request into multiple read requests and sends these multiple read requests to the storage controller. Each read request is used to read a portion of the initial data, and each read request carries a copy of metadata. For the initial data, the storage controller needs to maintain multiple copies of metadata, increasing the metadata processing load on the storage controller.

[0088] In this embodiment of the application, when reading the first data, the client does not send a split first read request. Instead, the client sends a single read request to the storage controller. When processing the first read request, the storage controller only needs to maintain the metadata carried by this single first read request. That is, for the first data, only one set of metadata is maintained, thus not increasing the metadata processing load of the storage controller.

[0089] 202. The storage control device receives the first read request from the client.

[0090] Upon receiving the first read request, the storage control device determines the amount of first data based on the first read request. For example, if the first read request carries the amount of first data, the amount of first data is obtained from the first read request; if the first read request does not carry the amount of first data, the amount of first data is determined by the storage capacity of the storage location indicated by the storage address information of the first data.

[0091] If the amount of the first data is less than or equal to the first threshold, the storage control device reads the first data at the data granularity of the first data, completing the first read request. If the amount of the first data is greater than or equal to the first threshold, the storage control device reads each data segment of the first data at the data segment granularity, and transmits the read data segment to the client after each data segment is read (as described in steps 203 and 204 below). When all data segments of the first data have been transmitted to the client, the first read request is completed.

[0092] The first threshold is the maximum data size of a data segment when processing read requests at the granularity of data segments. The first threshold can be less than or equal to, or greater than, the maximum transmission unit (MTU) of the first network between the client and the storage controller. The MTU of the first network is the size of the maximum data packet that the first network can transmit. In this embodiment, the first threshold is not limited.

[0093] For example, during the processing of a first read request, the storage control device generates at least one read task for the first read request based on a first threshold and the amount of first data. Each read task instructs the reading of a segment of the first data, the amount of which is less than or equal to the first threshold. For instance, if the amount of first data is greater than the first threshold, the storage control device generates multiple read tasks for the first read request. If the amount of first data is less than or equal to the first threshold, the storage control device generates only one read task for the first read request; in this case, the read task instructs the reading of the first data, i.e., the first data is the segment of data instructing the read task to read.

[0094] The data segment indicated by a read task is called the data segment corresponding to the read task; that is, each read task corresponds to a data segment of the first data. Each read task has task information, which includes sub-memory address information and / or sub-target address information of a data segment.

[0095] The sub-storage address information indicates the storage location of a data segment in the storage medium, including the starting address of the data segment in the storage medium and the offset between the starting and ending addresses. The starting address of the data segment in the storage medium can be a virtual address or a physical address. The sub-target address information indicates the storage location of a data segment on the client, which is a partial storage location within the target storage location of the first data. The sub-target address information includes the starting address of the data segment on the client and the offset between the starting and ending addresses of the data segment on the client. Both the starting and ending addresses of the data segment on the client are virtual addresses.

[0096] For example, suppose n read tasks are generated for the first read request to logically divide the first data into n data segments, namely data segment 1 to data segment n, where n is an integer greater than 0. The task information for each read task is shown in Table 1 below.

[0097] Table 1

[0098] In some embodiments, the storage control device generates status information for each data segment. The status information for each data segment includes a read status and a transmission status. The read status indicates whether the data segment has been read, and includes "not read" and "read." In some embodiments, the read status also includes "reading in progress." Different read statuses can be indicated using different status identifiers. The transmission status indicates whether the data segment has been transmitted to the client, and includes "not transmitted" and "transmitted." In some embodiments, the transmission status also includes "transmitting in progress." Different transmission statuses can be indicated using different status identifiers.

[0099] The storage device can add the status information of each data segment to the task information of the corresponding read task. Initially, the storage control device sets the read status of each data segment of the first data to "not read" and the transmission status of each data segment to "not transmit". Taking Table 1 as an example, the read status of each data segment from data segment 1 to data segment n is set to "not read", and the transmission status of each data segment is set to "not transmit".

[0100] The storage control device records the corresponding read task by storing the task information of each read task.

[0101] For each read task, the storage controller can read the corresponding data segment from the storage medium. Upon reading each data segment of the first data, the data segment is cached in the cache space associated with the storage controller, the data segment in the cache space is transmitted to the client, and after returning the data segment to the client, the storage space occupied by the data segment in the cache space is released. For example, steps 203 or 204 below.

[0102] 203. The storage control device reads the first data segment of the first data from the first storage medium, caches the first data segment in the cache space, returns the first data segment to the client, and releases the storage space occupied by the first data segment in the cache space after returning the first data segment to the first client.

[0103] Here, the cache space is the cache space associated with the storage control device. The first storage medium is the storage medium in the storage system that stores the first data. The first data segment is the data segment corresponding to any read task of the first read request. If the first read request has one read task, the first data segment is the first data; if the first read request has multiple read tasks, the first data segment is any segment of the first data. When the cache space is the memory of the storage control device, caching the first data segment to the cache space means caching the first data segment to memory; when the cache space is an on-chip cache on the storage control device, caching the first data segment to the cache space means caching the first data segment to the on-chip cache.

[0104] The storage control device can read a first data segment by sending a first read task to the first storage medium, wherein the first read task indicates that the first data segment should be read. The first read task can be carried out through a read request; for example, the storage control device can send a second read request to the first storage medium for reading the first data segment, wherein the second read request includes first sub-storage address information, which is the sub-storage address information of the first data segment. In response to the second read request, the first storage medium returns the first data segment to the storage control device. The storage control device receives the first data segment returned by the first storage medium, thereby reading the first data segment and completing the read task corresponding to the first data segment.

[0105] If the first data segment is read, the storage control device sets the read status of the first data segment in the first status information to "read". Here, the first status information refers to the status information of the first data segment. Taking Table 1 as an example, assuming the first data segment is data segment 1, after reading data segment 1, the read status of data segment 1 in the task information of read task 1 is updated to "read".

[0106] In some embodiments, before or after the first data segment is read, the storage control device requests storage space from the cache space for the first data segment.

[0107] Taking the allocation of storage space before reading the first data segment as an example, the storage control device allocates storage space for the first data segment from the cache space. For example, storage space can be allocated for the first data segment before sending a second read request to read the first data segment, or after sending the second read request but before receiving the first data segment returned by the first storage medium.

[0108] Taking the allocation of storage space after reading the first data segment as an example, let's assume the cache space is the memory of the storage controller. After receiving the first data segment returned by the first storage medium, the storage controller first temporarily caches the first data segment in the on-chip cache of the storage controller, and then allocates storage space for the first data segment from that memory. After allocating the storage space, the storage controller then caches the first data segment from the on-chip cache into that storage space in the memory.

[0109] The storage capacity of the storage space requested for the first data segment can be equal to or greater than the data volume of the first data segment. For example, if the storage control device requests storage space for the first data segment based on the data volume of the first data segment, the storage capacity of the requested storage space is equal to the data volume of the first data segment.

[0110] After caching the first data segment into the cache space, the storage control device transmits the first data segment from the cache space to the client.

[0111] The storage control device can return a first data segment to the client by sending a read response carrying the first data segment. For example, the storage control device sends a first read response to the client, which carries the first data segment and first sub-target address information, where the first sub-target address information is the sub-target address information of the first data segment. The client receives the first read response and stores the first data segment at the storage location indicated by the first sub-target address information. After receiving the first read response or after storing the first data segment, the client sends a successful transmission notification to the storage control device to inform the storage control device that the first data segment has been successfully transmitted. The storage control device receives the successful transmission notification.

[0112] In some embodiments, after the storage control device sends the first read response, the storage control device releases the storage space occupied by the first data segment in the cache space.

[0113] In other embodiments, the storage space is released if the first data segment is successfully transmitted. For example, if a successful transmission notification of the first data segment is received, the storage space is released; if no successful transmission notification of the first data segment is received, the storage space is not released until the successful transmission notification of the first data segment is received.

[0114] Releasing the storage space occupied by any data segment includes deleting the data segment from the cache space.

[0115] If the first data segment is successfully transmitted, the storage space occupied by the first data segment is released so that if the first data segment fails to be transmitted, the first data segment can be retrieved from the storage space and retransmitted to the client, thus meeting the requirement of retransmitting data segments.

[0116] In related technologies, when processing the first read request, the first data is only returned to the client after it has been fully read into memory, and the storage space occupied by the first data in memory is only released after all the first data in memory has been transferred to the client. In contrast to these technologies, in this embodiment, after reading the first data segment, it is not necessary to wait for all the first data to be read before returning the first data segment to the client. Furthermore, after returning the first data segment to the client, the storage space occupied by the first data segment in the cache space is released without waiting for other data segments of the first data to be transferred to the client. This reduces the time the first data segment occupies that storage space, allowing the storage space to provide caching services for other read and write requests as quickly as possible, thereby improving the utilization efficiency of the storage space.

[0117] If the first data segment is successfully transmitted, the storage control device can also set the transmission status of the first data segment in the first status information to "transmitted" to indicate that the first data segment has been successfully transmitted to the client. Taking Table 1 as an example, assuming the first data segment is data segment 1, after reading data segment 1, the transmission status of data segment 1 in the task information of read task 1 is updated to "transmitted".

[0118] 204. The storage control device reads the second data segment of the first data from the first storage medium, caches the second data segment in the cache space, returns the second data segment to the client, and releases the storage space occupied by the second data segment in the cache space after returning the second data segment to the client.

[0119] The second data segment is any data segment in the first data other than the first data segment. When the cache space is the memory of the storage control device, caching the second data segment into the cache space means caching the second data segment into memory; when the cache space is an on-chip cache on the storage control device, caching the second data segment into the cache space means caching the second data segment into the on-chip cache.

[0120] The storage control device can read the second data segment by sending a second read task to the first storage medium, wherein the second read task indicates that the second data segment should be read. The first read task can be carried out by a read request; for example, the storage control device sends a second read request to the first storage medium for reading the second data segment, wherein the second read request includes second sub-storage address information, which is the sub-storage address information of the second data segment. In response to the second read request, the first storage medium returns the second data segment to the storage control device. The storage control device receives the second data segment returned by the first storage medium, thereby reading the second data segment and completing the read task corresponding to the second data segment.

[0121] Before or after the second data segment is read, the storage control device requests storage space from the memory for the second data segment. This process is similar to the process of requesting storage space from the memory for the first data segment, and will not be described in detail here.

[0122] If the second data segment is read, the storage control device sets the read status of the second data segment in the second status information to "read," indicating that the second data segment has been read. Here, the second status information refers to the status information of the second data segment. Taking Table 1 as an example, assuming the second data segment is data segment 2, after reading data segment 2, the read status of data segment 2 in the task information of read task 2 is updated to "read."

[0123] The storage controller can return a second data segment to the client by sending a read response carrying the second data segment. For example, the storage controller sends a second read response to the client, carrying the second data segment and second sub-target address information, where the second sub-target address information is the sub-target address information of the second data segment. The client receives the second read response and stores the second data segment at the storage location indicated by the second sub-target address information. After receiving the second read response or after storing the second data segment, the client sends a successful transmission notification to the storage controller to inform the storage controller that the second data segment has been successfully transmitted. The storage controller receives the successful transmission notification.

[0124] In some embodiments, after the storage control device sends the second read response, the storage control device releases the storage space occupied by the second data segment in the cache space.

[0125] In other embodiments, if the second data segment is successfully transmitted, the storage control device releases the storage space occupied by the second data segment in the cache space. For example, if a successful transmission notification of the second data segment is received, the storage space is released; if no successful transmission notification of the second data segment is received, the storage space is not released temporarily, but is released after the successful transmission notification of the second data segment is received.

[0126] If the second data segment is successfully transmitted, the storage space occupied by the second data segment is released so that if the second data segment is not successfully transmitted, the second data segment can be retrieved from the storage space and retransmitted to the client, thus meeting the requirement of data segment retransmission.

[0127] If the second data segment is successfully transmitted, the storage control device can also set the transmission status of the second data segment in the second status information to "transmitted" to indicate that the second data segment has been successfully transmitted to the client. Taking Table 1 as an example, assuming the second data segment is data segment 2, after data segment 2 is transmitted to the client, the transmission status of data segment 2 in the task information of read task 2 is updated to "transmitted".

[0128] The above description uses the example of executing step 203 first, followed by step 204. In other embodiments, the storage control device may execute step 204 first, followed by step 203. For example, in a scenario where multiple data segments of the first data are processed sequentially, if the storage location of the second data segment is before the first data segment, then step 204 is executed first, followed by step 203. In other embodiments, the storage control device may execute steps 203 and 204 simultaneously. For example, in a scenario where multiple data segments of the first data are processed in parallel, the storage control device executes steps 203 and 204 simultaneously. Here, the execution order of steps 203 and 204 is not limited in this embodiment.

[0129] If the first data includes other data segments besides the first and second data segments, the storage control device reads these other data segments from the first storage medium, caches them in the cache space, returns them to the client, and releases the storage space occupied by these other data segments in the cache space after returning them to the client. This process is similar to steps 203 or 204 above and will not be described again here.

[0130] Once all data segments of the first data have been successfully transmitted to the client, the storage control device sends a read success notification to the client, indicating that the data has been successfully read. For example, if the read status of all data segments of the first data is "read" and the transmission status of all data segments is "transmitted," then the storage control device sends a read success notification to the client.

[0131] When the client and storage controller are connected via a first network, when sending any data segment of the first data to the client, the processor of the storage controller sends a read response (such as a first read response or a second read response) carrying that data segment to the network interface card (NIC) connected to the first network. The NIC then sends the read response to the client via the network. If the data size of the segment is greater than the MTU of the first network, the NIC first splits the data segment into multiple sub-segments based on the MTU, with each sub-segment having a data size less than or equal to the MTU. These sub-segments are then transmitted to the client via the first network. While transmitting a sub-segment, the NIC sends a read response carrying that sub-segment to the client. This process continues until all sub-segments have been successfully transmitted, at which point the read response processing is complete. If the data size of the segment is less than or equal to the MTU of the first network, the NIC sends the read response to the client via the first network.

[0132] For multiple data segments of the first data, the processor of the storage controller sends multiple read responses to the network interface card (NIC) of the storage controller. The NIC processes the multiple read responses sequentially according to the order of receipt. If the data size of the data segments in multiple read responses is greater than the MTU of the first network, then when processing a read response, the data segment carried by the read response needs to be split and sent, and other read responses need to wait for the processing of this read response to be completed. If the data size of the data segments in multiple read responses is less than or equal to the MTU of the first network, then the NIC sends each received read response to the client sequentially according to the order of receipt, that is, a read response is sent immediately upon receipt, so that the process of reading the data segments of the first data from the storage medium to the network transmission can be completely piped, thereby improving the processing efficiency of the first read request.

[0133] If the amount of data in the first read request is greater than or equal to a first threshold, multiple read tasks are generated for the first read request, and a second read request is generated for each of these tasks. This splits the first read request into multiple second read requests. The granularity of the data read by the second read requests is smaller than that of the first read request, thus breaking down a large read request from the client into multiple read requests. The large read request is then completed by processing these multiple smaller read requests.

[0134] A more intuitive description is as follows: In this embodiment, the storage controller converts a large I / O request (which can be either a read or write request) from the client into a small I / O request from the storage controller to the storage medium. When the large I / O request is a read request, it corresponds to the data requested by the client, and the small I / O request corresponds to a data segment. The same applies when the large I / O request is a write request. The following section uses a read request as an example and provides a detailed explanation of this process in conjunction with Figure 3.

[0135] As shown in Figure 3, in step S11, the client sends a large read request (such as the first read request) to the storage control device. In step S12, the storage control device splits the large read request into multiple smaller read requests (i.e., the second read request), and requests storage space from the cache space for each smaller read request to occupy the requested storage space. The storage control device also sends multiple smaller read requests to the storage medium to initiate a disk read operation for each smaller read request. In step S13, the storage medium processes each smaller read request individually and returns the data requested by each smaller read request (i.e., the data segment) to the storage control device. Each time the storage control device receives the data requested by a smaller read request, it writes the data into the storage space requested by that smaller read request. In step S14, after writing the data requested by each smaller read request into the storage space, the storage control device sends the data requested by that smaller read request from memory to the client. In step S15, each time the client receives the data requested by a smaller read request, the storage control device returns a notification of successful data transmission. In step S16, the storage controller receives a successful data transfer notification from the client. In step S17, after all data requested by the smaller read requests has been successfully transferred, the storage controller sends a successful read notification to the client for the larger read request.

[0136] The method provided in the embodiment shown in Figure 2, for a client's read request, reads each data segment of the data requested by the read request in segments. Each time a data segment is read, it is cached in a cache space (which may be the memory of the storage controller or an on-chip cache on the storage controller) and returned to the client. After returning the data segment to the client, the storage space occupied by this data segment in the cache space is released without waiting for other data segments to finish reading. This reduces the time a single data segment occupies a small portion of the cache space, thereby reducing the time the entire data requested by the client occupies a large portion of the cache space. This allows the entire cache space to provide caching services for more read and write requests, thus improving the overall utilization efficiency of the cache space. When the cache space is memory, this also improves memory utilization efficiency. When the cache space is an on-chip cache, there is no need to read or write data to the memory of the storage controller during the read request processing, thus avoiding the memory wall problem during this process.

[0137] Regarding the multiple data segments of the first data, the above explanation uses the example of allocating separate cache space for each data segment and having each data segment occupy its own dedicated storage space. In other embodiments, multiple data segments may reuse the same storage space.

[0138] For example, suppose n read tasks are generated for the first read request. The storage control device requests m storage spaces from the cache space for the n read tasks. The storage capacity of each of the m storage spaces is greater than or equal to a first threshold, and m is an integer greater than 0 and less than n.

[0139] For m storage spaces, the storage control device can execute the read tasks of the first read request in batches. For example, taking m read tasks out of n read tasks as a batch of read tasks, the storage control device executes this batch of read tasks first. During the execution of this batch of read tasks, the storage control device reads m data segments of the first data from the storage medium. Each time a data segment is read, it is cached in one of the m storage spaces. In this way, even if m data segments are read simultaneously, they can be cached separately in each of the m storage spaces. During the execution of this batch of read tasks, after each data segment is cached in a storage space, the storage control device transmits the data segment from that storage space to the client. If the data segment is successfully transmitted, the storage space occupied by that data segment is released. After this batch of read tasks is completed, if the number of remaining read tasks is greater than m, the storage control device selects m read tasks from the remaining read tasks as a new batch of read tasks and executes the new batch of read tasks. This process continues until the number of remaining read tasks is less than or equal to m. Then, the storage control device selects the remaining read tasks as the last batch of read tasks and executes the last batch of read tasks.

[0140] For example, suppose n equals 10 and m = 5, meaning the first read request has 10 read tasks, and 5 storage spaces are allocated for it. For these 5 storage spaces, 5 read tasks are executed first, each corresponding to one storage space. During the execution of these 5 read tasks, each task reads a data segment as instructed by the task, caches that segment in its corresponding storage space, and if the data segment is successfully transmitted, deletes the storage space occupied by that segment. After these 5 read tasks are completed, the remaining 5 read tasks are executed in a similar manner.

[0141] In this scenario, if the read tasks corresponding to the first and second data segments of the first data are in different batches—for example, the read task corresponding to the first data segment is in the first batch of read tasks, and the read task corresponding to the second data segment is in the second batch of read tasks—during the execution of the first batch of read tasks, after the first data segment is successfully transmitted, the storage space occupied by the first data segment is released. During the execution of the second batch of read tasks, after the second data segment is read, the storage space originally occupied by the first data segment is used to cache the second data segment, thus allowing the first and second data segments to reuse the same storage space.

[0142] For m storage spaces, the read tasks of the first read request can also be executed without batching. For example, the storage control device first executes m read tasks for the first read request. During this process, during the execution of each read task, the data segment read based on that read task is cached in one of the m storage spaces. After the data segment is successfully transmitted, the storage space is released, and the read task is completed. After each read task is completed, one storage space becomes free. The storage control device selects a new read task from the remaining read tasks to execute. When executing the new read task, the data segment read based on the new read task is cached in the free storage space. This process continues until all read tasks are completed.

[0143] For example, suppose the storage control device executes a read task corresponding to the first data segment. During execution, the first data segment is cached in one of m storage spaces. After the first data segment is successfully transmitted, the storage space occupied by the first data segment is released, and the read task corresponding to the first data segment is completed. Then, the storage control device executes a read task corresponding to the second data segment. During execution, after reading the second data segment, the storage space originally occupied by the first data segment is used to cache the second data segment. This allows the first and second data segments to reuse the same storage space.

[0144] For example, assuming n equals 4 and m = 2, the first read request has four read tasks, namely read task 1 to read task 4. Two storage spaces are allocated for the first read request, namely storage space 1 and storage space 2. The storage control device first executes read task 1 and read task 2, caching data segment 1 read based on read task 1 into storage space 1. After data segment 1 is successfully transmitted, storage space 1 is released, and read task 1 is completed. Then, data segment 2 read based on read task 2 is cached into storage space 2. After data segment 2 is successfully transmitted, storage space 2 is released, and read task 2 is completed. After read task 1 is completed, the storage control device executes read task 3, caching data segment 3 read based on read task 3 in storage space 1. After read task 2 is completed, the storage control device executes read task 4, caching data segment 4 read based on read task 4 in storage space 2.

[0145] In this embodiment of the application, multiple data segments of the first data can reuse the same storage space, thereby reducing the storage space occupied in the cache space during the reading of the first data and reducing the occupation of the cache space.

[0146] The above explanation uses the memory or on-chip cache of the storage control device as an example to illustrate the cache space associated with the storage control device. In some embodiments, the cache space includes both the memory of the storage control device and the on-chip cache on the storage control device. In this case, during the reading of the first data, the on-chip cache is used preferentially to store the read data segment, or the memory is used preferentially to store the read data segment. Next, these two preferential storage methods will be described in conjunction with Figures 4 and 5.

[0147] Taking the priority use of on-chip cache as an example, during the reading of the first data, the on-chip cache is used first to store the read data segment. If the on-chip cache's storage capacity is insufficient, then memory is used to store the data segment. For example, each time a data segment of the first data is read, if the on-chip cache is sufficient to store the data segment, the data segment is cached in the on-chip cache and returned to the client. If the on-chip cache is insufficient to store the data segment, the storage control device caches the data segment in memory and returns the data segment in memory to the client for transmission. Sufficient on-chip cache to store the data segment includes situations where the remaining storage capacity of the on-chip cache is greater than or equal to the storage amount of the data segment. Sufficient on-chip cache to store the data segment also includes situations where the remaining storage capacity of the on-chip cache is less than the storage amount of the data segment. Next, this implementation process will be described using Figure 4 as an example.

[0148] Figure 4 is a flowchart of a data reading method that prioritizes the use of on-chip cache according to an embodiment of this application. The method includes the following steps.

[0149] 401. The client sends a first read request to the storage control device, which instructs the reading of the first data.

[0150] Step 401 is similar to step 201, and will not be described again here.

[0151] 402. The storage control device receives the first read request from the client.

[0152] Step 402 is similar to step 202, and will not be described again here.

[0153] 403. The storage control device reads the first data segment of the first data from the first storage medium. If the on-chip cache on the storage control device is sufficient to store the first data segment, the first data segment is cached in the on-chip cache, and the first data segment is returned to the client. After returning the first data segment to the first client, the storage space occupied by the first data segment in the on-chip cache is released.

[0154] Step 403 is similar to step 203 above, except that in step 403, the first data segment is cached in the on-chip cache first. After reading the first data segment, if the on-chip cache is insufficient to store the first data segment, the storage control device caches the first data segment in the storage control device's memory, returns the first data segment in memory to the client, and releases the storage space occupied by the first data segment in memory after returning the first data segment to the first client.

[0155] 404. The storage control device reads the second data segment of the first data from the first storage medium. If the on-chip cache on the storage control device is sufficient to store the second data segment, the second data segment is cached in the on-chip cache, and the second data segment is returned to the client. After returning the second data segment to the client, the storage space occupied by the second data segment in the on-chip cache is released.

[0156] Step 404 is similar to step 204 above, except that in step 404, the second data segment is cached in the on-chip cache first. After reading the second data segment, if the on-chip cache is insufficient to store the second data segment, the storage control device caches the second data segment in the storage control device's memory, returns the second data segment to the client, and releases the storage space occupied by the second data segment in memory after returning the second data segment to the client.

[0157] The above description uses the example of executing step 403 first and then step 404. In other embodiments, the storage control device may execute step 404 first and then step 403, or execute steps 403 and 404 simultaneously. Here, the execution order of steps 403 and 404 is not limited in this application embodiment.

[0158] If the first data includes other data segments besides the first and second data segments, the storage control device reads these other data segments from the first storage medium. After reading the other data segments, if the on-chip cache is sufficient to store them, the other data segments are cached in the on-chip cache, returned to the client, and the storage space occupied by them in the on-chip cache is released. If the on-chip cache is insufficient to store them, the other data segments are cached in memory, the second data segment is returned to the client, and the storage space occupied by them in memory is released. This process is similar to steps 403 or 404 above and will not be described again here.

[0159] Once all data segments of the first data have been returned to the client, the storage control device sends a notification to the client that the first data has been successfully read.

[0160] In this embodiment, during the client's data reading process, the on-chip cache is used first to cache the read data segments. Only when the on-chip cache's storage capacity is insufficient will the data segment of the first data in the internal cache be used, thereby reducing the space occupied by the read data. During the processing of read requests, the on-chip cache is used first to store the data to be written, reducing the number of times data is read and written in memory, thus mitigating the memory wall problem.

[0161] Taking the priority use of memory as an example, in some embodiments, during the reading of the first data, memory is used first to store the read data segment. If the memory storage capacity is insufficient, an on-chip cache is then used to store the data segment. For example, each time a data segment of the first data is read, if the memory is sufficient to store the data segment, the data segment is cached in memory and returned to the client. If the memory is insufficient to store the data segment, the storage control device caches the data segment in an on-chip cache and returns the data segment in the on-chip cache to the client. Sufficient memory to store the data segment includes: the remaining memory storage capacity is greater than or equal to the storage amount of the data segment. Sufficient memory to store the data segment also includes: the remaining memory storage capacity is less than the storage amount of the data segment. Next, this implementation process will be described using Figure 5 as an example.

[0162] Figure 5 is a flowchart of a data reading method with priority given to memory usage, provided in an embodiment of this application. The method includes the following steps.

[0163] 501. The client sends a first read request to the storage control device, which instructs the reading of the first data.

[0164] Step 501 is similar to step 201, and will not be described again here.

[0165] 502. The storage control device receives the first read request from the client.

[0166] Step 502 is similar to step 202, and will not be described again here.

[0167] 503. The storage control device reads the first data segment of the first data from the first storage medium. If the memory of the storage control device is sufficient to store the first data segment, the first data segment is cached in the memory and returned to the client. After returning the first data segment to the client, the storage space occupied by the first data segment in the memory is released.

[0168] Step 503 is similar to step 203 above, except that in step 503, the first data segment is cached in memory first. After reading the first data segment, if the memory is insufficient to store the first data segment, the storage control device caches the first data segment in the on-chip cache on the storage control device, returns the first data segment in the on-chip cache to the client, and releases the storage space occupied by the first data segment in the on-chip cache after returning the first data segment to the first client.

[0169] 504. The storage control device reads the second data segment of the first data from the first storage medium. If the memory of the storage control device is sufficient to store the second data segment, the second data segment is cached in memory and returned to the client. After returning the second data segment to the client, the storage space occupied by the second data segment in memory is released.

[0170] Step 504 is similar to step 204 above, except that in step 504, the second data segment is cached in memory first.

[0171] After reading the second data segment, if there is insufficient memory to store the second data segment, the storage controller caches the second data segment in the on-chip cache on the storage controller, returns the second data segment in the on-chip cache to the client, and releases the storage space occupied by the second data segment in the on-chip cache after returning the second data segment to the client.

[0172] The above description uses the example of executing step 503 first and then step 504. In other embodiments, the storage control device may execute step 504 first and then step 503, or execute steps 503 and 504 simultaneously. Here, the execution order of steps 503 and 504 is not limited in this application embodiment.

[0173] If the first data includes other data segments besides the first and second data segments, the storage control device reads these other data segments from the first storage medium. After reading the other data segments, if there is sufficient memory to store them, the other data segments are cached in memory, returned to the client, and the storage space occupied by these other data segments in memory is released. If there is insufficient memory to store the other data segments, the other data segments are cached in an on-chip cache, returned to the client, and the storage space occupied by these other data segments in the on-chip cache is released. This process is similar to steps 503 or 504 above and will not be described again here.

[0174] Once all data segments of the first data have been returned to the client, the storage control device sends a notification to the client that the first data has been successfully read.

[0175] In this embodiment, by prioritizing the use of memory cache to read data segments during the client's data reading process, and caching the read data segments to on-chip cache when memory is insufficient, the data segments are prevented from waiting for memory space for a long time, thereby enabling the data segments to be transmitted to the client as quickly as possible, reducing the time for the client to read data, and thus improving data reading efficiency.

[0176] The embodiments shown in Figures 2 to 5 above are all described using the example of storing the first data in a single storage medium (i.e., the first storage medium). In other embodiments, the first data may also be stored in multiple storage media, all of which are the first storage media. In this case, the first data can also be read according to the data reading method shown in Figures 2 to 5. The difference lies in that, during the process of reading the first data, the storage control device reads different data segments of the first data from each of these multiple storage media.

[0177] In the embodiments of the above data access methods, when the client and the storage control device are connected through a first network, the storage control device and the client interact with each other. The interacted information includes a first read request, a read response, a transmission success notification, and a read success notification. The interacted information can be carried by different messages supported by the network transmission protocol of the first network, including any one of NVMe over ROCE, NFS over ROCE, NVMe over TCP, or NFS over TCP / UDP. For different network transmission protocols, the messages carrying the interaction messages during the data reading process differ between the client and the storage control device. The data reading process is described below for each of these network transmission protocols.

[0178] Taking the network transmission protocol of the first network as NVMe over ROCE as an example, the process of the client reading data is the same as that shown in Figure 3, and the process includes the following steps.

[0179] S21. The client sends a ROCE SEND message to the storage controller. The ROCE SEND message carries an NVMe read request, which instructs the target data of length Len to be read into the client address (ClientAddr).

[0180] The ROCE SEND message encapsulates an NVMe read request. This is the first read request, the target data is the first data item, and the NVMe read request carries the target data's ClientAddr and Offset. The target data's ClientAddr is the starting address (i.e., starting address B) on the client side, and the target data's Offset is the offset of the target data relative to its own ClientAddr. The target data's ClientAddr and Offset indicate the storage space starting from ClientAddr and extending to the specified offset.

[0181] In relevant technical standards, NVMe read requests in ROCE SEND messages support carrying the source address and the corresponding offset. The client can use the ClientAddr of the target data as the source address and the Offset of the target data as the offset of the source address, and encapsulate the ClientAddr and Offset of the target data in the NVMe read request in the ROCE SEND message according to the message format defined by the relevant technical standards, so that the message format of the ROCE SEND message does not need to be changed.

[0182] S22. The storage control device receives the ROCE SEND message, splits the NVMe read request into n read tasks, and sends a separate read request to the disk for each read task. The read request carries the DiskAddr and Offset of the DataCell.

[0183] Here, n represents the number of data cells (DataCells) of the target data, and each DataCell is a data segment of the target data. Each read task indicates the reading of one DataCell of the target data. The storage control device also records the starting address of each DataCell and the offset of each DataCell relative to its own starting address. The offset of a DataCell relative to its own starting address is called the DataCell's Offset. DiskAddr is the starting address of the DataCell on the disk. The DiskAddr and Offset of the DataCell indicate the storage space of the DataCell on the client side. The disk is the storage medium, and the disk read request is the second read request.

[0184] S23. The disk processes each read request separately and returns the requested DataCell to the storage controller. Each time the storage controller receives a DataCell, it writes the DataCell into memory.

[0185] S24. The storage control unit sends the data of each DataCell to the client via a separate RDMA Write message. The RDMA Write message carries the CellClientAddr and Offset of the DataCell.

[0186] The RDMA Write message encapsulates an NVMe read response. The NVMe read response is the response to an NVMe read request, carrying the DataCell's CellClientAddr and Offset. The DataCell's ClientAddr is the starting address of the DataCell on the client. The DataCell's CellClientAddr and Offset indicate that data at Offset should be written starting from the client's CellClientAddr.

[0187] In relevant technical standards, the NVMe read response in the RDMA Write message supports carrying the target address and its corresponding offset. For any DataCell, the storage controller can use the DataCell's ClientAddr as the target address and the DataCell's Offset as the offset corresponding to the target address, and encapsulate the DataCell's CellClientAddr and Offset in the NVMe read response of the RDMA Write message according to the message format defined in the relevant technical standards. This eliminates the need to change the message format of the RDMA Write message.

[0188] S25. Each time the client obtains a DataCell from an RDMA Write message, it sends an RDMA acknowledgment (ACK) response for the DataCell to the storage controller. The RDMA ACK response is used to indicate that the DataCell has been successfully transmitted.

[0189] Among them, the RDMA ACK response is the data cell's transmission completion notification.

[0190] S26. The storage control unit receives the RDMA ACK response from DataCell.

[0191] S27. Each time the storage control device receives an RDMA ACK response for a DataCell, it sets the transmission status of the DataCell to the first transmission status to realize the transmission status of each DataCell recording the target data. After all DataCells of the target data have been successfully transmitted, the storage control device sends a ROCE SEND message to the client. The ROCE SEND message carries a notification that the target data has been successfully read.

[0192] The method provided in this application allows the storage control device to interact with the client during data reading using messages supported by NVMe over ROCE as defined in relevant technical standards. This enables the storage control device and the client to interface via NVMe over ROCE as defined in relevant technical standards. Since the general-purpose client is already capable of interfaceing with NVMe over ROCE as defined in relevant technical standards, users can read data using the general-purpose client without modification.

[0193] When the network transmission protocol of the first network is NFS over ROCE, the process of the client reading data is similar to the process of reading data when the protocol is NVMe over ROCE. The difference is that, compared to the data reading process based on NVMe over ROCE, when the network transmission protocol of the first network is NFS over ROCE, the upper-layer protocol carried by the ROCE SEND message in steps S21 and S27 is replaced by NFS instead of NVMe.

[0194] Accordingly, during the data reading process, the storage controller can interact with the client using the messages supported by NFS over ROCE as defined in the relevant technical standards, thus enabling the storage controller and the client to interface via NFS over ROCE. The general-purpose client is already capable of interfaceing with NFS over ROCE as defined in the relevant technical standards, allowing users to read data using the general-purpose client without modification.

[0195] Taking the network transmission protocol of the first network as NVMe over TCP as an example, the process of the client reading data is the same as that shown in Figure 3, and the process includes the following steps.

[0196] S31. The client sends a read request to the storage controller via NVMe over TCP, which instructs the target data of length Len to be read into ClientAddr.

[0197] This read request is the first read request, and it carries the ClientAddr and Offset of the target data.

[0198] In relevant technical standards, NVMe over TCP-based read requests support carrying the source address and the corresponding offset. The client can use the ClientAddr of the target data as the source address and the Offset of the target data as the offset corresponding to the source address, and generate a read request with the ClientAddr and Offset of the target data according to the message format defined by NVMe over TCP, without having to change the message format of the NVMe over TCP read request.

[0199] S32. The storage control device receives a read request, splits the read request into n read tasks, and sends a separate read request to the disk for each read task. The read request carries the DiskAddr and Offset of the DataCell.

[0200] Step S33 is similar to step S23, and will not be described again here.

[0201] S33. The disk processes each read request separately and returns the requested DataCell to the storage controller. Each time the storage controller receives a DataCell, it writes the DataCell into memory.

[0202] S34. The storage control unit sends the data of each DataCell to the client through a separate C2HDATA PDU message. The C2HDATA PDU message carries the CellClientAddr and Offset of the DataCell, where Offset is located in the DATA0 field of the C2HDATA PDU message.

[0203] The C2HDATA PDU message encapsulates a read response. The read response carries the DataCell's CellClientAddr and Offset, where the DataCell's CellClientAddr is the starting address of the DataCell on the client side. The DataCell's CellClientAddr and Offset indicate that data at Offset should be written starting from the client's CellClientAddr.

[0204] In relevant technical standards, C2HDATA PDU messages support carrying a read response, which includes a target address field and a DATA0 field. The target address field stores the target address, and the DATA0 field stores the offset corresponding to the target address. For any DataCell, the storage control device can use the DataCell's ClientAddr as the target address and the DataCell's Offset as the offset corresponding to the target address. Following the message format defined in the relevant technical standards, the device encapsulates the DataCell's CellClientAddr and Offset in the read response within the C2HDATA PDU message, thus eliminating the need to modify the C2HDATA PDU message's message format.

[0205] S35. Each time the client obtains a DataCell from a C2HDATA PDU message, it sends a successful dataCell transmission acknowledgment to the storage controller.

[0206] Among them, the successful transmission response is the transmission completion notification from DataCell.

[0207] S36. The storage control device receives a successful transmission response from DataCell.

[0208] S37. Each time the storage control device receives a successful transmission response for a DataCell, it sets the transmission status of the DataCell to the first transmission status to record the transmission status of each DataCell of the target data. After all DataCells of the target data have been successfully transmitted, the storage control device sends a successful response notification for the read request to the client via NVMe over TCP.

[0209] Among them, the successful response notification is the notification that the target data has been successfully read.

[0210] The method provided in this application allows the storage control device to interact with the client during data reading using packets supported by NVMe over TCP as defined by relevant technical standards. This enables the storage control device and the client to interface via NVMe over TCP. Since general-purpose clients are already capable of interfaceing with NVMe over TCP as defined by relevant technical standards, users can read data using general-purpose clients without modification.

[0211] The NFS over TCP protocol supports data transmission in a pre-order manner. When the network transport protocol is NFS over TCP, the process of the client reading data is similar to that described above when it's NVMe over TCP, with the following two differences:

[0212] Difference 1: Compared to the NVMe over TCP-based data reading process, when the network transmission protocol of the first network is NFS over TCP, the upper-layer protocol carried by the interaction information between the client and the storage controller in steps S31 to S37 is replaced by NFS instead of NVMe. The interaction information between the client and the storage controller includes read requests, data transmission of DataCells, successful data transmission acknowledgments for DataCells, and successful response notifications for read requests. The interaction information between the client and the storage controller does not carry offsets.

[0213] Difference 2: Because NVMe over TCP packets support carrying offsets, during the reading of target data based on NVMe over TCP, the order of DataCells in the target data segment is irrelevant; DataCells can be transmitted out of order or in sequence. In contrast, NFS over TCP packets do not support carrying offsets. During the reading of target data based on NVMe over TCP, the storage controller transmits DataCells in sequence. For example, when sending C2HDATA PDU messages to the client, the storage controller sends the C2HDATA PDU messages to the first network according to the order of each DataCell in the target data. For instance, it first sends the C2HDATA PDU message carrying the first DataCell of the target data, then sends the C2HDATA PDU message carrying the second DataCell of the target data, and so on. This ensures that the DataCells of the target data are submitted to the first network in sequence, so that the client can read the DataCells of the target data in order and recover the target data.

[0214] In scenarios where the network transport protocol of the first network is NFS over UDP, or where DataCells are transmitted via RDMA immediate values ​​in NFS / NVMe, or where out-of-order transmission exists in the first network, the storage control device can add an offset field to the read response carrying the DataCell during the client's data reading process. For example, as shown in Figure 6, an offset field carrying the offset of the DataCell is added after the UDP or ROCE header of the read response. This allows the client to store the DataCell at the storage location indicated by the corresponding CellClientAddr and Offset, based on the DataCell's CellClientAddr in the read response and the DataCell's Offset in the offset field.

[0215] In related technologies, a client can access a storage medium through a storage controller to write data to the storage medium. The data writing process can be as follows: the client sends a write request to the storage controller, indicating that second data should be written; after receiving the write request, the storage controller requests the client to send the second data and requests storage space for the second data from its memory. The client sends the second data to the storage controller according to the request. The storage controller caches the second data in the requested storage space. When all the second data is cached, it writes the second data in that storage space to the storage medium. After all the second data is successfully written, it releases the storage space occupied by the second data and sends a write success notification to the client.

[0216] However, during the process of the client accessing the storage medium, the storage space occupied by the first data is only released after the memory space for the second data is allocated and the second data is successfully written to the storage medium. This results in a single write request taking too long to occupy memory, and the memory cannot provide data caching services for more write requests, thus leading to low memory utilization efficiency of the storage control device.

[0217] Based on this, in this application, for a client's write request, the storage control device caches each data segment to be written in the cache space associated with the storage control device at the granularity of data segments. Each time a data segment is written to the storage medium, the storage space occupied by that data segment in the cache space is released. The process will now be described with reference to Figure 7.

[0218] Figure 7 is a flowchart of another data access method provided in an embodiment of this application. The method is applied to the storage system described above and includes the following steps.

[0219] 701. The client sends a first write request to the storage control device, which indicates that second data should be written.

[0220] The second write request is any write request from the client, while the first write request is a large I / O request. The second data is the business data written by the client in this request. The second read request carries the target address information of the second data. This target address information indicates the target storage location of the second data in the storage medium. The target address information includes a starting address C and a third target offset. The starting address C is the starting address of the second data in the storage medium, and C is a virtual address. The third target offset is the offset between the ending address of the second data in the storage medium and the starting address C.

[0221] Optionally, the first write request also carries storage address information for the second data. This storage address information indicates the storage location of the first data on the client. The storage address information includes a starting address D and a fourth target offset. The starting address D is the starting address of the second data on the client, i.e., the starting address where the client stores the second data. The fourth target offset is the offset between the ending address of the second data on the client and the starting address D. The ending address of the second data on the client side is a virtual address. The starting address D and the fourth target offset indicate the storage location of the second data on the client side. In some embodiments, the first write request may also omit the starting address D and the fourth target offset.

[0222] Optionally, the first write request may also carry the amount of data for the second data; of course, it may also not carry the amount of data for the second data.

[0223] 702. The storage control device receives the client's first write request.

[0224] Upon receiving the first write request, the storage control device determines the amount of the second data based on the first write request. For example, if the first write request carries the amount of the second data, the amount of the second data is obtained from the first write request; if the first write request does not carry the amount of the second data, the storage capacity of the storage location indicated by the target address information or the client address information of the second data is determined as the amount of the second data.

[0225] If the amount of the second data is less than or equal to the second threshold, the storage control device writes the second data at the data granularity of the second data. If the amount of the second data is greater than or equal to the second threshold, the storage control device writes each data segment of the second data at the data segment granularity (as described in steps 703 and 704 below). When all data segments of the second data have been successfully written, the first write request processing is complete.

[0226] The second threshold is the maximum data size of a data segment when processing a write request at the granularity of a data segment. The second threshold can be less than or equal to the MTU of the first network between the client and the storage controller, or it can be greater than the MTU. In this embodiment of the application, the second threshold is not limited.

[0227] For example, during the processing of a first write request, the storage control device generates at least one write task for the first write request based on a second threshold and the amount of second data. Each write task instructs the writing of a segment of the second data, the amount of which is less than or equal to the second threshold. For instance, if the amount of the second data is greater than the second threshold, the storage control device generates multiple write tasks for the first write request. If the amount of the second data is less than or equal to the first threshold, the storage control device generates one write task for the first write request; in this case, the write task instructs the writing of the second data, i.e., the second data is the segment of data instructed to be written by the write task.

[0228] The data segment indicated by the write task is called the data segment corresponding to the write task; that is, each write task corresponds to a data segment of the second data. Each write task has task information, which includes sub-target address information and / or sub-memory address information of a data segment.

[0229] The sub-target address information indicates the target storage location of the data segment on the storage medium, which is a partial storage location among the target storage locations indicated by the target address information of the second data. This sub-target address information includes the target start address of the data segment on the storage medium and the offset between the target start address and the target end address of the data segment on the storage medium. The target start address can be a virtual address or a physical address. If the start address of the data segment on the storage side is a virtual address, the end address of the data segment on the storage side is also a virtual address; if the start address of the data segment on the storage side is a physical address, the end address of the data segment on the storage side is also a physical address. The sub-storage address information indicates the storage location of the data segment on the client, which is a partial storage location among the storage locations indicated by the storage address information of the second data. This sub-storage address information includes the start address of the data segment on the client and the offset between the start address and the end address of the data segment on the client. Both the start address and the end address of the data segment on the client are virtual addresses.

[0230] For example, suppose p write tasks are generated for the first write request to logically divide the second data into p data segments, namely data segment 1 to data segment p, where p is an integer greater than 0. The task information for each write task is shown in Table 2 below.

[0231] Table 2

[0232] In some embodiments, the storage control device generates status information for each data segment. The status information for each data segment includes the read status and write status of the data segment. The read status is as described above and will not be repeated here. The write status indicates whether the data segment has been written to the storage medium. The write status includes not written and written. In some embodiments, the write status also includes writing in progress. Different write statuses can be represented by different status identifiers.

[0233] The storage device can add the status information of each data segment to the task information of the corresponding write task. Initially, the storage control device sets the read status of each data segment of the second data to "not read" and the write status of each data segment to "not written". Taking Table 1 as an example, the read status of each data segment from data segment 1 to data segment p is set to "not read", and the write status of each data segment is set to "not written".

[0234] The storage control device records the corresponding write task by storing the task information of each write task.

[0235] For each write task, the storage controller can read the corresponding data segment from the client. Upon reading each data segment of the second data, the data segment is cached in the cache space associated with the storage controller. The data segment is then written to the storage medium from the cache space. After the data segment is written, the memory occupied by the data segment in the cache space is released. For example, steps 703 or 704 below.

[0236] 703. The storage control device reads the third data segment of the second data from the client, caches the third data segment in the cache space, writes the third data segment to the second storage medium, and releases the storage space occupied by the third data segment in the cache space after the third data segment is written.

[0237] The second storage medium is the storage medium in the storage system that stores the second data. The third data segment is the data segment corresponding to any write task of the first write request. If the first write request has one write task, the third data segment is the second data; if the first write request has multiple write tasks, the third data segment is any segment of the second data. When the cache space is the memory of the storage controller, caching the third data segment to the cache space means caching the third data segment to memory; when the cache space is an on-chip cache on the storage controller, caching the third data segment to the cache space means caching the third data segment to the on-chip cache.

[0238] The storage controller can read a third data segment by sending a third read task to the client. This third read task instructs the reader to read the third data segment. The third read task can be carried out in a read request; for example, the storage controller sends a third read request to the client to read the third data segment, where the third read request carries the sub-storage address information of the first data segment. In response to the third read request, the client retrieves the third data segment from the storage location indicated by the sub-storage address information and returns the third data segment to the storage controller. The storage controller receives the third data segment returned by the client.

[0239] If the third data segment is read, the storage control device sets the read status of the third data segment in the third status information to "read". The third status information refers to the status information of the third data segment. Taking Table 2 as an example, assuming the third data segment is data segment 1, after reading data segment 1, the read status of data segment 1 in the task information of write task 1 is updated to "written".

[0240] In some embodiments, before or after the third data segment is read, the storage control device requests storage space from the cache space for the third data segment.

[0241] Taking the allocation of storage space before reading the third data segment as an example, the storage control device allocates storage space for the third data segment from the cache space. For example, storage space can be allocated for the third data segment before sending the third read request to read the third data segment, or after sending the third read request but before receiving the third data segment returned by the client.

[0242] Taking the allocation of storage space after reading the third data segment as another example, let's assume the cache space is the memory of the storage controller. After receiving the third data segment returned by the client, the storage controller first temporarily caches the third data segment in the on-chip cache of the storage controller, and then allocates storage space for the third data segment from that memory. After allocating the storage space, the storage controller then caches the third data segment from the on-chip cache into that storage space, thus achieving the goal of caching the third data segment into memory.

[0243] The storage capacity requested for the third data segment can be equal to or greater than the data size of the third data segment. For example, if the storage control device requests storage space for the third data segment from the cache space based on the data size of the third data segment, the storage capacity of the requested storage space is equal to the data size of the third data segment.

[0244] The storage control device can set the read state of the third data segment to the first read state after reading the third data segment, or it can set the read state of the third data segment to the first read state after caching the third data segment into memory. In this embodiment of the application, the timing of setting the read state of the third data segment to the first read state is not limited, as long as it is set when the third data segment is read.

[0245] After caching the third data segment into the cache space, the storage control device writes the third data segment to the second storage medium.

[0246] The storage control device can write a third data segment by sending a first write task to the client. The first write task carries the third data segment and first sub-target address information, which is the sub-target address information of the third data segment. The first write task can be a write request. For example, the storage control device sends a second write request to the second storage medium for writing the third data segment, wherein the second write request carries the third data segment and the first sub-target address information. The second storage medium receives the second write request and writes the third data segment to the storage location indicated by the first sub-target address information. After the third data segment is written, it sends a write completion notification to the storage control device. The storage control device receives the write completion notification.

[0247] After the third data segment is written, the storage control device releases the storage space occupied by the third data segment in the cache space. For example, if a write completion notification for the third data segment is received, the storage space is released; if no write completion notification for the third data segment is received, the storage space is not released temporarily, but is released after a successful transmission notification for the third data segment is received.

[0248] After the third data segment is written, the storage space occupied by the third data segment is released so that if the writing of the third data segment fails, the third data segment can be retrieved from the storage space and rewritten to the second storage medium, thus meeting the requirement of rewriting the data segment.

[0249] In related technologies, when processing the first write request, the first data is cached entirely in memory before the second data is written to the second storage medium. Furthermore, the storage space occupied by the second data in memory is released only after all the second data in memory has been successfully written. In contrast to these technologies, in this embodiment, after the third data segment is read and cached in the cache space, it is not necessary to wait for the second data to be fully cached before writing the third data segment to the second storage medium. Moreover, after the third data segment is written, the storage space occupied by the third data segment in the cache space is released without waiting for the second data to be fully written, reducing the time the third data segment occupies that storage space. This allows the storage space to provide caching services for other read and write requests as quickly as possible, thereby improving the utilization efficiency of the memory space. This enables the entire memory to provide caching services for more read and write requests, ultimately improving the overall efficiency of the storage space.

[0250] If the third data segment is successfully written, the storage control device can also set the write status of the third data segment in the third status information to "written," indicating that the third data segment has been successfully written to the storage medium. Here, the third status information refers to the status information of the third data segment. Taking Table 2 as an example, assuming the third data segment is data segment 1, after writing data segment 1, the write status of data segment 1 in the task information of write task 1 is updated to "written."

[0251] After the third data segment is written, the storage control device generates and stores the metadata of the third data segment. The method of generating the metadata is not limited.

[0252] 704. The storage control device reads the fourth data segment of the second data from the client, caches the fourth data segment in the cache space, writes the fourth data segment to the second storage medium, and releases the storage space occupied by the fourth data segment in the cache space after the fourth data segment is written.

[0253] The fourth data segment is any data segment in the second data other than the fourth data segment, and the first write request has multiple write tasks, including the write task corresponding to the fourth data segment. When the cache space is the memory of the storage controller, caching the fourth data segment to the cache space means caching the fourth data segment to memory; when the cache space is an on-chip cache on the storage controller, caching the fourth data segment to the cache space means caching the fourth data segment to the on-chip cache.

[0254] The storage controller can read the fourth data segment by sending a fourth read task to the client. This fourth read task instructs the reader to read the fourth data segment. The fourth read task can be carried out in a read request; for example, the storage controller sends a third read request to the client to read the fourth data segment, where the third read request carries sub-storage address information for the fourth data segment. In response to the third read request, the client retrieves the fourth data segment from the storage location indicated by the sub-storage address information and returns the fourth data segment to the storage controller. The storage controller receives the fourth data segment returned by the client.

[0255] In some embodiments, before or after the fourth data segment is read, the storage control device requests storage space from the cache space for the fourth data segment. This process can be referred to as the process of requesting storage space for the first data segment, and will not be described again here.

[0256] If the fourth data segment is read, the storage control device sets the read status of the fourth data segment in the fourth status information to "read," indicating that the fourth data segment has been read. Here, the fourth status information refers to the status information of the fourth data segment. Taking Table 2 as an example, assuming the fourth data segment is data segment 2, after reading data segment 2, the read status of data segment 2 in the task information of write task 2 is updated to "read."

[0257] After the fourth data segment is cached in the cache space, the storage control device writes the fourth data segment to the second storage medium.

[0258] The storage controller can write a fourth data segment by sending a second write task to the client. The second write task carries the fourth data segment and second sub-target address information, which is the sub-target address information of the fourth data segment. The second write task can be a write request. For example, the storage controller sends a second write request to the second storage medium for writing the fourth data segment, wherein the second write request carries the fourth data segment and second sub-target address information. The second storage medium receives the second write request, writes the fourth data segment to the storage location indicated by the second sub-target address information, and after the fourth data segment is written, sends a write completion notification to the storage controller. The storage controller receives the write completion notification.

[0259] If the fourth data segment is written successfully, the storage control device releases the storage space occupied by the fourth data segment in the cache space. For example, if a write completion notification for the fourth data segment is received, the storage space is released; if no write completion notification for the fourth data segment is received, the storage control device does not release the storage space temporarily, but releases it after receiving the write completion notification for the fourth data segment.

[0260] After the fourth data segment is written, the storage space occupied by the fourth data segment in memory is released so that if the writing of the fourth data segment fails, the fourth data segment can be retrieved from the storage space and rewritten to the second storage medium, thus meeting the requirement of rewriting the data segment.

[0261] If the fourth data segment is successfully written, the storage control device can also set the write status of the fourth data segment in the fourth status information to "written" to indicate that the fourth data segment has been successfully written to the storage medium. Taking Table 2 as an example, assuming the fourth data segment is data segment 2, after data segment 2 is successfully written, the transmission status of data segment 2 in the task information of write task 2 will be updated to "written".

[0262] After the fourth data segment is written, the storage control device generates and stores the metadata of the fourth data segment. The method of generating the metadata is not limited.

[0263] The above description uses the example of executing step 703 first, followed by step 704. In other embodiments, the storage control device may execute step 704 first, followed by step 703. For example, in a scenario where multiple data segments of the second data are processed sequentially, if the storage location of the fourth data segment in the second data is before the third data segment, then step 704 is executed first, followed by step 703. In other embodiments, the storage control device may execute steps 703 and 704 simultaneously. For example, in a scenario where multiple data segments of the second data are processed in parallel, the storage control device executes steps 703 and 704 simultaneously. Here, the execution order of steps 703 and 704 is not limited in this embodiment.

[0264] If the second data includes other data segments besides the third and fourth data segments, the storage control device reads these other data segments from the client, caches them in the cache space, writes them to the second storage medium, and releases the storage space occupied by these other data segments in the cache space after the writing is complete. This process is the same as step 703 or step 704 described above, and will not be repeated here.

[0265] After successfully writing each data segment of the second data to the second storage medium, the storage control device can also generate metadata for the second data based on the metadata of each data segment. After successfully writing each data segment of the second data to the second storage medium, the storage control device sends a write completion notification to the client, indicating that the second data has been written completely. For example, if the read status of each data segment of the second data is "read" and the write status of each data segment is "written," then the storage control device sends a write completion notification to the client. The write completion notification can carry the metadata of the second data so that the client can subsequently read the second data based on this metadata.

[0266] When the client and storage controller are connected via a first network, during the reading of any data segment of second data from the client, the client sends the data segment to the network interface card (NIC) connected to the first network. If the data size of the segment is greater than the MTU of the first network, the NIC first splits the data segment into multiple sub-segments based on the MTU, with each sub-segment having a data size less than or equal to the MTU. These sub-segments are then transmitted sequentially to the storage controller via the first network until all sub-segments have been successfully transmitted. If the data size of the segment is less than or equal to the MTU of the first network, the NIC does not split the data segment and transmits it to the client.

[0267] When reading multiple data segments of the second data, the client sends multiple data segments to the network interface card (NIC). The NIC transmits these segments to the storage controller sequentially according to the order of reception. If the data size of each segment is greater than the MTU of the first network, each segment needs to be split before transmission, and other segments must wait for the first segment to complete. If the data size of each segment is less than or equal to the MTU of the first network, the NIC can transmit each received segment to the storage controller immediately without splitting it. This allows the NIC to send a segment as soon as it is received, enabling a completely pipelined process from client to network transmission of the second data segments, thereby improving the processing efficiency of the first write request.

[0268] When the amount of data in the second data set is greater than or equal to the second threshold, multiple write tasks are generated for the first write request, and a second write request is generated for each of these multiple write tasks. This effectively splits the first write request into multiple second write requests. The granularity of the data written in the second write requests is smaller than that in the first write request, thus splitting a large write request from the client into multiple smaller write requests. The large write request is then completed by processing these smaller write requests. This process will be described in detail below with reference to Figure 8.

[0269] As shown in Figure 8, in step S41, the client sends a large write request (such as the first write request) to the storage control device. In step S42, the storage control device generates multiple smaller read requests (i.e., the third read request) based on the large write request, and allocates memory space for each smaller read request to occupy the allocated memory space. The storage control device also sends multiple smaller read requests to the client, and each smaller write request initiates data transmission independently. In step S43, the client processes each smaller read request individually, returning the data requested by each smaller read request (i.e., the data segment) to the storage control device. Each time the storage control device receives the data requested by a smaller read request, it writes the data to the storage space allocated for that smaller read request. In step S44, after writing the data requested by each smaller read request to the storage space, the storage control device sends a smaller write request (i.e., the second write request) to the storage medium. Each smaller write request carries the data requested by the smaller read request, thus achieving independent disk partitioning of the data requested by each smaller read request. In step S45, upon receiving each small write request, the data carried by the small write request is written to disk. After successful disk writing (i.e., write completion), the storage medium returns a disk writing success notification (i.e., write completion notification) to the storage controller. In step S46, the storage controller receives the disk writing success notification returned by the storage medium. In step S47, after all data requested by small read requests has been written, the storage controller sends a write completion notification for the large write request to the client.

[0270] The method provided in the embodiment shown in Figure 7, for a client's write request, segments the data to be written from the client into segments. Each time a data segment is read, it is cached in a cache space (which may be the memory of the storage controller or an on-chip cache on the storage controller) and then written to the storage medium. Once this data segment is written, the storage space occupied by it is released without waiting for other data segments to finish writing. This reduces the time a single data segment occupies a small portion of the cache space, thereby reducing the time the entire data requested by the client occupies a large portion of the cache space. This allows the entire cache space to provide caching services for more read and write requests, thus improving the overall utilization of the cache space. When the cache space is memory, this also improves memory utilization efficiency. When the cache space is an on-chip cache, there is no need to read or write data to memory during the write request processing, thus avoiding the memory wall problem during this process.

[0271] Regarding the multiple data segments of the second data, the above describes the allocation of separate storage space for each data segment, with each data segment occupying its own dedicated storage space. In other embodiments, the multiple data segments of the second data can reuse the same storage space.

[0272] For example, suppose p write tasks are generated for the first write request. The storage control device requests k storage spaces from the cache space for the p write tasks, where the storage capacity of each of the k storage spaces is greater than or equal to a first threshold, and k is an integer greater than 0 and less than p.

[0273] For k storage spaces, the storage control device can execute the write tasks of the first write request in batches. For example, k write tasks out of p write tasks can be considered as a batch of write tasks, and the storage control device executes this batch of write tasks first. During the execution of this batch of write tasks, the storage control device reads k data segments of the second data from the client. Each time a data segment is read, it is cached in one of the k storage spaces. This way, even if k data segments are read simultaneously, they can be cached in the respective k storage spaces. During the execution of this batch of write tasks, after each data segment is cached in a storage space, the storage control device writes the data segment from that storage space to the second storage medium. After the data segment is written, the storage space occupied by that data segment is deleted, thus releasing the storage space occupied by the data segment. After this batch of write tasks is completed, if the number of remaining write tasks is greater than k, the storage control device selects k write tasks from the remaining write tasks as a new batch of write tasks and executes the new batch of write tasks. This process continues until the number of remaining write tasks is less than or equal to k. Then, the storage control device selects the remaining write tasks as the last batch of write tasks and executes the last batch of write tasks.

[0274] For example, suppose p = 10 and k = 5, meaning the first write request has 10 write tasks and 5 storage spaces are allocated for it. For these 5 storage spaces, 5 write tasks are executed first, each corresponding to one storage space. During the execution of these 5 write tasks, each task reads a data segment as instructed by the write task, caches that segment in its corresponding storage space, and if the data segment is successfully transmitted, deletes the storage space occupied by that segment. After these 5 write tasks are completed, the remaining 5 write tasks are executed in a similar manner.

[0275] In this scenario, if the write tasks corresponding to the third and fourth data segments of the second data are in different batches—for example, the write task corresponding to the third data segment is in the first batch of write tasks, and the write task corresponding to the fourth data segment is in the second batch of write tasks—during the execution of the first batch of write tasks, after the third data segment is written, the storage space occupied by the third data segment is released. During the execution of the second batch of write tasks, after the fourth data segment is read, the storage space originally occupied by the third data segment is used to cache the fourth data segment, thus allowing the third and fourth data segments to reuse the same storage space.

[0276] For k storage spaces, the write tasks of the first write request can also be executed without batching. For example, the storage control device first executes k write tasks for the first write request. During the execution of each write task, the data segment read based on that write task is cached in one of the k storage spaces. After the data segment is written, the storage space occupied by that data segment is released, and the write task is completed. After each write task is completed, one storage space becomes free. A new write task can be selected from the remaining write tasks for execution. When the new write task is executed, the data segment read based on the new write task is cached in the free storage space. This process continues until all write tasks are completed.

[0277] For example, suppose the storage control device executes a write task corresponding to the third data segment. During execution, the third data segment is cached in one of the k storage spaces. After the third data segment is written, the storage space occupied by the third data segment is released, and the write task corresponding to the third data segment is completed. Then, the storage control device executes a write task corresponding to the fourth data segment. During execution, after reading the second data segment, the storage space originally occupied by the third data segment is used to cache the fourth data segment, thus allowing the third and fourth data segments to reuse the same storage space.

[0278] For example, suppose p = 4, k = 2, and the first write request has 4 write tasks, namely write task 1 to write task 4. Two storage spaces are allocated for the first write request, namely storage space 1 and storage space 2. The storage control device first executes write task 1 and write task 2, caching data segment 1 read based on write task 1 into storage space 1. After data segment 1 is written, storage space 1 is released, and write task 1 is completed. Then, data segment 2 read based on write task 2 is cached into storage space 2. After data segment 2 is written, storage space 2 is released, and write task 2 is completed. After write task 1 is completed, the storage control device executes write task 3, caching data segment 3 read based on write task 3 in storage space 1. After write task 2 is completed, the storage control device executes write task 4, caching data segment 4 read based on write task 4 in storage space 2.

[0279] In the embodiments of this application, multiple data segments of the second data can reuse the same storage space, thereby reducing the storage space occupied during the second data reading process and reducing memory usage.

[0280] The above explanation uses the memory or on-chip cache of the storage control device as an example to illustrate the cache space associated with the storage control device. In some embodiments, the cache space includes both the memory of the storage control device and the on-chip cache. In this case, during the writing of the second data, the on-chip cache is used preferentially to store the data segment of the second data, or the memory is used preferentially to store the data segment of the second data. Next, these two preferential storage methods will be described in conjunction with Figures 9 and 10.

[0281] Taking the priority use of on-chip cache as an example, during the reading of the second data, the on-chip cache is used first to store the read data segment. Only when the on-chip cache's storage capacity is insufficient is memory used to store the data segment. For example, each time a data segment of the second data is read, if the on-chip cache is sufficient to store the data segment, the data segment is cached in the on-chip cache and then written to the second storage medium. If the on-chip cache is insufficient to store the data segment, the storage control device caches the data segment in memory and then writes the data segment in memory to the second storage medium. Sufficient on-chip cache to store the data segment includes situations where the remaining storage capacity of the on-chip cache is greater than or equal to the storage amount of the data segment. Sufficient on-chip cache to store the data segment also includes situations where the remaining storage capacity of the on-chip cache is less than the storage amount of the data segment. Next, this implementation process will be described using Figure 9 as an example.

[0282] Figure 9 is a flowchart of a data writing method that prioritizes the use of on-chip cache according to an embodiment of this application. The method includes the following steps.

[0283] 901. The client sends a first write request to the storage control device, which indicates that second data should be written.

[0284] Step 901 is similar to step 701, and will not be described again here.

[0285] 902. The storage control device receives the client's first write request.

[0286] Step 902 is similar to step 702, and will not be described again here.

[0287] 903. The storage control device reads the third data segment of the second data from the client. If the on-chip cache on the storage control device is sufficient to store the third data segment, the third data segment is cached in the on-chip cache, and the third data segment is written to the second storage medium. After the third data segment is written, the storage space occupied by the third data segment in the on-chip cache is released.

[0288] Step 903 is similar to step 703 above, except that in step 903, the on-chip cache is used to store the third data segment first. After the third data segment is read, if the on-chip cache is insufficient to store the third data segment, the storage control device caches the third data segment in the memory of the storage control device, writes the third data segment to the second storage medium, and releases the storage space occupied by the third data segment in memory after the third data segment is written.

[0289] 904. The storage control device reads the fourth data segment of the second data from the client. If the on-chip cache on the storage control device is sufficient to store the fourth data segment, the fourth data segment is cached in the on-chip cache, and the fourth data segment is written to the second storage medium. After the fourth data segment is written, the storage space occupied by the fourth data segment in the on-chip cache is released.

[0290] Step 904 is similar to step 704 above, except that in step 904, the on-chip cache is used to store the fourth data segment first. After the fourth data segment is read, if the on-chip cache is insufficient to store the fourth data segment, the storage control device caches the fourth data segment in the memory of the storage control device, and the second storage medium writes the fourth data segment. After the fourth data segment is written, the storage space occupied by the fourth data segment in the memory is released.

[0291] The above description uses the example of executing step 903 first and then step 904. In other embodiments, the storage control device may execute step 904 first and then step 903, or execute steps 903 and 904 simultaneously. Here, the execution order of steps 903 and 904 is not limited in this application embodiment.

[0292] If the second data includes other data segments besides the third and fourth data segments, the storage control device reads these other data segments from the client. After reading the other data segments, if the on-chip cache is sufficient to store them, the storage control device caches the other data segments in the on-chip cache, writes them to the second storage medium, and releases the storage space occupied by the other data segments in the on-chip cache after the writing is complete. If the on-chip cache is insufficient to store the other data segments, the storage control device caches them in memory, writes them to the second storage medium, and releases the storage space occupied by the other data segments in memory after the writing is successful. This process is similar to steps 903 or 904 above and will not be described again here.

[0293] Once all data segments of the second data have been successfully written to the second storage medium, the storage control device sends a write completion notification to the client for the second data.

[0294] In this embodiment, during the writing of the second data, on-chip cache is prioritized to store the data segment of the second data. Only when the on-chip cache's storage capacity is insufficient is memory used to store the second data segment. This reduces the space occupied by the first write request, allowing memory to provide caching services for more write requests from the client, thus improving memory utilization efficiency. Prioritizing the use of on-chip cache to store the data to be written during write request processing reduces the number of times data is read and written from memory, thereby mitigating the memory wall problem.

[0295] Taking the priority use of memory as an example, during the writing of the second data, memory is used first to store the data segments of the second data. When the memory storage capacity is insufficient, the on-chip cache is used to store the data segments. For example, each time a data segment of the second data is read, if the memory is sufficient to store the data segment, the data segment is cached in memory and then written to the second storage medium. If the memory is insufficient to store the data segment, the storage control device caches the data segment in the on-chip cache and then writes the data segment from the on-chip cache to the second storage medium. Sufficient memory to store the data segment includes: the remaining memory storage capacity is greater than or equal to the storage amount of the data segment. Sufficient memory to store the data segment also includes: the remaining memory storage capacity is less than the storage amount of the data segment. Next, this implementation process will be described using Figure 10 as an example.

[0296] Figure 10 is a flowchart of a data writing method with priority given to memory usage, provided in an embodiment of this application. The method includes the following steps.

[0297] 1001. The client sends a first write request to the storage control device, which indicates that second data should be written.

[0298] Step 1001 is similar to step 701, and will not be repeated here.

[0299] 1002. The storage control device receives the client's first write request.

[0300] Step 1002 is similar to step 702, and will not be repeated here.

[0301] 1003. The storage control device reads the third data segment of the first data from the client. If the memory of the storage control device is sufficient to store the third data segment, the third data segment is cached in the memory and written to the second storage medium. After the third data segment is written, the storage space occupied by the third data segment in the memory is released.

[0302] Step 1003 is similar to step 703 above, except that in step 1003, the third data segment is stored in memory first. After the third data segment is read, if the memory is insufficient to store the third data segment, the storage control device caches the third data segment in the on-chip cache on the storage control device, writes the third data segment to the second storage medium, and releases the storage space occupied by the third data segment in the on-chip cache after the third data segment is written.

[0303] 1004. The storage control device reads the fourth data segment of the second data from the client. If the memory of the storage control device is sufficient to store the fourth data segment, the fourth data segment is cached in memory and written to the second storage medium. After the fourth data segment is written, the storage space occupied by the fourth data segment in memory is released.

[0304] Step 1004 is similar to step 704 above, except that in step 1004, the fourth data segment is stored in memory first. After the fourth data segment is read, if the memory is insufficient to store it, the storage control device caches the fourth data segment in the on-chip cache of the storage control device, writes the fourth data segment to the second storage medium, and releases the storage space occupied by the fourth data segment in the on-chip cache after the writing of the fourth data segment is completed.

[0305] The above description uses the example of executing step 1003 first and then step 1004. In other embodiments, the storage control device may execute step 1004 first and then step 1003, or execute steps 1003 and 1004 simultaneously. Here, the execution order of steps 1003 and 1004 is not limited in this application embodiment.

[0306] If the second data includes other data segments besides the third and fourth data segments, the storage control device reads these other data segments from the client. After reading the other data segments, if there is sufficient memory to store them, the other data segments are cached in memory and written to the second storage medium. After the other data segments are written, the storage space occupied by the other data segments in memory is released. If there is insufficient memory to store the other data segments, the other data segments are cached in an on-chip cache and written to the second storage medium. After the other data segments are written, the storage space occupied by the other data segments in the on-chip cache is released. This process is similar to steps 1003 or 1004 above and will not be described again here.

[0307] After all data segments of the second data have been written to the second storage medium, the storage control device sends a notification to the client that the writing of the second data is complete.

[0308] In this embodiment of the application, by prioritizing the use of memory to store the data segment of the second data during the writing of the second data, and caching the data segment of the second data to an on-chip cache when memory is insufficient, the data segment is prevented from waiting for memory space for a long time, thereby enabling the data segment to be written to the second storage medium as soon as possible, which can reduce the time for the storage control device to process the first write request, thereby improving the processing efficiency of the first write request.

[0309] The embodiments shown in Figures 7 to 10 above are all described using the example of the second data being stored on a single storage medium (i.e., the second storage medium). In other embodiments, the second data may also be stored on multiple storage media, all of which are second storage media. In this case, the second data can also be written according to the data writing method shown in Figures 7 to 10. The difference lies in that, during the writing process, the storage control device writes different data segments of the second data to each of these multiple storage media.

[0310] In the embodiments of the above data access methods, when the client and the storage control device are connected through the first network, the storage control device and the client interact with each other. The interacted information includes a first write request, a third read request, and a write completion notification. The interacted information can be carried by different messages supported by the network transmission protocol of the first network, including any one of NVMe over ROCE, NFS over ROCE, NVMe over TCP, or NFS over TCP / UDP. For different network transmission protocols, the messages carrying the interaction messages during the data writing process differ when the client and the storage control device interact. The data writing process is described below for each of these network transmission protocols.

[0311] Taking the first network's network transmission protocol as NVMe over ROCE as an example, the process of the client reading data is described below. This process is similar to the process shown in Figure 8, and includes the following steps.

[0312] S51. The client sends a ROCE SEND message to the storage controller. The ROCE SEND message carries an NVMe write request, which indicates that target data of length Len, starting from the client address (ClientAddr), will be written to the storage medium.

[0313] The ROCE SEND message encapsulates an NVMe write request. This is the first write request, and the target data is the second data. The NVMe write request carries the ClientAddr and Offset of the target data. The ClientAddr is the starting address (i.e., starting address D) of the target data on the client side, and the Offset is the offset of the target data relative to its own ClientAddr. The ClientAddr and Offset indicate the storage space starting at the offset from the ClientAddr.

[0314] In relevant technical standards, NVMe write requests in ROCE SEND messages support carrying the source address and the corresponding offset. The client can use the ClientAddr of the target data as the source address and the Offset of the target data as the offset of the source address, and encapsulate the ClientAddr and Offset of the target data in the NVMe write request in the ROCE SEND message according to the message format defined by the relevant technical standards, so that the message format of the ROCE SEND message does not need to be changed.

[0315] S52. The storage control device receives the ROCE SEND message, splits the NVMe write request into p write tasks, and sends a separate read request to the client for each write task. The read request carries the DiskAddr and Offset of the DataCell.

[0316] Here, p represents the number of data cells (DataCells) of the target data, and a DataCell represents a data segment of the target data. Each write task indicates the reading of one DataCell of the target data. The storage control unit also records the starting address of each DataCell and the offset of each DataCell relative to its own starting address. The offset of a DataCell relative to its own starting address is called the DataCell's Offset. DiskAddr is the starting address of the DataCell on the disk. The DiskAddr and Offset of the DataCell indicate the storage space of the DataCell on the client side. This read request is the third read request.

[0317] S53. The client processes each read request separately and returns an RDMA response message corresponding to each read request to the storage controller. The RDMA response message carries the DataCell requested by the corresponding read request.

[0318] S54. Each time the storage control device receives an RDMA Response message, it writes the DataCell in the RDMA Response message into memory, and performs an independent disk write operation on the DataCell according to the recorded DataCell Offset, so as to write the DataCell into the storage medium.

[0319] S55. The storage control unit receives the disk dismantling success notification from the storage medium for each DataCell.

[0320] Among them, the successful write notification is the write completion notification for DataCell.

[0321] S56. The storage control unit receives the RDMA ACK response from DataCell.

[0322] S57. Each time the storage control device receives a successful write notification for a DataCell, it sets the write status of the DataCell to the first write status to realize the write status of each DataCell recording the target data. After all DataCells of the target data have been successfully written, the storage control device sends a ROCE SEND message to the client. The ROCE SEND message carries a notification that the write of the target data is complete.

[0323] The method provided in this application allows the storage controller to interact with the client during data writing using messages supported by NVMe over ROCE as defined in relevant technical standards. This enables the storage controller and the client to interface via NVMe over ROCE. Since the general-purpose client is already capable of interfacing with NVMe over ROCE as defined in relevant technical standards, users can write data using the general-purpose client without modification.

[0324] When the network transmission protocol of the first network is NFS over ROCE, the process of the client writing data is similar to the process of writing data when the protocol is NVMe over ROCE. The difference is that, compared to the data reading process based on NVMe over ROCE, when the network transmission protocol of the first network is NFS over ROCE, the upper-layer protocol carried by the ROCE SEND message in steps S51 and S57 is replaced by NFS instead of NVMe.

[0325] Accordingly, during the data writing process, the storage controller can interact with the client using the messages supported by NFS over ROCE as defined in the relevant technical standards, thus enabling the storage controller and the client to interface via NFS over ROCE. The general-purpose client is already capable of interfaceing with NFS over ROCE as defined in the relevant technical standards, allowing users to write data using the general-purpose client without modification.

[0326] Taking the NVMe over TCP network transmission protocol of the first network as an example, the process of the client writing data is described below. This process is similar to the process shown in Figure 8, and includes the following steps.

[0327] S61. The client sends a write request to the storage controller via NVMe over TCP, which instructs the target data of length Len to be written to the storage medium.

[0328] This write request is the first write request, and it carries the ClientAddr and Offset of the target data.

[0329] In relevant technical standards, NVMe over TCP-based read requests support carrying the source address and the corresponding offset. The client can use the ClientAddr of the target data as the source address and the Offset of the target data as the offset of the source address, and generate a write request with the ClientAddr and Offset of the target data according to the message format defined by NVMe over TCP, without having to change the message format of the NVMe over TCP write request.

[0330] S62. The storage controller receives the ROCE SEND message, splits the NVMe write request into p write tasks, and sends an R2T PDU message to the client for each write task. The R2T PDU message carries the DiskAddr and Offset of the DataCell. The Offset of the DataCell is located in the R2T0 field of the R2T PDU message.

[0331] The R2T PDU message is a message encapsulating a read request. The read request carries the CellClientAddr and Offset of the DataCell.

[0332] In relevant technical standards, R2T PDU messages support carrying read requests, which include a target address field and an R2T0 field. The target address field stores the target address, and the R2T0 field stores the offset corresponding to the target address. For any DataCell, the storage control device can use the DataCell's ClientAddr as the target address and the DataCell's Offset as the offset corresponding to the target address. Following the message format defined in the relevant technical standards, the device encapsulates the DataCell's CellClientAddr and Offset into a read request within the R2T PDU message, without altering the message format.

[0333] S63. The client processes each read request separately and returns an H2CDATA PDU message corresponding to each read request to the storage controller. The H2CDATA PDU message carries the DataCell requested by the corresponding read request and the Offset of the DataCell. The Offset of the DataCell is located in the DATA0 field of the H2CDATA PDU message.

[0334] In relevant technical standards, the H2CDATA PDU message includes a DATA0 field. The destination address field stores the destination address, and the DATA0 field stores the offset of the transmitted data. For any DataCell, the storage control device can use that DataCell as the transmitted data, and according to the message format of the H2CDATA PDU message defined in the relevant technical standards, encapsulate the DataCell and its offset in the H2CDATA PDU message to request a read, without changing the message format of the H2CDATA PDU message.

[0335] S64. Each time the storage control device receives an H2CDATA PDU message, it writes the DataCell in the H2CDATA PDU message into memory, and performs an independent disk write operation on the DataCell according to the recorded DataCell Offset, so as to write the DataCell into the storage medium.

[0336] S65. Each time the client obtains a DataCell from an H2CDATA PDU message, it sends a successful dataCell transmission acknowledgment to the storage controller.

[0337] Among them, the successful transmission response is the transmission completion notification from DataCell.

[0338] S66. The storage control unit receives the disk dismantling success notification from the storage medium for each DataCell.

[0339] S67. Each time the storage controller receives a successful write notification for a DataCell, it sets the write status of the DataCell to the first write status to record the write status of each DataCell containing the target data. After all DataCells containing the target data have been successfully written, the storage controller sends a write completion notification for the target data to the client via NVMe over TCP.

[0340] The method provided in this application allows the storage control device to interact with the client during data writing using packets supported by NVMe over TCP as defined by relevant technical standards. This enables the storage control device and the client to interface via NVMe over TCP. Since general-purpose clients are already capable of interfacing with NVMe over TCP as defined by relevant technical standards, users can write data using general-purpose clients without modification.

[0341] NFS over TCP supports data transmission in an order-preserving manner. When the network transmission protocol is NFS over TCP, the process of writing data by the client is similar to the process of writing data when it is NVMe over TCP. The differences are as follows:

[0342] Difference 1: Compared to the NVMe over TCP-based write / read process, when the network transmission protocol of the first network is NFS over TCP, the upper layer carried by the interaction information between the client and the storage controller in steps S61 to S67 above is replaced by NFS instead of NVMe. The interaction information between the client and the storage controller does not carry offsets.

[0343] Difference 2: Because NVMe over TCP packets support carrying offsets, the order of DataCells is irrelevant during NVMe over TCP data cell reading; DataCells can be transmitted out of order or in sequence. In contrast, NFS over TCP packets do not support carrying offsets. During NVMe over TCP data cell reading, the storage controller reads DataCells in sequence. For example, when sending H2CDATA PDU messages to the client, the storage controller sends the messages to the first network according to the order of each DataCell in the target data. For instance, it might first send an H2CDATA PDU message requesting the first DataCell of the target data, then send an H2CDATA PDU message requesting the second DataCell, and so on. This allows the client to return the target data's DataCells to the storage controller in sequence, facilitating the storage controller's recovery of the target data.

[0344] In scenarios where the network transport protocol of the first network is NFS over UDP, or in scenarios where DataCells are transmitted via RDMA immediate values ​​in NFS / NVMe, or in scenarios where out-of-order transmission exists in the first network, the storage controller can also add an offset field to the read request during the client's data writing process. For example, as shown in Figure 7, an offset field carrying the offset of the DataCell is added to the UDP or ROCE header of the read response. This allows the client to return the corresponding DataCell to the storage controller based on the DataCell offset in the read request.

[0345] The methods of the embodiments of this application have been described above, and the apparatus of the embodiments of this application will be described below. It should be understood that the apparatus described below has any of the functions of the storage control device in the above-described methods. The data access method provided by the embodiments of this application has been described in detail above with reference to Figures 2 to 10. Based on the same inventive concept, the storage control apparatus according to the embodiments of this application will be described below with reference to Figures 11 and 12. It should be understood that the technical features described in the method embodiments are also applicable to the following apparatus embodiments.

[0346] Figure 11 is a schematic diagram of a storage control device provided in an embodiment of this application. The storage control device 1100 shown in Figure 11 can be the storage control device involved in the previous embodiments, used to execute the data access method executed by the storage control device. As shown in Figure 11, the storage control device 1100 includes:

[0347] The receiving module 1101 is used to receive a read request from the client, the read request indicating that the first data should be read;

[0348] The first processing module 1102 is used to read the first data segment of the first data from the storage medium, cache the first data segment in the cache space, return the first data segment to the client, and release the storage space occupied by the first data segment in the cache space after returning the first data segment to the client.

[0349] The first processing module 1102 is further configured to read the second data segment of the first data from the storage medium, cache the second data segment in the cache space, return the second data segment to the client, and release the storage space occupied by the second data segment in the cache space after returning the second data segment to the client.

[0350] Optionally, the first processing module 1102 is further configured to:

[0351] Send the first read task to the storage medium, receive the first data segment returned by the storage medium, and the first read task indicates that the first data segment should be read.

[0352] Send a second read task to the storage medium, receive the second data segment returned by the storage medium, and the second read task indicates that the second data segment should be read.

[0353] Optionally, the data size of the first data segment is less than or equal to the maximum transmission unit of the network between the storage controller and the client, and the data size of the second data segment is less than or equal to the maximum transmission unit.

[0354] Optionally, the read request carries the target address information of the first data, the target address information indicating the target storage location of the first data on the client; the first processing module 1102 is further configured to:

[0355] Send a first read response to the client. The first read response carries a first data segment and first sub-target address information. The first sub-target address information indicates a portion of the storage location in the target storage location.

[0356] Send a second read response to the client. The second read response carries a second data segment and a second sub-target address information, which indicates a portion of the storage location in the target storage location.

[0357] Optionally, the network protocol used between the client and the storage controller 1100 may be any one of NVMe based on ROCE, a network file system based on ROCE, or NVMe based on the Transmission Control Protocol, wherein NVMe is the Non-Volatile Memory Host Controller Interface Specification, and ROCE is Remote Direct Memory Access based on Converged Ethernet.

[0358] Optionally, the read request carries metadata of the first data, and the first processing module 1102 is further configured to:

[0359] Based on metadata, determine the storage location of the first data in the storage medium.

[0360] Optionally, the cache space includes the memory of the storage control device 1100, and the first processing module 1102 is further used for:

[0361] Before or after the first data segment is read, allocate storage space for the first data segment from memory;

[0362] Before or after the second data segment is read, allocate storage space for the second data segment from memory.

[0363] Optionally, the first processing module 1102 is further configured to: after the storage space occupied by the first data segment is released, use the storage space originally occupied by the first data segment to cache the second data segment.

[0364] Optionally, the first processing module 1102 is also used for any of the following:

[0365] The first data segment is cached in the memory of the storage control device 1100; or,

[0366] The first data segment is cached in the on-chip cache of the storage controller 1100; or,

[0367] If the on-chip cache is sufficient to store the first data segment, cache the first data segment in the on-chip cache; if the on-chip cache is insufficient to store the first data segment, cache the first data segment in memory; or...

[0368] If there is enough memory to store the first data segment, cache the first data segment in memory; if there is not enough memory to store the first data segment, cache the first data segment in the on-chip cache.

[0369] Optionally, the first processing module 1102 is also used for any of the following:

[0370] The second data segment is cached in the memory of the storage control device 1100; or,

[0371] The second data segment is cached in the on-chip cache of the storage controller 1100; or,

[0372] If the on-chip cache is sufficient to store the second data segment, cache the second data segment in the on-chip cache; if the on-chip cache is insufficient to store the second data segment, cache the second data segment in memory; or,

[0373] If there is enough memory to store the second data segment, cache the second data segment in memory; if there is not enough memory to store the second data segment, cache the second data segment in the on-chip cache.

[0374] Optionally, the first processing module 1102 is further configured to:

[0375] The storage control device generates first status information and second status information. The first status information indicates the read status and transmission status of the first data segment, and the second status information indicates the read status and transmission status of the second data segment. The read status indicates whether the data segment has been read, and the transmission status indicates whether the data segment has been transmitted to the client.

[0376] If the first data segment is read, the storage control device sets the read status of the first data segment to read in the first status information; if the first data segment is successfully transmitted, the transmission status of the first data segment is set to transmitted in the first status information.

[0377] If the second data segment is read, the storage control device sets the read status of the second data segment to read in the second status information. If the second data segment is successfully transmitted, the transmission status of the second data segment is set to transmitted in the second status information.

[0378] Figure 12 is a schematic diagram of a storage control device provided in an embodiment of this application. The storage control device 1200 shown in Figure 12 can be the storage control device involved in the previous embodiments, used to execute the data access method executed by the storage control device. As shown in Figure 12, the storage control device 1200 includes:

[0379] The receiving module 1201 is used to receive a write request from the client, which indicates that second data should be written.

[0380] The second processing module 1202 is used to read the third data segment of the second data from the client, cache the third data segment in the cache space, write the third data segment to the storage medium, and release the storage space occupied by the third data segment in the cache space after the third data segment is written.

[0381] The second processing module 1202 is also used to read the fourth data segment of the second data from the client, cache the fourth data segment in the cache space, write the fourth data segment to the storage medium, and release the storage space occupied by the fourth data segment in the cache space after the fourth data segment is written.

[0382] Optionally, the second processing module 1202 is also used for:

[0383] Send a third read task to the client, receive the third data segment returned by the client, and the third read task indicates that the third data segment should be read.

[0384] Send a fourth read task to the client, receive the fourth data segment returned by the client, and the fourth read task indicates that the fourth data segment should be read.

[0385] Optionally, the write request carries the target address information of the second data, the target address information indicating the target storage location of the second data on the storage medium; the second processing module 1202 is further configured to:

[0386] Send a first write task to the storage medium. The first write task carries a third data segment and first sub-target address information. The first sub-target address information indicates a portion of the storage location in the target storage location.

[0387] A second write task is sent to the storage medium. The second write task carries a fourth data segment and second sub-target address information, which indicates a portion of the storage location in the target storage location.

[0388] Optionally, the network protocol used between the client and the storage controller 1200 may be any one of NVMe based on ROCE, a network file system based on ROCE, or NVMe based on the Transmission Control Protocol, wherein NVMe is the Non-Volatile Memory Host Controller Interface Specification, and ROCE is Remote Direct Memory Access based on Converged Ethernet.

[0389] Optionally, the data size of the third data segment is less than or equal to the maximum transmission unit of the network between the storage controller and the client, and the data size of the fourth data segment is less than or equal to the maximum transmission unit.

[0390] Optionally, the second processing module 1202 is also used for:

[0391] After the third data segment is written, the metadata for the third data segment is generated;

[0392] After the fourth data segment is written, the metadata for the fourth data segment is generated.

[0393] Optionally, the cache space includes the memory of the storage control device 1200, and the second processing module 1202 is further used for:

[0394] Before or after the third data segment is read, allocate storage space for the third data segment from memory;

[0395] Before or after reading the fourth data segment, allocate storage space for the fourth data segment from memory.

[0396] Optionally, the second processing module 1202 is further configured to: after the storage space occupied by the third data segment is released, use the storage space originally occupied by the third data segment to cache the fourth data segment.

[0397] Optionally, the second processing module 1202 is also used for any of the following:

[0398] The third data segment is cached in the memory of the storage control unit 1200; or,

[0399] Cache the third data segment into the on-chip cache of the storage controller 1200; or,

[0400] If the on-chip cache is sufficient to store the third data segment, cache the third data segment in the on-chip cache; if the on-chip cache is insufficient to store the third data segment, cache the third data segment in memory; or,

[0401] If there is enough memory to store the third data segment, cache the third data segment in memory; if there is not enough memory to store the third data segment, cache the third data segment in the on-chip cache.

[0402] Optionally, the second processing module 1202 is also used for any of the following:

[0403] The fourth data segment is cached in the memory of the storage control unit 1200; or,

[0404] Cache the fourth data segment into the on-chip cache of the storage controller 1200; or,

[0405] If the on-chip cache is sufficient to store the fourth data segment, cache the fourth data segment in the on-chip cache; if the on-chip cache is insufficient to store the fourth data segment, cache the fourth data segment in memory; or,

[0406] If there is enough memory to store the fourth data segment, cache the fourth data segment in memory; if there is not enough memory to store the fourth data segment, cache the fourth data segment in the on-chip cache.

[0407] Optionally, the second processing module 1202 is also used for:

[0408] The storage control device generates third status information and fourth status information. The third status information indicates the read status and write status of the third data segment, and the fourth status information indicates the read status and write status of the fourth data segment. The read status indicates whether the data segment has been read, and the write status indicates whether the data segment has been written to the storage medium.

[0409] If the third data segment is read, the storage control device sets the read status of the third data segment to read in the third status information; if the third data segment is successfully written, the write status of the third data segment is set to written in the third status information.

[0410] If the fourth data segment is read, the storage control device sets the read status of the fourth data segment to read in the fourth status information. If the fourth data segment is successfully written, the write status of the fourth data segment is set to written in the fourth status information.

[0411] It should be understood that storage control devices 1100 and 1200 correspond to the storage control devices in the above method embodiments. The modules in storage control devices 1100 and 1200 and the other operations and / or functions described above are respectively for implementing various steps and methods implemented by the storage control devices in the method embodiments. For specific details, please refer to the above method embodiments. For the sake of brevity, they will not be repeated here.

[0412] It should be understood that the data access operations performed by storage control devices 1100 and 1200 are only illustrated using the above-described division of functional modules. In practical applications, the functions described above can be assigned to different functional modules as needed, that is, the internal structure of storage control devices 1100 and 1200 can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the storage control devices 1100 and 1200 provided in the above embodiments belong to the same concept as the method embodiments described above, and their specific implementation process is detailed in the above method embodiments, and will not be repeated here.

[0413] Figure 13 is a schematic diagram of a storage control device provided in an embodiment of this application. It should be understood that the device described below can implement any function of the storage control device in any of the above methods.

[0414] As shown in Figure 13, the storage control device 1300 includes a bus 1302, a processor 1304, a memory 1306, a cache 1308, and a communication interface 1310. The processor 1304, memory 1306, cache 1308, and communication interface 1310 communicate with each other via the bus 1302. The bus 1302 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 13, but this does not imply that there is only one bus or one type of bus. The bus 1302 can include pathways for transmitting information between the various components of the storage control device 1300.

[0415] Processor 1304 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1304 may be implemented using at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), programmable logic array (PLA), and application-specific integrated circuit (ASIC). Processor 1301 may also include a main processor and a coprocessor. The main processor is used to process data in the wake-up state, also known as the central processing unit (CPU); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1304 may integrate a graphics processing unit (GPU), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1304 may also include an artificial intelligence (AI) processor, which is used to handle computational operations related to machine learning, such as a neural network processing unit (NPU) or a tensor processing unit (TPU), but the type of AI processor is not limited to these.

[0416] In some embodiments, the on-chip cache in the processor 1304 can be used to provide cache space so that the processor 1304 caches data segments into the cache space during data read and write operations.

[0417] The memory 1306 can serve as the main memory or secondary storage of the storage control device 1300. The memory 1306 can be a non-transitory memory, which can be volatile memory, non-volatile memory, or a combination of both. Volatile memory, for example, is random access memory (RAM). Non-volatile memory, for example, can be read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD).

[0418] The memory 1306 stores executable program code, which the processor 1304 reads and executes, enabling the storage control device 1300 to implement the aforementioned data access method. In some embodiments, the memory 1306 may be used to provide a cache space so that the processor 1304 caches data segments in the cache space during data read and write operations.

[0419] Cache 1308 provides cache space so that during data read and write operations, processor 1304 caches data segments into cache 1308. Cache 1308 can be the memory of storage control device 1300, for example, cache 1308 includes non-transient memory and / or volatile memory. Cache 1308 can also be an on-chip cache, such as a graphics card cache, network card cache, data processing unit (DPU) cache, etc. Cache 1308 is an optional component; in some embodiments, storage control device 1300 does not include cache 1308.

[0420] The communication interface 1310 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the storage control device 1000 and other devices or communication networks.

[0421] This application also provides a chip that can be applied to the aforementioned memory control device. The chip includes logic circuitry for executing the aforementioned data access method. In one possible implementation, the chip further includes a power supply circuit coupled to the logic circuitry. The functional circuitry provides power to the logic circuitry so that it can execute the aforementioned data access method. Exemplarily, the logic circuitry is composed of a DSP, FPGA, PLA, ASIC, programmable logic device (PLD), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0422] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including program code that can be executed by a processor of a storage control device to perform the data access method described above. For example, the computer-readable storage medium is a non-transitory computer-readable storage medium, such as a read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device.

[0423] This application also provides a computer program product or computer program, which includes program code. The computer instructions are stored in a computer-readable storage medium. The processor of the storage control device reads the program code from the computer-readable storage medium and executes the program code, causing the storage control device to perform the above-described data access method.

[0424] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the data access methods in the above-described method embodiments.

[0425] In this embodiment, the apparatus, device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0426] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0427] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0428] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the data involved in this application was obtained with full authorization.

[0429] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0430] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A data access method, characterized in that, The method includes: The storage control device receives a read request from the client, the read request indicating that first data should be read; The storage control device reads a first data segment of the first data from the storage medium, caches the first data segment in the cache space, returns the first data segment to the client, and releases the storage space occupied by the first data segment in the cache space after returning the first data segment to the client. The storage control device reads a second data segment of the first data from the storage medium, caches the second data segment in the cache space, returns the second data segment to the client, and releases the storage space occupied by the second data segment in the cache space after returning the second data segment to the client.

2. The method according to claim 1, characterized in that, The first data segment that the storage control device reads from the storage medium for the first data includes: The storage control device sends a first read task to the storage medium and receives the first data segment returned by the storage medium. The first read task indicates that the first data segment should be read. The storage control device reads a second data segment of the first data from the storage medium, including: The storage control device sends a second read task to the storage medium and receives the second data segment returned by the storage medium. The second read task instructs the reading of the second data segment.

3. The method according to claim 1 or 2, characterized in that, The data size of the first data segment is less than or equal to the maximum transmission unit of the network between the storage control device and the client, and the data size of the second data segment is less than or equal to the maximum transmission unit.

4. The method according to claim 1 or 2, characterized in that, The read request carries the target address information of the first data, and the target address information indicates the target storage location of the first data on the client. Returning the first data segment to the client includes: Send a first read response to the client. The first read response carries the first data segment and first sub-target address information. The first sub-target address information indicates a portion of the storage location in the target storage location. Returning the second data segment to the client includes: A second read response is sent to the client. The second read response carries a second data segment and second sub-target address information, which indicates a portion of the storage location in the target storage location.

5. The method according to claim 4, characterized in that, The network protocol used between the client and the storage control device is any one of NVMe based on ROCE, a network file system based on ROCE, or NVMe based on the Transmission Control Protocol, wherein NVMe is the Non-Volatile Memory Host Controller Interface Specification, and ROCE is Remote Direct Memory Access based on Converged Ethernet.

6. The method according to claim 1 or 2, characterized in that, The read request carries metadata of the first data. After the storage control device receives the read request from the client, the method further includes: The storage control device determines the storage location of the first data in the storage medium based on the metadata.

7. The method according to claim 1 or 2, characterized in that, The cache space includes the memory of the storage control device, and the method further includes: Before or after the first data segment is read, the storage control device requests storage space from the memory for the first data segment; Before or after the second data segment is read, the storage control device requests storage space from the memory for the second data segment.

8. The method according to claim 1 or 2, characterized in that, The step of caching the second data segment into the cache space includes: After the storage space occupied by the first data segment is released, the storage space originally occupied by the first data segment is used to cache the second data segment.

9. The method according to claim 1 or 2, characterized in that, The step of caching the first data segment into the cache space includes any one of the following: Cache the first data segment in the memory of the storage control device; or Cache the first data segment into the on-chip cache of the storage control device; or... If the on-chip cache is sufficient to store the first data segment, the first data segment is cached in the on-chip cache; if the on-chip cache is insufficient to store the first data segment, the first data segment is cached in the memory; or, If the memory is sufficient to store the first data segment, the first data segment is cached in the memory; if the memory is insufficient to store the first data segment, the first data segment is cached in the on-chip cache.

10. The method according to claim 1 or 2, characterized in that, After the storage control device receives a read request from the client, the method further includes: The storage control device generates first status information and second status information. The first status information indicates the read status and transmission status of the first data segment, and the second status information indicates the read status and transmission status of the second data segment. The read status indicates whether the data segment has been read, and the transmission status indicates whether the data segment has been transmitted to the client. If the first data segment is read, the storage control device sets the read status of the first data segment to read in the first status information; if the first data segment is successfully transmitted, the transmission status of the first data segment is set to transmitted in the first status information. If the second data segment is read, the storage control device sets the read status of the second data segment to read in the second status information; if the second data segment is successfully transmitted, the transmission status of the second data segment is set to transmitted in the second status information.

11. A data access method, characterized in that, The method includes: The storage control device receives a write request from the client, the write request indicating that second data should be written; The storage control device reads the third data segment of the second data from the client, caches the third data segment in the cache space, writes the third data segment to the storage medium, and releases the storage space occupied by the third data segment in the cache space after the third data segment is written. The storage control device reads the fourth data segment of the second data from the client, caches the fourth data segment in the cache space, writes the fourth data segment to the storage medium, and releases the storage space occupied by the fourth data segment in the cache space after the fourth data segment is written.

12. The method according to claim 11, characterized in that, The third data segment that the storage control device reads from the client includes: The storage control device sends a third read task to the client and receives the third data segment returned by the client. The third read task indicates that the third data segment should be read. The fourth data segment that the storage control device reads from the client includes: The storage control device sends a fourth read task to the client and receives the fourth data segment returned by the client. The fourth read task instructs the reading of the fourth data segment.

13. The method according to claim 11 or 12, characterized in that, The write request carries the target address information of the second data, and the target address information indicates the target storage location of the second data on the storage medium; The step of writing the third data segment to the storage medium includes: A first write task is sent to the storage medium, the first write task carrying the third data segment and first sub-target address information, the first sub-target address information indicating a portion of the target storage location; Writing the fourth data segment to the storage medium includes: A second write task is sent to the storage medium. The second write task carries the fourth data segment and second sub-target address information, which indicates a portion of the storage location in the target storage location.

14. The method according to claim 11 or 12, characterized in that, The data volume of the third data segment is less than or equal to the maximum transmission unit of the network between the storage control device and the client, and the data volume of the fourth data segment is less than or equal to the maximum transmission unit.

15. The method according to claim 11 or 12, characterized in that, The method further includes: After the third data segment is written, the storage control device generates the metadata of the third data segment; After the fourth data segment is written, the storage control device generates metadata for the fourth data segment.

16. The method according to claim 11 or 12, characterized in that, The cache space includes the memory of the storage control device, and the method further includes: Before or after the third data segment is read, the storage control device requests storage space from the memory for the third data segment; Before or after the fourth data segment is read, the storage control device requests storage space from the memory for the fourth data segment.

17. The method according to claim 11 or 12, characterized in that, The step of caching the fourth data segment into the cache space includes: After the storage space occupied by the third data segment is released, the storage space originally occupied by the third data segment is used to cache the fourth data segment.

18. The method according to claim 11 or 12, characterized in that, The step of caching the third data segment into the cache space includes any one of the following: Cache the third data segment in the memory of the storage control device; or, Cache the third data segment into the on-chip cache of the storage control device; or, If the on-chip cache is sufficient to store the third data segment, the third data segment is cached in the on-chip cache; if the on-chip cache is insufficient to store the third data segment, the third data segment is cached in the memory; or, If the memory is sufficient to store the third data segment, the third data segment is cached in the memory; if the memory is insufficient to store the third data segment, the third data segment is cached in the on-chip cache.

19. A storage control device, characterized in that, The device includes: A receiving module is used to receive a read request from a client, the read request indicating the reading of first data; The first processing module is configured to read a first data segment of the first data from the storage medium, cache the first data segment in the cache space, return the first data segment to the client, and release the storage space occupied by the first data segment in the cache space after returning the first data segment to the client. The first processing module is further configured to read a second data segment of the first data from the storage medium, cache the second data segment in the cache space, return the second data segment to the client, and release the storage space occupied by the second data segment in the cache space after returning the second data segment to the client.

20. The apparatus according to claim 19, characterized in that, The first processing module is also used for: Send a first read task to the storage medium, receive the first data segment returned by the storage medium, and the first read task indicates reading the first data segment; A second read task is sent to the storage medium, and the second data segment is received from the storage medium. The second read task indicates that the second data segment should be read.

21. The apparatus according to claim 19 or 20, characterized in that, The data size of the first data segment is less than or equal to the maximum transmission unit of the network between the device and the client, and the data size of the second data segment is less than or equal to the maximum transmission unit.

22. A storage control device, characterized in that, The device includes: A receiving module is used to receive a write request from a client, the write request indicating that second data should be written; The second processing module is used to read the third data segment of the second data from the client, cache the third data segment in the cache space, write the third data segment to the storage medium, and release the storage space occupied by the third data segment in the cache space after the third data segment is written. The second processing module is further configured to read the fourth data segment of the second data from the client, cache the fourth data segment in the cache space, write the fourth data segment to the storage medium, and release the storage space occupied by the fourth data segment in the cache space after the fourth data segment is written.

23. The apparatus according to claim 22, characterized in that, The second processing module is also used for: Send a third read task to the client, receive the third data segment returned by the client, and the third read task indicates that the third data segment should be read; A fourth read task is sent to the client, and the fourth data segment returned by the client is received. The fourth read task indicates that the fourth data segment should be read.

24. The apparatus according to claim 22 or 23, characterized in that, The data volume of the third data segment is less than or equal to the maximum transmission unit of the network between the device and the client, and the data volume of the fourth data segment is less than or equal to the maximum transmission unit.

25. A storage control device, characterized in that, The storage control device includes a processor for executing program code that causes the storage control device to perform the method as described in any one of claims 1 to 18.

26. A computer-readable storage medium, characterized in that, The storage medium stores at least one piece of program code, which is read by a processor to cause the storage control device to perform the method as described in any one of claims 1 to 18.

Images