Data access method and apparatus, and device

Abstract

A data access method and apparatus, and a device, which are applied to the field of communications. In the present application, a data exchange device is interposed between a data access device and a storage cluster, and information exchanged between the data access device and the storage cluster needs to pass through the data exchange device. The data exchange device receives a data access request from the data access device. The data access request is used for requesting to access target data in the storage cluster. For example, the data access request is used for requesting to read the target data from the storage cluster or write the target data into the storage cluster. On the basis of a first storage node to which the target data of the data access request belongs, the data exchange device accesses the target data in the first storage node. The data exchange device replaces the storage cluster to process the data access request, and the data access request does not need to reach the storage cluster, thereby preventing the data access request from bypassing the storage cluster, and improving the efficiency of processing the data access request.

Description

A data access method, apparatus, and device

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411433709.0, filed on October 14, 2024, entitled “A Data Access Method, Apparatus and Device”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of communication technology, and in particular to a data access method, apparatus, and device. Background Technology

[0004] A distributed storage system comprises multiple storage nodes, with data distributed across these nodes. Each storage node manages the data it stores. Storage tasks on each node can be offloaded to its data processing unit (DPU), which handles the data access requests received by that node. In this storage cluster architecture, when a storage node's DPU receives a data access request, if the requested data belongs to another storage node, the DPU will forward the request to that other storage node. The DPU on that other storage node then receives and processes the data access request.

[0005] In this type of storage cluster architecture, the storage node receiving a data access request may not be the same storage node to which the requested data belongs. Therefore, data access requests may be routed (i.e., a data access request is forwarded from one storage node to another), which can affect the processing efficiency of data access requests. Summary of the Invention

[0006] This application provides a data access method, apparatus, and device to improve the processing efficiency of data access requests.

[0007] In a first aspect, embodiments of this application provide a data access method, which can be executed by a data exchange device located between the data access device and a storage cluster. Information exchanged between the data access device and the storage cluster must pass through the data exchange device. The data exchange device can receive data access requests from the data access device, which are used to request access to target data in the storage cluster, such as requesting to read target data from or write target data to the storage cluster.

[0008] Upon receiving the data access request, the data exchange device determines the ownership node of the target data based on the request. The ownership node refers to the storage node that stores the target data or the storage node that needs to store the target data. Here, we take the example where the target data belongs to the first storage node in the storage cluster; the data exchange device can access the target data in the first storage node.

[0009] Using the above method, the data exchange device processes the data access request instead of the storage cluster. The data access request does not need to reach the storage cluster, thus avoiding the data access request from detouring through the storage cluster and improving the processing efficiency of the data access request.

[0010] In one possible implementation, the data exchange device is connected to the storage cluster via a system bus. For example, the data exchange device can be connected to the memory (such as RAM) in each storage node of the storage cluster via the system bus. When the data exchange device accesses target data in the first storage node, it can access the target data in the first storage node via the system bus. In the embodiments of this application, the system bus can also be simply referred to as the bus.

[0011] Using the methods described above, data access based on the system bus becomes faster and more efficient, further improving the processing efficiency of data access requests.

[0012] In one possible implementation, before receiving a data access request, the data access device can establish a connection with the storage cluster. The connection establishment process is as follows: The data exchange device receives a connection establishment request from the data access request, which requests to establish a connection with the storage cluster. The data exchange device detects the destination address carried in the connection establishment request. If the destination address carried in the connection establishment request is the address of the storage cluster, the data exchange device generates and sends back a positive connection establishment response. This positive connection establishment response is a positive response to the connection establishment request, indicating that the connection establishment is permitted.

[0013] Using the above method, the data exchange device can handle connection establishment requests in place of the storage nodes in the storage cluster. The storage nodes no longer need to handle the connection establishment requests, which reduces the computing power occupied by the storage nodes.

[0014] In one possible implementation, the data exchange device can also perform operations on target metadata in the storage cluster.

[0015] Through the above methods, the data exchange device has the function of metadata operation, which can assist the storage nodes in the storage cluster to manage metadata and further release the computing power of the storage memory.

[0016] In one possible implementation, there are many scenarios in which the data exchange device performs operations on target metadata in the storage cluster. For example, the data access device can send a metadata operation request to perform operations on target metadata in the storage cluster, and the data exchange device receives the metadata operation request and performs the operations on the target metadata according to the request. As another example, during the process of writing data to the storage cluster, the data exchange device can generate metadata for the data to be written and can also update the metadata of that data in the storage cluster.

[0017] The above method enables data exchange devices to perform operations on target metadata in the storage cluster in a wide range of scenarios, effectively expanding the applicability of this data access method.

[0018] In one possible implementation, the embodiments of this application do not limit the type of operation performed on the target metadata. For example, the operation may be some or all of the following: modification operation, query operation, read operation, and generation operation.

[0019] Using the above methods, the data access device can perform various operations on the target metadata according to the actual situation, making its application scenarios more extensive.

[0020] In one possible implementation, the data exchange device can store metadata for all or a portion of the data in the storage cluster. For example, the data exchange device stores target metadata, and when performing operations on the target metadata in the storage cluster, it also performs operations on the target metadata cached in the data exchange device. When target metadata needs to be read, the data exchange device retrieves it from its local storage or obtains and caches it from the storage cluster. When modification is required, the data exchange device modifies the target metadata within the data exchange device and can also update the modified target metadata in the storage cluster. When target metadata needs to be generated, the data exchange device can update the temporarily cached target metadata within the data exchange device and can also update the target metadata in the storage cluster.

[0021] By using the above method, since the data exchange device can store the target metadata in the storage cluster, the data exchange device can perform operations on the target metadata in a short time, thereby improving the execution efficiency of metadata operations.

[0022] In one possible implementation, the data exchange device includes a switching chip and a processing chip. The switching chip receives data access requests. The switching chip can detect the destination address carried in the data access request. When the destination address is detected to be an address of the storage cluster, the switching chip can send the data access request to the processing chip; alternatively, the switching chip can send the data access request to the processing chip without detecting the destination address in the data access request.

[0023] Using the above method, the data exchange device has two built-in chips with different functions, which work together to process data access requests.

[0024] In one possible implementation, when the processing chip accesses target data in the first storage node, it can obtain the storage address of the target data. The processing chip transmits the storage address of the target data to the switching chip, and sending this storage address to the switching chip can trigger the switching chip to access the target data in the first storage node. After receiving the storage address of the target data, the switching chip accesses the target data in the first storage node based on the storage address.

[0025] Using the above method, the processing chip only needs to obtain the storage address of the target data and does not need to access the first storage node. Instead, the switching chip accesses the target data in the first storage node, which saves the processing chip's computing power and enables the processing chip to handle more data access requests.

[0026] In one possible implementation, the switching chip connects to the storage cluster via a system bus. The switching chip is also connected to each storage node in the storage cluster via the system bus, such as connecting the switching chip to the memory within each storage node via the system bus. When the switching chip accesses target data in the first storage node based on the storage address of the target data, it accesses the target data in the first storage node via the system bus and based on the storage address of the target data.

[0027] Using the above method, the switching chip can quickly access the target data in the first storage node via the system bus.

[0028] In one possible implementation, when the switching chip accesses the target data in the first storage node via the system bus and based on the storage address of the target data, it can read the target data from the storage address of the target data in the first storage node via the system bus; it can also write the target data to the storage address of the target data in the first storage node via the system bus.

[0029] In one possible implementation, when the switching chip sends a data access request to the processing chip, it can remove the Layer 2 header from the data access request and transmit the data access request with the Layer 2 header removed to the processing chip.

[0030] By employing the above method, the data access request, which eliminates the layer 2 header, is relatively small, and the transmission of this data access request between the switching chip and the processing chip does not require a large amount of bandwidth. Furthermore, this data access request transmission method allows the switching chip and processing chip to simultaneously transmit multiple data access requests, further improving the processing efficiency of the data switching device.

[0031] In one possible implementation, after receiving a data access request, the processing chip can generate a data access response header; the processing chip then transmits the data access response header to the switching chip.

[0032] The switching chip encapsulates the data access response based on the data access response header and then sends back the data access response.

[0033] Using the above method, the processing chip generates the header of the data access response, the switching chip is responsible for encapsulating the data access response, and the processing chip and the switching chip work together to quickly send the data access response back to the data access device.

[0034] In one possible implementation, a data access request is used to read target data, and the payload of the data access response includes the target data. Since the switching chip can access the target data of the first storage node, it can directly obtain the payload of the data access response to ensure the encapsulation efficiency of the data access response.

[0035] In one possible implementation, a data access request is used to write target data, the interaction chip can obtain the payload of the data access response, the switching chip obtains the payload from the processing chip, and the payload indicates whether the target data writing was successful or failed.

[0036] Using the above method, the processing chip provides the payload of the data access response to the switching chip, enabling the switching chip to quickly encapsulate the data access response and promptly feed it back to the data access device.

[0037] Secondly, embodiments of this application also provide a data exchange apparatus that has the function of implementing the behavior in the method example of the first aspect described above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the data exchange apparatus includes a transmission module and a processing module, and optionally, a data module. The transmission module and the data module are deployed on a switching chip in the data exchange device, and the processing module is deployed on a processing chip in the data exchange device. These modules can perform the corresponding functions in the method example of the first aspect described above, as detailed in the method example description and will not be repeated here.

[0038] Thirdly, embodiments of this application provide a data exchange device, the beneficial effects of which can be found in the relevant description of the first aspect, and will not be repeated here. The data exchange device includes a switching chip and a processing chip.

[0039] The switching chip is used to receive data access requests, which are used to request access to target data in the storage cluster.

[0040] The processing chip is used to: determine the first storage node in the storage cluster to which the target data belongs based on the data access request, and access the target data in the first storage node.

[0041] In one possible implementation, the data exchange device is connected to the storage cluster via a system bus, and the processing chip triggers the exchange chip to access the target data in the first storage node through the system bus.

[0042] In one possible implementation, the switching chip receives a connection establishment request, which is used to request a connection with the storage cluster, and determines that the destination address carried in the connection establishment request is the address of the storage cluster.

[0043] The processing chip generates and establishes a positive response through a feedback connection with the switching chip.

[0044] In one possible implementation, the processing chip performs operations on the target metadata in the storage cluster.

[0045] In one possible implementation, before the processing chip performs operations on the target metadata in the storage cluster, the switching chip receives a metadata operation request for performing operations on the target metadata in the storage cluster.

[0046] In one possible implementation, the operations performed on the target metadata are some or all of the following:

[0047] Modify, query, read, and generate operations.

[0048] In one possible implementation, the processing chip performs operations on the target metadata cached in the data exchange device.

[0049] In one possible implementation, the switching chip sends a data access request to the processing chip. The switching chip can send the data access request to the processing chip if it detects that the destination address carried in the data access request is the address of the storage cluster, or it can send the data access request directly to the processing chip without checking the destination address carried in the data access request.

[0050] In one possible implementation, the processing chip obtains the storage address of the target data and transmits the storage address of the target data to the switching chip. The switching chip then accesses the target data in the first storage node based on the storage address of the target data.

[0051] In one possible implementation, the switching chip is connected to the storage cluster via a system bus. For example, the switching chip is connected to each storage node in the storage cluster via the system bus, and the switching chip accesses the target data in the first storage node via the system bus and based on the storage address of the target data.

[0052] In one possible implementation, the switching chip can read target data from the storage address of the target data in the first storage node via the system bus; it can also write target data to the storage address of the target data in the first storage node via the system bus.

[0053] In one possible implementation, the switching chip removes the Layer 2 header from the data access request and transmits the data access request with the Layer 2 header removed to the processing chip.

[0054] In one possible implementation, the processing chip generates the header of the data access response; the processing chip transmits the header of the data access response to the switching chip.

[0055] The switching chip obtains the header of the data access response from the processing chip. The data access response is a response to the data access request. The switching chip obtains the payload of the data access response (such as reading the target data as the payload or obtaining the payload from the processing chip).

[0056] The switching chip encapsulates the data access response based on the message header and payload, and then sends back the data access response.

[0057] In one possible implementation, the data access request is used to read the target data, and the payload of the data access response includes the target data.

[0058] In one possible implementation, a data access request is used to write target data, and the interaction chip obtains a payload from the processing chip, which indicates whether the target data writing was successful or failed.

[0059] Fourthly, this application provides a computer-readable storage medium that, when executed by a computing device, allows the computing device to perform the methods provided in the first aspect or any possible implementation thereof. The storage medium stores computer program instructions. The storage medium includes, but is not limited to, volatile memory, such as random access memory, and non-volatile memory, such as flash memory, hard disk drive (HDD), and solid-state drive (SSD).

[0060] Fifthly, this application provides a computer device program product, which includes computer program instructions. When executed by a computing device, the computing device performs the methods provided in the first aspect or any possible implementation thereof. The computer program product can be a software installation package, and when it is necessary to use the methods provided in the first aspect or any possible implementation thereof, the computer program product can be downloaded and executed on the computing device.

[0061] In a sixth aspect, this application also provides a computer chip connected to a memory, the chip being used to read and execute computer program instructions stored in the memory, and to execute the methods provided in the first aspect or any possible implementation thereof. Attached Figure Description

[0062] Figure 1 is a schematic diagram of an interaction based on shared memory provided in an embodiment of this application;

[0063] Figure 2 is a schematic diagram of the structure of a data access system provided in an embodiment of this application;

[0064] Figure 3 is a schematic diagram of the structure of a data exchange device provided in an embodiment of this application;

[0065] Figures 4A and 4B are schematic diagrams illustrating the deployment of a data exchange device according to an embodiment of this application;

[0066] Figure 5 is a schematic diagram of a data access method provided in an embodiment of this application;

[0067] Figure 6A is a flowchart illustrating the processing of a connection establishment request according to an embodiment of this application;

[0068] Figure 6B is a flowchart illustrating the processing of metadata operation requests according to an embodiment of this application;

[0069] Figure 6C is a flowchart illustrating the processing of a data read request according to an embodiment of this application;

[0070] Figure 6D is a flowchart of a data write request processing method provided in an embodiment of this application;

[0071] Figure 7 is a schematic diagram of the structure of a data exchange device provided in an embodiment of this application. Detailed Implementation

[0072] Before introducing the data access method, apparatus, and device provided in the embodiments of this application, the following concepts involved in the embodiments of this application shall be clarified:

[0073] (1) Metadata.

[0074] Metadata, also known as intermediary data or relay data, is data about data. It mainly describes information about data properties, such as the storage address of the data, the modification history of the data, the size of the data, and the creation date of the data.

[0075] (2) Message.

[0076] In this embodiment of the application, the messages exchanged between the data access device and the data exchange device exist in the form of messages. The messages exchanged between the data access device and the data exchange device include, but are not limited to: connection establishment request, connection establishment positive response, connection establishment negative response, metadata operation request, metadata operation response, data access request, and data access response (such as data write response and data read response).

[0077] A message consists of a header and a payload. The header contains forwarding information for the message. According to a layered model of communication protocols, such as the OSI model, the first layer is the physical layer, the second is the data link layer, the third is the network layer, and the fourth is the transport layer. Correspondingly, the message header includes a layer 2 header, a layer 3 header, and a layer 4 header. Taking the TCP / IP protocol as an example, the layer 2 header is the MAC header, corresponding to the data link layer. The layer 2 header records information such as the destination media access control address (MAC) and the source MAC address. The layer 3 header is the IP header, corresponding to the network layer. The layer 3 header records information such as the source internet protocol (IP) address and the destination IP address. The layer 4 header is either the TCP header or the UDP header, corresponding to the transport layer, and records information such as the destination port and the source port.

[0078] The payload is the content of the message. It includes the information the message needs to transmit. Taking a data access request as an example, for a data access request to read data, the payload records the type of data access (i.e., data read), and may also record the address of the data to be read, the data identifier, etc. For a data access request to write data, the payload records the type of data access request (i.e., data write), and may also record the data to be written, the address of that data, etc.

[0079] (3) File system.

[0080] A file system is a structured form of data storage and organization. It organizes data using the concept of "files," grouping data for the same purpose into different types of files according to the structural requirements of different applications. Different file extensions are typically used to indicate different file types, and each file is assigned an easy-to-remember name, or "filename." When there are many files, they are grouped according to a certain method, with each group of files placed in the same directory (or folder). Furthermore, directories can contain subdirectories (or subfolders) in addition to files, forming a tree-like structure. This tree-like structure has a specific name: file system. There are many types of file systems, common ones being Windows' FAT / FAT32 / NTFS, and Linux's EXT2 / EXT3 / EXT4 / XFS / BtrFS, etc. To facilitate searching, directories are listed level by level from the root node down to the file itself. The names of these directories, subdirectories, and files are concatenated using special characters (e.g., "\" in Windows / DOS, " / " in Unix-like systems). This string of characters is called a file path, such as " / etc / systemd / system.conf" in Linux or "C:\Windows\System32\taskmgr.exe" in Windows. A path is a unique identifier for accessing a specific file. For example, D:\data\file.exe in Windows is a file path, representing the file.exe file located in the data directory on the D partition.

[0081] The file system is built on block devices. The file system not only records the file path, but also records which blocks make up a file and which blocks record directory / subdirectory information.

[0082] (4) Object storage.

[0083] Object storage uses a flat address space to store data, without hierarchical directories and files. An object can include data, metadata (such as size, date, owner, etc.), and other attributes (such as access characteristics). Each object has a unique identifier (ID), called the object ID. The object ID is generated using a specialized algorithm (such as a hash value of the data) to ensure that each object's ID is unique. In object storage, objects are stored in buckets, and new buckets can be created when the storage space in a bucket is insufficient.

[0084] (5) Shared memory.

[0085] Typically, two components within the same device can interact via shared memory, which refers to a memory space accessible to both components. These two components are internal data processing units within the device, such as processors, offloading cards, chips, or chip systems. For ease of explanation, these two components will be referred to as the first processing unit and the second processing unit, respectively. The shared memory can be located within either the first or second processing unit, meaning it is part of the memory of either the first or second processing unit. Alternatively, the shared memory can be located in memory outside of the first and second processing units.

[0086] When the first processing unit and the second data unit interact via shared memory, the first processing unit can store the request to be sent to the second processing unit in the shared memory and notify the second processing unit to retrieve the request from the shared memory. Upon receiving the notification from the first processing unit, the second processing unit accesses the shared memory and retrieves the message. Similarly, the second processing unit can transmit the processing result for the request to the first processing unit in a similar manner.

[0087] The following section introduces an interaction method based on shared memory, still using the first processing unit and the second processing unit as examples.

[0088] As shown in Figure 1, this shared memory-based interaction method involves three basic elements: a descriptor array (description, desc), a receive ring buffer (RX ring buffer), and a transmit ring buffer (TX ring buffer).

[0089] The descriptor array contains multiple descriptors, each of which can record the address in the buffer. The descriptor array records the identifier of each descriptor and the content described by each descriptor, such as the address of the buffer.

[0090] The receive ring buffer can be understood as a ring-shaped structure comprising multiple slots, each capable of holding a descriptor. This receive ring buffer indicates the descriptors currently available on the first processing unit side. Similarly, the transmit ring buffer can also be understood as a ring-shaped structure. The transmit ring buffer indicates which descriptors have been used on the second processing unit side.

[0091] When the first processing unit needs to send a request to the second processing unit, it stores the request in a buffer. When writing the request to the buffer, the first processing unit finds an available descriptor from the descriptor group, adds the address of the request in the buffer to the descriptor, and adds the descriptor to the transmit ring buffer. The first processing unit notifies the second processing unit that there is a request to be processed. After receiving the notification from the first processing unit, the second processing unit finds the descriptor in the transmit ring buffer, retrieves the address of the request in the buffer from the descriptor, and reads the request based on that address. After reading the request, the second processing unit processes it. The second processing unit can write the processing result of the request to the buffer, that is, write the processing result of the request to the buffer, and update the descriptor recording the address of the processing structure of the request in the receive ring buffer.

[0092] The second processing unit notifies the first processing unit that the request has been processed. After receiving the notification from the second processing unit, the first processing unit finds the descriptor in the receive ring buffer, obtains the address of the processing result of the request in the buffer from the descriptor, and reads the processing result of the request based on the address.

[0093] Furthermore, when the second processing unit retrieves descriptors from the send ring buffer, it can do so in batches, that is, determine the descriptors of multiple buffers containing requests at once. Then, it reads and processes the multiple requests from these multiple buffers.

[0094] As described above, the receiver ring buffer records requests initiated by the first processing unit to the second processing unit. The second processing unit can maintain a head pointer, which indicates the descriptor it needs to read next. That is, after each read request, the second processing unit only points the head pointer to the slot following the currently read descriptor in the receiver ring buffer. This way, when the second processing unit needs to read again, it can read the descriptor from the position indicated by the head pointer. The first processing unit can maintain a tail pointer, which indicates the last descriptor added by the first processing unit in the receiver ring buffer. That is, after each write request, the first processing unit only points the tail pointer to the slot following the currently added descriptor in the receiver ring buffer. This way, when the first processing unit needs to write again, it can add the descriptor from the position indicated by the tail pointer.

[0095] The purpose of the send ring buffer is to record the requests processed by the second processing unit. The first processing unit maintains a head pointer for the receive ring buffer, which indicates the descriptor to be read next. The second processing unit can also maintain a tail pointer, which indicates the descriptor most recently written to it.

[0096] It should be understood that the aforementioned transmit buffer or receive buffer can be in the form of a ring structure or other structures (e.g., a queue structure or a chain structure, etc.), and this application does not limit it in this regard.

[0097] In this embodiment, the switching chip and processing chip within the data exchange device can interact via shared memory. The processing chip can be understood as the aforementioned first processing unit, and the switching chip can be understood as the aforementioned second processing unit. The shared memory can be located inside the processing chip or in a memory accessible to both the switching chip and the processing chip.

[0098] (6) Remote direct memory access (RDMA).

[0099] RDMA is a technique that bypasses the operating system kernel of a remote device to access its data in memory. Because it does not go through the operating system, it not only saves a lot of processor resources, but also improves system throughput and reduces system network communication latency. It is especially suitable for widespread use in large-scale parallel computer clusters.

[0100] RDMA has several key features: (1) Data is transmitted between the network and remote devices; (2) No operating system kernel is involved, and all content related to sending and transmitting is offloaded to the smart network card; (3) Data is transmitted directly between the user space virtual memory and the smart network card without involving the operating system kernel, and there is no additional data movement or copying.

[0101] RDMA transmission modes include both bilateral and unilateral operations. SEND / RECEIVE is a bilateral operation, meaning the remote application needs to be aware of and participate in the transmission to complete the message. READ and WRITE are unilateral operations, requiring only the local end to specify the source and destination addresses. The remote application doesn't need to be aware of the communication; data reading or storage is handled by the remote network interface card (NIC), which then encapsulates the data into a message and returns it to the local end. In practice, SEND / RECEIVE is often used to transmit control messages, while data messages are mostly transmitted using READ / WRITE.

[0102] Here, the two ends that need to exchange information are referred to as the first device and the second device, respectively. The first device and the second device can interact via RDMA. Two methods for the first device and the second device to interact via RDMA are listed below.

[0103] In the first scenario, the first device reads data from the second device via bilateral RDMA.

[0104] In a bilateral RDMA read operation, the first device does not know the location of the target data in the second device's memory. Therefore, the request initiated by the first device to read the target data does not carry the target data's location information. Instead, the request indicates the identifier of the data set (such as a file, object, or data block) to which the target data belongs, its offset, and the length of the target data. Upon receiving this message, the second device's processor queries the target data's location information and returns it to the first device. The first device then initiates another request to read the target data, this time including the target data's location information. The second device's network interface card (NIC) reads and returns the target data based on this location information.

[0105] The second method involves the first device writing data to the second device via bilateral RDMA.

[0106] The first device is unaware of the target data's location in the second device's memory. Therefore, the request initiated by the first device to write data does not include the target data's location information. Instead, the request indicates the identifier of the data set (such as a file, object, or data block) to which the target data belongs, its offset, and the length of the target data. Upon receiving this request, the second device's processor queries the target data's location information and returns it to the first device. The first device then initiates another request to the second device to write the target data, this time including the target data's location information and the target data itself. The second device's network interface card (NIC) saves the target data based on its location information.

[0107] Figure 2 shows a schematic diagram of a data access system provided in an embodiment of this application. The data access system includes a data access device 100, a data exchange device 200, and a storage cluster 300, which includes multiple storage devices.

[0108] Data access device 100 has communication capabilities and can access storage cluster 300 through data exchange device 200 upon user triggering. Data access device 100 can initiate a connection establishment request to storage cluster 300 through data exchange device 200 to request a connection with storage nodes 310 in storage cluster 300. After establishing a connection with storage cluster 300, data access device 100 can also access data or metadata in storage cluster 300 through data exchange device 200. Data access device 100 can initiate metadata operation requests to request operations on metadata in storage cluster 300; data access device 100 can also initiate data access requests to request access to data in storage cluster 300, such as reading or writing data in storage cluster 300.

[0109] This application does not limit the deployment location of the data access device 100 or the scenario in which the data access device 100 accesses the storage cluster 300. For example, the data access device 100 may be deployed close to the user, allowing the user to directly operate the device. After detecting the user's operation, the data access device 100 establishes a connection with the storage cluster 300 through the data exchange device 200 and accesses the data in the storage cluster 300 through this connection. Alternatively, the data access device 100 may be deployed far from the user, allowing the device to establish a connection with the user's computing device and receive requests from the user's computing device requesting access to data in the storage cluster 300. The data access device then accesses the storage cluster 300 through the data exchange device 200 based on these requests. Furthermore, the data access device 100 may spontaneously access the storage cluster 300 through the data exchange device 200. When the data access device 100 has stored a certain amount of data locally, it can migrate the stored data to the storage cluster 300. The data access device 100 establishes a connection with the storage cluster 300 through the data exchange device 200, and transmits the data stored locally by the data access device 100 to the storage cluster 300 through this connection.

[0110] This application does not limit the number of data access devices 100 or their specific form; Figure 2 only illustrates one data access device 100 as an example. The data access device 100 can be a hardware device, such as a terminal device, server, chip, or chip system including multiple chips. The data access device 100 can also be a software module, such as a virtual machine or container computing instance, and can be deployed on hardware devices such as mobile terminals or servers.

[0111] The data exchange device 200 is deployed between the data access device 100 and the storage cluster 300. Messages exchanged between the data access device 100 and the storage cluster 300 (such as connection establishment requests, data access requests, metadata operation requests, etc.) need to pass through the data exchange device 200.

[0112] The data exchange device 200 communicates with the data access device 100 via a communication protocol, receiving messages from the data access device 100 (such as connection establishment requests, metadata operation requests, and data access requests) or sending messages back to the data access device 100 (such as connection establishment affirmative acknowledgment, connection establishment negative acknowledgment, metadata operation response, and data access response). This communication protocol can be Transmission Control Protocol / Internet Protocol (TCP / IP) or RDMA, etc., and the specific type of this communication protocol is not limited in this embodiment. Given the interaction method between the data exchange device 200 and the data access device 100, the data access device 100 can be deployed in a location far from the data exchange device 200, improving the deployment flexibility of the data access device 100.

[0113] The data exchange device 200 can be connected to each storage node 310 in the storage cluster 300 via a system bus. This system bus includes, but is not limited to, the Peripheral Component Interconnect Express (PCIe) bus, Compute Express Link (CXL) bus, Universal Serial Bus (USB) protocol, and Unified Bus (UB). Because the data exchange device 200 is connected to each storage node 310 in the storage cluster 300 via the system bus, the data exchange device 200 can read or write data from or to each storage node 310 in the storage cluster 300 more quickly.

[0114] The embodiments of this application do not limit the number of data exchange devices 200; only one data exchange device 200 is shown as an example in FIG2.

[0115] In this embodiment of the application, the data exchange device 200 has the following functions:

[0116] Function 1: Connection establishment function.

[0117] When data access device 100 requests to establish a connection with storage cluster 300 through data exchange device 200, data exchange device 200 can decide whether data access device 100 can establish a connection with storage cluster 300. In other words, data exchange device 200 is the "decision-maker." After receiving a connection establishment request from data access device 100, data exchange device 200 determines whether to establish a connection based on the request, and can also provide a positive or negative response. For example, data exchange device 200 can detect the information carried in the connection establishment request and determine whether to establish a connection based on the detection result.

[0118] In this embodiment, since the data exchange device 200 has a connection establishment function, the data exchange device 200 can represent the storage cluster 300 and complete the connection establishment with the data exchange device 200, without the storage cluster 300 needing to participate in the connection establishment process.

[0119] Function 2: Metadata operation function.

[0120] The data exchange device 200 can store metadata from the storage cluster 300. The data exchange device 200 can store all or part of the metadata from the storage cluster 300 (e.g., only metadata accessed more frequently than a frequency threshold is retained, or metadata of data written recently is retained). When the data exchange device 200 receives a metadata operation request from the data access device 100, the data exchange device 200 can process the metadata operation request and perform operations on the metadata according to the request.

[0121] The embodiments of this application do not limit the specific type of operation performed on the metadata. The operation types include, but are not limited to: querying metadata, reading metadata, and modifying metadata.

[0122] In this embodiment, the data exchange device 200 possesses metadata operation capabilities. This data exchange device 200 can act as the "front end" of the storage cluster 300, responding to metadata operation requests. The metadata operation requests do not actually need to reach the storage cluster 300; instead, they are processed by the data exchange device 200. The storage cluster 300 only needs to have storage functionality, effectively reducing the computing power requirements of the storage cluster 300.

[0123] Function 3: Data access function.

[0124] The data exchange device 200 acts as an intermediary between the data access device 100 and the storage cluster 300. Data access requests sent by the data access device 100 first reach the data exchange device 200. In this embodiment, the data exchange device 200 can process data access requests and complete data reading or writing based on the data access requests.

[0125] When a data access request is used to request the writing of target data in storage cluster 300, data exchange device 200 obtains the storage address of the target data and writes the target data in storage cluster 300 (storage node 310) according to the storage address of the target data.

[0126] When a data access request is made to request the reading of data in storage cluster 300, data exchange device 200 obtains the target data from storage cluster 300 and feeds the target data back to data access device 100.

[0127] In this embodiment, the data exchange device 200 can replace the storage node 310 in the storage cluster 300 to handle the data access request and complete the reading and writing of data. On the one hand, the processing of the data access request no longer occupies the computing power of the storage node 310; on the other hand, the data exchange device 200 can complete the reading and writing of data through the system bus, thereby improving the processing efficiency of the data access request.

[0128] The data switching device 200 in this embodiment can also be called a network forwarding device, forwarding device, intermediate node, switching node, etc., and is a device with data switching (forwarding) and data processing functions. The data switching device 200 can exist in the form of a switch, router, gateway, etc. It can also exist in the form of other devices or equipment with data switching and data processing functions, or as components of these devices (such as chips or chip systems). This embodiment is not limited in this respect; any device with the above functions can be used as the data switching device 200.

[0129] It should be noted that, unlike traditional devices with data exchange capabilities (such as switches, routers, and gateways), the data exchange device 200 in this embodiment does not simply perform data exchange. The data exchange device 200 can parse the address information in received messages (such as metadata operation requests, data access requests, and connection establishment requests) to determine whether the message is destined for the storage cluster 300. If it is determined that the message is destined for the storage cluster 300, the data exchange device 200 can process the message on behalf of the storage nodes 310 in the storage cluster 300.

[0130] The storage cluster 300 includes multiple storage nodes 310. Each storage node 310 has data storage capabilities and stores data and its metadata. The storage node 310 is connected to the data exchange device 200 via a system bus.

[0131] In this embodiment of the application, the storage cluster 300 may provide one or more IP addresses to the outside world, and these one or more IP addresses can be regarded as the IP addresses of the storage cluster 300. If the destination address of the message (such as connection establishment request, metadata operation request, data access request) sent by the data access device 100 is the IP address of the storage cluster 300, then the message can be considered as a message that needs to be sent to the storage cluster 300.

[0132] For storage node 310 in storage cluster 300, storage node 310 may include components such as processor, network interface card (NIC), and storage (e.g., RAM, hard disk). Since data exchange device 200 can handle connection establishment requests, data access requests, and metadata operation requests on behalf of storage node 310 in storage cluster 300, and can also respond to or acknowledge these requests, the computational power consumption of storage node 310 is reduced. Therefore, the performance requirements for the processor deployed in storage node 310 are lower; that is, there is no need to deploy a high-performance processor in storage node 310. Of course, in some possible application scenarios, the deployment of a processor in storage node 310 can be omitted, and a NIC with data processing capabilities (e.g., a smart NIC) can be deployed instead. This smart NIC can interact with data exchange device 200 and manage the data in storage node 310.

[0133] As described in the description of the data exchange device 200, the data exchange device 200 can respond to messages from the data access device 100 on behalf of the storage nodes 310 in the storage cluster 300. Messages from the data access device 100 do not need to directly reach the storage cluster 300, nor do they require the coordinated processing of multiple storage nodes 310 within the storage cluster 300, effectively reducing the computing power requirements on the storage nodes 310 in the storage cluster 300. Furthermore, since the data exchange device 200 is connected to each storage node 310 in the storage cluster 300 via a system bus, the interaction efficiency between the data exchange device 200 and the storage nodes 310 is higher, further improving the processing efficiency of the data exchange device 200 for messages from the data access device 100, enabling a faster response to messages from the data access device 100.

[0134] The structure of the data exchange device 200 will be described below. Figure 3 shows a schematic diagram of the structure of a data exchange device 200 provided in an embodiment of this application. The data exchange device 200 includes two different types of chips: a switching chip 210 and a processing chip 220.

[0135] The switching chip 210 has a data switching function and can communicate with the data access device 100 or the storage node 310 in the storage cluster 300.

[0136] For the data access device 100, the switching chip 210 supports a communication protocol and can interact with the data access device 100 based on the communication protocol, such as receiving messages from the data access device 100 (such as connection establishment request, metadata operation request, data access request) or sending messages back to the data access device 100 (such as connection establishment positive response, connection establishment negative response, metadata operation response, data access response).

[0137] For storage nodes 310 in storage cluster 300, the switching chip 210 can access storage nodes 310 in storage cluster 300 via the system bus and read metadata or data from storage nodes 310. For example, the switching chip 210 can access storage nodes 310 in storage cluster 300 via direct memory access (DMA) to read metadata or data.

[0138] Inside the data exchange device 200, the switching chip 210 and the processing chip 220 are connected via a system bus. The switching chip 210 and the processing chip 220 interact with each other. The switching chip 210 can transmit messages from the data access device 100 to the processing chip 220, such as metadata operation requests and data access requests. For example, the switching chip 210 can obtain the message header and / or message payload constructed by the processing chip 220. The switching chip 210 encapsulates the message based on the header and / or payload obtained from the processing chip 220 and sends the encapsulated message back to the data access device 100. The encapsulated message may be a connection establishment response (e.g., a positive connection establishment response, a negative connection establishment response), a data access response (e.g., a data read response, a data write response), or a metadata operation response. When encapsulating the message, the switching chip 210 can encapsulate other headers. For example, the switching chip 210 can obtain the Layer 3 and Layer 4 headers of the message from the processing chip 220, and it can also encapsulate a Layer 2 header. It is worth noting that when the switching chip 210 obtains the payload from the processing chip 220, it can obtain the payload or its storage address. If the switching chip 210 directly obtains the payload from the processing chip 220, then the switching chip 210 can perform message encapsulation. If the switching chip 210 obtains the storage address of the payload from the processing chip 220, the switching chip 210 can obtain the payload from the storage node 310 through DMA based on the storage address of the payload.

[0139] Inside the data exchange device 200, the switching chip 210 and the processing chip 220 can interact based on shared memory. For the interaction method based on shared memory, please refer to the aforementioned description of shared memory, which will not be repeated here.

[0140] The processing chip 220 has data access and metadata operation functions, enabling it to access the storage cluster 300. There are two ways the processing chip 220 accesses the storage cluster 300: One is that the processing chip 220 interacts with the storage nodes 310 in the storage cluster 300 to obtain metadata and / or data from or read metadata and / or data from the storage nodes 310. For example, the processing chip 220 can access the storage cluster 300 based on DMA. The other way is that the processing chip 220 triggers the switching chip 210 to access the storage nodes 310 in the storage cluster 300. For example, the processing chip 220 transmits the storage address of the target data to the switching chip 210 to trigger the exchange information to access the storage nodes 310 in the storage cluster 300, reading the target data from or writing the target data to the storage nodes 310.

[0141] The processing chip 220 can also store metadata of the data in the storage cluster 300. When the processing chip 220 needs to perform operations on the metadata (such as obtaining a metadata operation request from the switching chip 210), it can retrieve the metadata from the locally stored metadata and perform operations on the metadata; it can also retrieve the metadata from the storage cluster 300 and perform operations on the metadata.

[0142] When data needs to be read (e.g., a data access request obtained from the switching chip 210 is a data read request), the processing chip 220 determines the storage address of the target data based on the data access request and informs the switching chip 210 of the storage address of the target data, so as to trigger the switching chip to obtain the target data based on the storage address of the target data.

[0143] When data needs to be written (e.g., a data access request obtained from the switching chip 210 is a data write request), the processing chip 220 determines the storage address of the target data based on the data write request. After determining the storage address of the target data, the processing chip 220 can inform the switching chip 210 of the storage address of the target data to trigger the switching chip to store the data based on the storage address of the target data, or it can store the target data according to the storage address of the target data.

[0144] The processing chip 220 also has message processing capabilities. It contains a protocol stack and can process messages (such as connection establishment requests, data access requests, and metadata operation requests) transmitted by the switching chip 210 based on this stack. The protocol stack can be understood as a hierarchical structure of protocol layers that work together to enable data communication with the data access device 100. For example, the protocol stack within the processing chip 220 can be based on RDMA or TCP / IP. The processing chip 220 can parse messages transmitted by the switching chip 210 based on the protocol stack, such as obtaining the message payload and parsing the data within it. The switching chip 210 can also provide feedback on messages transmitted by itself based on the protocol stack, i.e., generating the header and payload of a feedback message. The feedback message is a response to the messages transmitted by the switching chip 210. When generating the header of the feedback message, the processing chip 220 can generate both a Layer 3 header and a Layer 4 header. When generating the payload of the feedback message, the processing chip 220 can either generate the payload directly or generate the storage address of the payload. After generating the header and payload of the feedback message, the processing chip can transmit the header and payload (or the address of the payload) of the feedback message to the switching chip 210 through shared memory.

[0145] When the message transmitted by the switching chip 210 is a connection establishment request, the processing chip 220 can generate a connection establishment response header and payload. The connection establishment response is a feedback message to the connection establishment request, and it can be a positive connection response or a negative connection response.

[0146] When the message transmitted by the switching chip 210 is a metadata operation request, the processing chip 220 can generate the header and payload of the metadata operation response. The metadata operation response is the feedback message of the metadata operation request. If the metadata operation request is used to request the execution of a read metadata operation, the processing chip 220, when generating the payload of the metadata operation response, can obtain the metadata and use it as the payload, or obtain the storage address of the metadata and use that storage address as the payload storage address. If the metadata operation request is used to request the execution of a query metadata operation, the processing chip 220, when generating the payload of the metadata operation response, can obtain the operation identifier of the target data and use it as the payload, or obtain the storage address of the operation identifier of the target data and use that storage address as the payload storage address. If the metadata operation request is used to request the execution of a modify metadata operation, the payload generated by the processing chip 220 can indicate whether the metadata modification was successful or failed.

[0147] When the message transmitted by the switching chip 210 is a data read request, the processing chip 220 can generate a data read response header and obtain the storage address of the target data to be read. This target data storage address is the address where the target data is stored in the storage cluster 300, and this storage address can be used as the payload storage address. The processing chip 220 transmits the data read response header and the target data storage address to the switching chip 210. The switching chip 210 can then retrieve the target data from the storage node 310 based on this target data storage address, thereby completing message encapsulation and generating the data read response.

[0148] When the message transmitted by the switching chip 210 is a data write request, the processing chip 220 can generate a data write response header and payload. The payload of this data write response indicates whether the target data write is complete or has failed. The processing chip 220 transmits the data write response header and payload to the switching chip 210. The switching device encapsulates the message based on the data write response header and payload, generating a data write response.

[0149] It should be noted that regarding the "metadata of data in storage cluster 300 stored by processing chip 220", there are two ways in which processing chip 220 stores the metadata of data in storage cluster 300. One way is that processing chip 220 stores the metadata of data in storage cluster 300 in its own internal memory (suitable for scenarios where processing chip 220 contains internal memory). The other way is that processing chip 220 stores the metadata of data in storage cluster 300 in a memory connected to processing chip 220 (suitable for scenarios where processing chip 220 does not contain internal memory or where the internal memory of processing chip 220 has no free storage space). In this embodiment, the example of processing chip 220 storing the metadata of data in storage cluster 300 in its own internal memory will be used for explanation. The only difference in the other method is that the memory is externally connected to processing chip 220.

[0150] Referring to Figure 3, the processing chip 220 includes a processing module 221 and a memory 222. The processing module 221 can be a data processing unit (DPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a neural network processing unit (NPU), a graphics processing unit (GPU), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0151] The memory 222 provides data storage space for the processing module 221. This memory 222 can be dynamic random access memory (DRAM). Besides DRAM, the memory 222 can also be other types of random access memory, such as static random access memory (SRAM), storage class memory (SCM), etc. Alternatively, the memory 222 can also be read-only memory (ROM). The memory 222 can also be a dual in-line memory module (DIMM) or a dual in-line memory module (DIMM). The memory 222 can store computer program instructions that the processing module 221 needs to call. The memory 222 can also store data, such as metadata from the storage cluster 300.

[0152] This application embodiment does not limit the number of processing chips 220 included in the data exchange device 200. The data exchange device 200 may include one or more processing chips 220. When the data exchange device 200 includes multiple processing chips 220, the multiple processing chips 220 have the same function. The switching chip 210 can send different messages to each processing chip 220 respectively, wherein each processing chip 220 receives a different message. The multiple processing chips 220 can process the messages from the switching chip 210 in parallel.

[0153] The following describes two deployment methods for the data exchange device 200:

[0154] Deployment Method 1: Data exchange devices 200 are deployed in the same hardware device.

[0155] In this deployment method, the processing chip 220 and the switching chip 210 of the data exchange device 200 belong to the same hardware device. For example, the data exchange device 200 can be deployed in the form of a switch.

[0156] For example, as shown in Figure 4A, the data exchange device 200 is deployed as a switch between the data access device 100 and the storage cluster 300. All packets sent by the data access device 100 first reach this switch. This switch can process packets from the data access device 100, such as determining whether a connection can be established with the data access device 100, and can also perform metadata operations or access data in the storage cluster 300.

[0157] Assuming that storage node 310 in storage cluster 300 is a server, and that all storage nodes in storage cluster 300 are located in the same rack, then the switch can be a switch deployed at the top of the rack (TOR), a switch deployed at the end of the rack (EOR), a switch deployed in the middle of the rack (MOR), a spine switch (which is a switch deployed between racks), or a core switch.

[0158] Deployment Method 2: The processing chip 220 and the switching chip 210 in the data exchange device 200 are deployed in a distributed manner.

[0159] In this deployment method, the processing chip 220 and the switching chip 210 of the data exchange device 200 are located in different hardware devices. For example, the switching chip 210 of the data exchange device 200 is located in a switch, which is externally connected to one or more acceleration cards, and each acceleration card is equipped with a processing chip 220.

[0160] For example, as shown in Figure 4B, the processing chip 220 of the data exchange device 200 is located in a switch, which is connected to one or more accelerator cards via a system bus. All messages sent by the data access device 100 first reach this switch. The switch can then transmit messages from the data access device 100 to the one or more accelerator cards. Each accelerator card can process the received messages. The method by which the accelerator card processes messages can be found in the aforementioned description of the processing chip 220, and will not be repeated here.

[0161] The data access method provided in the embodiments of this application will be described below. As shown in Figure 5, which is a schematic diagram of a data access method provided in the embodiments of this application, the data access method includes three parts. The first part is the connection establishment process, specifically referred to in steps 501 to 504. The second part is the metadata operation process, specifically referred to in steps 505 to 506. The third part is the data access process, specifically referred to in steps 507 to 508.

[0162] Regarding the execution order and number of executions of these three parts: the first part must be executed before the second and third parts. The second and third parts can be executed independently and have no dependency relationship. Furthermore, the number of executions of the second and third parts is not limited in this embodiment.

[0163] Step 501: Data access device 100 sends a connection establishment request, which is used to request to establish a connection with storage node 310 in storage cluster 300. The connection establishment request carries the IP address of storage cluster 300.

[0164] When accessing metadata or data in the storage cluster 300, the data access device 100 needs to establish a connection with the storage node 310 in the storage cluster 300 first.

[0165] In this embodiment, the storage cluster 300 can provide an Internet Protocol (IP) address to the outside world, so that devices outside the storage cluster 300 can establish connections with the storage nodes 310 in the storage cluster 300. From the perspective of the data access device 100, the data access device 100 can obtain the IP address provided by the storage cluster 300, and the data access device 100 can request to establish a connection with the storage node 310 in the storage cluster 300 by initiating a connection establishment request carrying the IP address (that is, the destination IP address of the connection establishment request is the IP address).

[0166] Regarding the IP address provided by storage cluster 300, this IP address can be understood as the IP address of storage cluster 300 itself. Packets destined for this IP address (such as connection establishment requests, metadata operation requests, and data access requests) are the packets that need to be sent to storage cluster 300.

[0167] This application embodiment does not limit the number of IP addresses that the storage cluster 300 provides to the outside world. The storage cluster 300 may provide one or more IP addresses to the outside world.

[0168] When storage cluster 300 provides an IP address to the outside world, this IP address is the one that the storage cluster 300 exposes to external devices. In this case, the IP address can be considered as a shared IP address among multiple storage nodes 310 in the storage cluster 300. In practical applications, this IP address can be a virtual IP address, meaning that this IP address is not the IP address of any single storage node 310 in the storage cluster 300.

[0169] When storage cluster 300 provides multiple IP addresses to the outside world, these IP addresses are the IP addresses that storage cluster 300 exposes to external devices. These multiple IP addresses can all be virtual IP addresses, or they can be the IP addresses of storage nodes 310 in storage cluster 300.

[0170] This application embodiment does not limit the communication method between the data access device 100 and the storage cluster 300 (middle storage node 310). The data access device 100 can establish a connection with the storage cluster 300 (middle storage node 310) based on TCP / IP; the data access device 100 can also establish a connection with the storage cluster 300 (middle storage node 310) based on RDMA. Therefore, the connection establishment request can be a message sent by the data access device 100 during the three-way handshake process.

[0171] It should be noted that, in this embodiment, since the data exchange device 200 has a connection establishment function, the process of establishing a connection between the data access device 100 and the storage cluster 300 (storage node 310) is essentially a process in which the data exchange device 200 responds to the connection establishment request initiated by the data access device 100. That is, during the connection establishment process, the data exchange device 200, on behalf of the storage cluster 300 (storage node 310), completes a three-way handshake with the data access device 100.

[0172] Step 502: The data exchange device 200 receives a connection establishment request and checks the information carried in the connection establishment request.

[0173] After receiving a connection establishment request, the data exchange device 200 parses the connection establishment request, obtains the information carried in the connection establishment request, and checks the information carried in the connection establishment request.

[0174] The data exchange device 200 determines whether to allow the data access device 100 to establish a connection with the storage cluster 300 by detecting the information carried in the connection establishment request.

[0175] When performing step 502, the data exchange device 200 may only detect the destination IP address in the connection establishment request. If it determines that the destination IP address in the connection establishment request is the IP address of the storage cluster 300, the data exchange device 200 determines that the storage cluster 300 can establish a connection with the data access device 100, that is, the detection result indicates that the connection establishment is allowed. Otherwise, the data exchange device 200 determines that it is not necessary to establish a connection with the data access device 100, that is, the detection result indicates that the connection establishment is not allowed.

[0176] When performing step 502, in addition to detecting the destination IP address in the connection establishment request, the data exchange device 200 also detects other information carried in the connection establishment request. This other information includes, but is not limited to, the source IP address carried in the connection establishment request and the source media access control address (MAC) address carried in the connection establishment request.

[0177] For example, the data exchange device 200 stores a connection whitelist, which records the IP addresses of one or more data access devices 100. If the destination IP address in a connection establishment request is the IP address of the storage cluster 300, the data exchange device 200 can further determine whether the source IP address carried in the connection establishment request belongs to the connection whitelist. If the source IP address carried in the connection establishment request belongs to the connection whitelist, the data exchange device 200 determines that a connection can be established with the data access device 100, meaning the detection result indicates that the connection is allowed. If the source IP address carried in the connection establishment request does not belong to the connection whitelist, the data exchange device 200 determines that a connection does not need to be established with the data access device 100, meaning the detection result indicates that the connection is not allowed.

[0178] Step 503: The detection result indicates that the connection can be established, and the data exchange device 200 sends a connection establishment affirmative response to the data access device 100. The connection establishment affirmative response is a positive response to the connection establishment request, indicating that the connection is agreed to be established or that the connection establishment is complete.

[0179] If the detection result indicates that the connection can be established, the data exchange device 200 generates a connection establishment affirmative response and sends the connection establishment affirmative response to the data access device 100.

[0180] Taking TCP / IP and RDMA as examples again, if the connection establishment request is the first message sent by data access device 100 to storage cluster 300 in the three-way handshake, then after receiving the positive connection establishment acknowledgment, data access device 100 needs to send another message to storage cluster 300. This message is a response from data access device 100 to the positive connection establishment acknowledgment. This message will be transmitted to data exchange device 200 for processing to complete the third handshake. After completing the third handshake, the connection between data access device 100 and storage cluster 300 is successfully established. Data access device 100 can then communicate with storage cluster 300 based on this connection and access metadata or data in storage cluster 300. The process of data access device 100 accessing metadata or data in storage cluster 300 can be found in steps 504-506 and steps 507-508.

[0181] As can be seen from the above description, the connection establishment request from the data access device 100 does not actually need to reach the storage cluster 300. The data exchange device 200 has a connection establishment function and responds to the connection establishment request to establish a connection between the data access device 100 and the storage cluster 300.

[0182] Within the data exchange device 200, the process for handling the connection establishment request is as follows:

[0183] Method 1: Switching chip 210 executes step 502 to detect the information carried in the connection establishment request.

[0184] Upon receiving a connection establishment request, the switching chip 210 detects the information carried in the request. If the detection result indicates that connection establishment is permitted, the switching chip 210 can transmit the connection establishment request to the processing chip 220. After receiving the connection establishment request, the processing chip 220 generates a connection establishment affirmative response payload and header. The processing chip 220 transmits the connection establishment affirmative response payload and header to the switching chip 210. The switching chip 210 encapsulates the received connection establishment affirmative response payload and header to obtain a connection establishment affirmative response. The switching chip 210 sends the connection establishment affirmative response to the data access device 100.

[0185] Method 2: Processing chip 220 executes step 502 to detect the information carried in the connection establishment request.

[0186] After receiving a connection establishment request, the switching chip 210 transmits the request to the processing chip 220 via shared memory. The processing chip 220 detects the information carried in the connection establishment request. If the detection result indicates that the connection can be established, the processing chip 220 generates a connection establishment affirmative acknowledgment payload and header. The processing chip 220 transmits the connection establishment affirmative acknowledgment payload and header to the switching chip 210. The switching chip 210 encapsulates the received connection establishment affirmative acknowledgment payload and header to obtain a connection establishment affirmative acknowledgment. The switching chip 210 sends the connection establishment affirmative acknowledgment to the data access device 100.

[0187] Specifically, for the processing chip 220, the connection establishment affirmative acknowledgment generated by the processing chip 220 can have a Layer 3 header or a Layer 4 header. For the switching chip 210, when encapsulating the payload and header of the received connection establishment affirmative acknowledgment, the switching chip 210 can also generate other headers, such as a Layer 2 header; the switching chip 210 encapsulates the payload and header of the received connection establishment affirmative acknowledgment, as well as the header it generates, to obtain the connection establishment affirmative acknowledgment.

[0188] Step 504: If the detection result indicates that connection establishment is not allowed, the data exchange device 200 sends a connection establishment negative response to the data access device 100 or discards the connection establishment request. The connection establishment negative response is a negative response to the connection establishment request, indicating that the connection establishment is refused or the connection establishment has failed.

[0189] The difference between how data exchange device 200 executes step 504 and how it executes step 503 lies in the type of response constructed by data exchange device 200. Furthermore, when switching chip 210 executes step 502, if the detection result indicates that connection establishment is not allowed, switching chip 210 can discard the connection establishment request and not transmit it to processing chip 220. When processing chip 220 executes step 502, if the detection result indicates that connection establishment is not allowed, processing chip 220 can discard the connection establishment request, or it can generate a connection establishment negative response header and payload and transmit them to switching chip 210. Switching chip 210 encapsulates the connection establishment negative response header and payload to obtain the connection establishment negative response, and refrigerator data access device 100 responds with the connection establishment negative response. For details, please refer to the foregoing content; further elaboration is not provided here.

[0190] Figure 6A illustrates the processing flow of a connection establishment request. In Figure 6A, when the connection establishment request sent by data access device 100 is transmitted to data exchange device 200, it first passes through the switching chip 210 of data exchange device 200. The switching chip 210 detects the information carried in the connection establishment request. After successful detection, the switching chip 210 transmits the connection establishment request to processing chip 220. Processing chip 220 receives the connection establishment request, processes it, and sends a connection establishment response header and payload back to switching chip 210. This connection establishment response can be a positive or negative response. Switching chip 210 encapsulates the connection establishment response header and payload, generates a connection establishment response, and sends the response to data access device 100.

[0191] The data exchange device 200 can establish a connection with the data access device 100 on behalf of the storage cluster 300, without the storage cluster 300 needing to participate in the connection establishment process, thus improving the efficiency of connection establishment.

[0192] After the connection between the storage cluster 300 and the data access device 100 is established, the data access device 100 can access the storage cluster 300 through the data exchange device 200, including metadata operations and data access. The process of the data access device 100 accessing the storage cluster 300 through the data exchange device 200 will be described below. First, the process of the data access device 100 performing metadata operations on the storage cluster 300 through the data exchange device 200 will be described.

[0193] Step 505: Data access device 100 sends a metadata operation request, which is used to request an operation to be performed on metadata.

[0194] This application does not limit the types of metadata requests performed by the data access device 100. Several operations performed on metadata are listed here.

[0195] Operation 1: Read metadata.

[0196] When data access device 100 needs to access target data (such as a file, object, or data block) or view the attribute information of the target data, data access device 100 can first request to obtain the metadata of the target data. In this case, data access device 100 can initiate a metadata operation request to request to read the metadata of the target data. The metadata operation request (such as the payload of the metadata operation request) can carry the type of metadata operation (such as the type is read metadata) and the identifier of the target data.

[0197] Taking a storage cluster 300 with a file system deployed as an example, when data access device 100 needs to obtain file A or view some attribute information of file A, it can first request to obtain the metadata of file A. In this case, data access device 100 initiates a metadata operation request to obtain the metadata of file A. The operation type in this metadata operation request is read metadata, and the metadata operation request can also carry the file name of file A. When data access device 100 needs to obtain directory B (that is, to obtain all directories or files contained in directory B) or view some attribute information of directory B, it can first request to obtain the metadata of directory B. In this case, data access device 100 initiates a metadata operation request to obtain the metadata of directory B. The operation type requested in this metadata operation request is read metadata, and the metadata operation request can also carry the directory name of directory B.

[0198] Step 2: Query metadata.

[0199] The operation identifier of target data is an identifier used when operating on target data. It is more easily recognized by software modules that operate on the file. The operation identifier of target data can be considered a type of metadata. Therefore, obtaining the operation identifier of target data is an operation performed on metadata. In this embodiment, the operation of obtaining the operation identifier of target data is called querying metadata. In practical applications, the operation of obtaining the operation identifier of target data can also be defined by other names; this embodiment does not limit the specific name of the operation identifier for obtaining target data.

[0200] Taking a file as the target data as an example, the file operation identifier can be called a file handle. When data needs to be read from a file, an operating system function must first be called, along with the filename, to open the file. This operating system function retrieves a sequence number, which is the file handle. This file handle is the unique identifier for the opened file. Any subsequent operations on the opened file must be identified using this file handle.

[0201] When data access device 100 needs to access target data (such as a file, object, or data block), it can first request the operation identifier of the target data. In this case, data access device 100 can initiate a metadata operation request to request the operation identifier of the target data. The metadata operation request (such as the payload of the metadata operation request) can carry the operation type (such as the type is query metadata) and the identifier of the target data.

[0202] Step 3: Modify metadata.

[0203] In some scenarios, data access device 100 is allowed to modify the metadata of target data. For example, the metadata of target data records the access permissions for that target data, such as the target data being read-only (i.e., only readable, not modifiable), write-only (i.e., only modifiable, not readable), or the target data only allowing access from specified users. Data access device 100 can modify the access permissions of target data according to actual needs, such as changing the read-only type of target data to write-only, deleting users allowed to access the target data, or adding users allowed to access the target data.

[0204] When data access device 100 needs to modify the metadata of target data (such as a file, object, or data block), data access device 100 can initiate a metadata operation request to request the metadata of the target data to be modified. The metadata operation request (such as the payload of the metadata operation request) can carry the operation type (such as the type is to modify metadata), and can also carry the identifier of the target data and the metadata modification information. The metadata modification information indicates the method of modifying the metadata, such as indicating to update the write-only type of the target data to the read-only type, or adding a user A who is allowed to access the target data.

[0205] Step 506: The data exchange device 200 receives the metadata operation request and performs an operation on the metadata according to the metadata operation request.

[0206] The data exchange device 200 can store metadata of the data in the storage cluster 300. The metadata stored in the data exchange device 200 can be metadata of all the data in the storage cluster 300, or metadata of some of the data in the storage cluster 300.

[0207] After receiving a metadata operation request, the data exchange device 200 can first determine whether the metadata stored in the data exchange device 200 contains the metadata targeted by the operation (such as metadata to be read, metadata to be queried, or metadata to be modified). The following are some examples.

[0208] 1. If the metadata stored in the data exchange device 200 contains the metadata targeted by this operation.

[0209] Since the metadata targeted by the operation is already stored in the data exchange device 200, the data exchange device 200 can directly perform the operation on the metadata stored in the data exchange device 200.

[0210] For example, when the operation type is to read metadata, the data exchange device 200 can obtain the metadata locally (that is, within the data exchange device 200), carry the metadata in the metadata operation response, and send the metadata operation response to the data access device 100.

[0211] Taking a storage cluster 300 with a file system deployed as an example, when a metadata operation request is used to request the reading of the metadata of file A, the data exchange device 200 queries the metadata of file A in the metadata of each file and directory in the stored file system. After retrieving the metadata of file A, the data exchange device 200 carries the metadata of file A in the metadata operation response and sends the metadata operation response to the data access device 100.

[0212] When the operation type is to query metadata, the data exchange device 200 can carry the operation identifier of the target data in the metadata operation response and send the metadata operation response to the data access device 100.

[0213] Taking a storage cluster 300 with a file system deployed as an example, when a metadata operation request is used to query the metadata of file A, the data exchange device 200 queries the metadata of file A in the metadata of each file and directory in the stored file system. After retrieving the metadata of file A, the data exchange device 200 sends the metadata operation response to the data access device 100, carrying the handle of file A in the metadata operation response.

[0214] 2. If the metadata stored in the data exchange device 200 does not contain the metadata targeted by this operation.

[0215] Since the data exchange device 200 does not store the metadata for this operation, the data exchange device 200 can access the storage cluster 300 to obtain the metadata.

[0216] This application embodiment does not limit the method by which the data exchange device 200 accesses the storage cluster 300 to obtain the metadata. For example, the data exchange device 200 may send a first metadata instruction to the storage node 310 in the storage cluster 300 that stores the metadata, the first metadata instruction being used to request the storage address of the metadata. After receiving the first metadata instruction, the storage node 310 obtains the storage address of the metadata according to the first metadata instruction and returns the storage address of the metadata to the data exchange device 200. The data exchange device 200 may then obtain the metadata from the storage cluster 300 via DMA based on the storage address of the metadata. As another example, the data exchange device 200 may send a second metadata instruction to the storage node 310 that stores the metadata, the second metadata instruction being used to request the metadata. After receiving the second metadata instruction, the storage node 310 obtains the metadata according to the second metadata instruction and returns the metadata to the data exchange device 200, thereby enabling the data exchange device 200 to obtain the metadata.

[0217] After obtaining the metadata, the data exchange device 200 can perform operations on the metadata. The method of performing operations by the data exchange device 200 can be found in the foregoing content and will not be repeated here.

[0218] As can be seen from the above description, the executor of the metadata operation is the data exchange device 200, which has metadata operation function and responds to the metadata operation request.

[0219] Inside the data exchange device 200, the processing chip 220 can store metadata of the data in the storage cluster 300. The processing procedure for metadata operation requests within the data exchange device 200 is as follows:

[0220] After receiving a metadata operation request, the switching chip 210 parses the information carried in the request. For example, the switching chip 210 can parse the payload in the metadata operation request to determine the type of metadata operation. If the switching device determines that the metadata operation request is for requesting an operation on metadata, it will transmit the metadata operation request to the processing chip 220.

[0221] After receiving the metadata operation request, the processing chip 220 determines whether the metadata stored in the processing chip 220 contains the metadata targeted by the operation. The following describes two scenarios.

[0222] 1. If included, the processing chip 220 performs operations on the metadata of the target data stored locally (i.e., the metadata of the target data stored by the processing chip 220).

[0223] When the operation is to read metadata, the processing chip 220 obtains the metadata of the target data stored locally, generates a metadata operation response payload and header, and the payload of the metadata operation response contains the metadata of the target data. The processing chip 220 transmits the metadata operation response payload and header to the switching chip 210. The switching chip 210 completes the packet encapsulation according to the metadata operation response payload and header, obtains the metadata operation response, and sends the metadata operation response to the data access device 100. The header of the metadata operation response generated by the processing chip 220 can be a Layer 3 header or a Layer 4 header. When completing the packet encapsulation according to the metadata operation response payload and header, the switching chip 210 can also generate other headers for the metadata operation response, such as a Layer 2 header. The switching chip 210 encapsulates the payload and header obtained from the processing chip 220 and its own generated header together to generate the metadata operation response.

[0224] When the operation is to query metadata, the processing chip 220 obtains the operation identifier of the target data from the metadata of the target data stored locally, or generates the operation identifier of the target data by calling an operating system function, and generates a payload and header for the metadata operation response. The payload of the metadata operation response includes the operation identifier of the target data. The processing chip 220 transmits the payload and header of the metadata operation response to the switching chip 210. The switching chip 210 completes the packet encapsulation according to the payload and header of the metadata operation response, obtains the metadata operation response, and sends the metadata operation response to the data access device 100. The header of the metadata operation response generated by the processing chip 220 can be a Layer 3 header or a Layer 4 header. When completing the packet encapsulation according to the payload and header of the metadata operation response, the switching chip 210 can also generate other headers for the metadata operation response, such as a Layer 2 header. The switching chip 210 encapsulates the payload and header obtained from the processing chip 220 and its own generated header together to generate the metadata operation response.

[0225] When the metadata operation involves modifying metadata, the processing chip 220 can modify the metadata of the target data stored locally and update the modified metadata in the storage node 310 of the storage cluster 300. This embodiment does not limit the timing of the processing chip 220 updating the modified metadata in the storage node 310 of the storage cluster 300. For example, after modifying the metadata of the target data stored locally, the processing chip 220 may immediately update the modified metadata in the storage node 310 of the storage cluster 300. Alternatively, after modifying the metadata of the target data stored locally, the processing chip 220 may wait for a period of time before updating the modified metadata in the storage node 310 of the storage cluster 300. The processing chip 220 generates a payload and header for a metadata operation response. The payload of the metadata operation response indicates whether the metadata modification was successful or failed. The processing chip 220 transmits the payload and header of the metadata operation response to the switching chip 210. The switching chip 210 encapsulates the message based on the payload and header of the metadata operation response, obtains the metadata operation response, and sends the metadata operation response to the data access device 100. The header of the metadata operation response generated by the processing chip 220 can be a Layer 3 header or a Layer 4 header. When encapsulating the message based on the payload and header of the metadata operation response, the switching chip 210 can also generate other headers for the metadata operation response, such as a Layer 2 header. The switching chip 210 encapsulates the payload and header obtained from the processing chip 220, along with its own generated header, together to generate the metadata operation response.

[0226] 2. If not included, the processing chip 220 accesses the storage cluster 300 and performs operations on the metadata of the target data stored in the storage node 310.

[0227] When the operation is to read metadata, the processing chip 220 obtains the metadata of the target data from the storage node 310 in the storage cluster 300. The operations performed by the processing chip 220 after obtaining the metadata of the target data are similar to those performed by the processing chip 220 after obtaining the metadata of the target data stored locally. For details, please refer to the foregoing description, which will not be repeated here.

[0228] When the operation is to query metadata, the processing chip 220 obtains the operation identifier of the target data from the storage node 310 in the storage cluster 300. The operation performed by the processing chip 220 after obtaining the operation identifier of the target data from the storage node 310 in the storage cluster 300 is similar to the operation performed by the processing chip 220 after obtaining the operation identifier of the target data stored locally. For details, please refer to the above description, which will not be repeated here.

[0229] When the operation involves modifying metadata, the processing chip 220 updates the modified metadata to the storage node 310 of the storage cluster 300. This embodiment does not limit the timing of the processing chip 220 updating the modified metadata to the storage node 310 of the storage cluster 300. For example, after receiving a metadata operation request, the processing chip 220 immediately updates the modified metadata to the storage node 310 of the storage cluster 300. Alternatively, the processing chip 220 may wait for a period of time before updating the modified metadata to the storage node 310 of the storage cluster 300. The operation performed by the processing chip 220 is similar to the operation performed after modifying the metadata of the target data stored locally; please refer to the foregoing description for details, which will not be repeated here.

[0230] Within the data exchange device 200, the switching chip 210 and the processing chip 220 can interact based on shared memory. For details on the shared memory-based interaction process, please refer to the relevant documentation on shared memory. The following explanation uses a virtual queue-based interaction method as an example to illustrate the interaction between the switching chip 210 and the processing chip 220:

[0231] The switching chip 210 writes the metadata operation request into the buffer and adds a corresponding descriptor to the receive ring buffer, notifying the processing chip 220 that there is a request to be processed. The processing chip 220 retrieves the descriptor from the receive ring buffer and retrieves the metadata operation request from the buffer based on the descriptor.

[0232] Processing chip 220 generates the payload and header of the metadata operation response, writes the payload and header to a buffer, and adds a descriptor to the transmit ring buffer. This descriptor records the payload (or storage address of the payload) and the address of the header in the buffer, notifying switching chip 210 that processing is complete. Switching chip 210 retrieves the descriptor from the transmit ring buffer and, based on the descriptor, retrieves the payload (or storage address of the payload) and header of the metadata operation response from the buffer. When switching chip 210 retrieves the payload and header of the metadata operation response, it encapsulates them to generate a metadata operation response. When switching chip 210 retrieves the storage address and header of the metadata operation response, it retrieves the payload from storage cluster 300 (storage node 310) based on the storage address, encapsulates the payload and header of the metadata operation response, and generates a metadata operation response. After generating a metadata operation response, the switching chip 210 sends the metadata operation response to the data access device 100.

[0233] It should be noted that when a data access device requests to perform an operation on the metadata of target data via a metadata operation request, the processing chip 220 does not save the metadata of the target data. In some scenarios, the processing chip 220 can also obtain the storage address of the metadata from the storage node 310. The processing chip 220 constructs a metadata operation response header and transmits the metadata operation response header and the metadata storage address (or the operation identifier storage address) to the switching chip 210. The switching chip 210 obtains or writes the metadata of the target data from the storage node 310 based on the metadata storage address, or obtains the operation identifier from the storage node 310 based on the operation identifier storage address. For example, the processing chip 220 writes the metadata operation response header and the metadata storage address into the buffer of the virtual queue and adds a corresponding descriptor to the transmit ring buffer to notify the switching chip 210 that the request processing is complete. The switching chip 210 obtains a descriptor from the transmit ring buffer, and obtains the metadata operation response, message header, and storage address of the metadata from the buffer based on the descriptor. Then, the switching chip 210 obtains the payload of the metadata operation response based on the storage address of the metadata, and then obtains the metadata operation response and sends the metadata operation response to the data access device 100.

[0234] Figure 6B illustrates the processing flow of a metadata operation request. In Figure 6B, when the metadata operation request sent by the data access device 100 is transmitted to the data exchange device 200, the request first passes through the switching chip 210 of the data exchange device 200. The switching chip 210 transmits the metadata operation request to the processing chip 220. The processing chip 220 receives the metadata operation request, processes it, and sends back the header and payload of the metadata operation response to the switching chip 210. Based on the header and payload of the metadata operation response, the switching chip 210 generates a metadata operation response and sends it to the data access device 100.

[0235] The data exchange device 200 has metadata operation capabilities. As the "front end" of the storage cluster 300, the data exchange device 200 processes metadata operation requests, effectively reducing the computational power consumed by the storage cluster 300. Internally, the data exchange device 200 uses a switching chip 210 to forward metadata operation requests, and a processing chip 220 to execute metadata operations. The presence of the processing chip 220 improves the performance of the data exchange device 200 and ensures that the data exchange chip 210 can process metadata operation requests.

[0236] It should be noted that steps 505 and 506 describe the process of data exchange device 200 performing metadata operations in a scenario where data access device 100 initiates a metadata operation request. In practical applications, data exchange device 200 can perform metadata operations in other scenarios. For example, when data exchange device 200 receives a data access request for metadata to be written to target data, in addition to processing the data access request to access the target data in storage cluster 300, data exchange device 200 also generates metadata for the target data and updates the metadata of the target data in storage cluster 300. When garbage collection is performed on data within storage cluster 300, and the garbage collection process involves updating the data's metadata, data exchange device 200 can synchronously update the metadata of the stored data, such as deleting metadata or modifying metadata (this embodiment does not limit the synchronous update process of metadata).

[0237] The following section describes the process by which data access device 100 accesses the storage cluster 300 through data exchange device 200 to complete data reading and writing.

[0238] Step 507: Data access device 100 sends a data access request, which is used to request access to data.

[0239] In this application embodiment, data access requests include two types: one is a data access request for reading data, which can be called a data read request. This data read request may carry the storage address of the target data to be read, or it may carry the identifier of the data set to which the target data belongs (the identifier of the data set can be a filename or an object identifier), and the location information of the target data within the data set. Correspondingly, the data access response can also be called a data read response. The other type is a data access request for writing data, which can be called a data write request. This data write request may carry the storage address of the target data to be written, or it may carry the identifier of the data set to which the target data belongs (the identifier of the data set can be a filename or an object identifier), and the location information of the target data within the data set. Correspondingly, the data access response can also be called a data write response.

[0240] Step 508: The data exchange device 200 receives the data access request and processes it.

[0241] After receiving the data access request, the data exchange device 200 parses the request. The data exchange device 200 itself stores metadata of the data in the storage cluster 300 or has the ability to access metadata of the data in the storage cluster 300, and can determine the storage node 310 to which the target data in the storage cluster 300 belongs. For ease of explanation, in this embodiment, the storage node 310 to which the target data belongs is referred to as the first storage node. After determining the first storage node, the data exchange device 200 accesses the target data in the first storage node.

[0242] After accessing the target data in the storage cluster 300, the data exchange device 200 can also generate a data access response and send it back to the data access device 100. This data access response is a response to the data access request.

[0243] Inside the data exchange device 200, the processing chip 220 can parse and process data access requests, obtain the storage address of the target data, and transmit the storage address of the target data to the switching chip 210, triggering the switching chip 210 to access the target data in the first storage node based on the storage address of the target data. The interaction process between the processing chip 220 and the switching chip 210 inside the data exchange device 200 is described below:

[0244] After receiving a data access request, the switching chip 210 can send the request to the processing chip 220 for processing. The processing chip 220 can then process the request. Specifically, upon receiving the request, the switching chip 210 can detect the destination address carried in the request. If the destination address matches the address of the storage cluster 300, the switching chip 210 will transmit the request to the processing chip 220. Alternatively, the switching chip 210 can send the request directly to the processing chip 220 without detecting the destination address. When sending the data access request to the processing chip 220, the switching device can remove the Layer 2 header from the request and transmit the request with the header removed to the processing chip 220.

[0245] After receiving a data access request (a data access request with the Layer 2 header removed), the processing chip 220 obtains the storage address of the target data and transmits the storage address of the target data to the switching chip 210 via shared memory. After obtaining the storage address of the target data, the switching chip 210 accesses the target data in the first storage node via DMA.

[0246] When the data access request is a data read request, the switching chip 210 reads the target data from the storage address of the target data in the first storage node through DMA (i.e., system bus).

[0247] When the data access request is a data write request, the switching chip 210 writes the target data to the storage address of the target data in the first storage node via DMA (i.e., system bus).

[0248] The processing chip 220 can also generate a data access response, and can construct the header and payload (or the storage address of the payload) of the data access response. When the data access request is a data read request, the payload is the target data, and the storage address of the payload is the storage address of the target data. The processing chip 220 can transmit the header and payload storage address of the data access response to the switching chip 210 through shared memory. After reading the target data from the storage address of the target data in the first storage node, the switching chip 210 encapsulates the header and payload of the data access response to obtain the data access response, and sends the data access response back to the data switching device 200.

[0249] When the data access request is a data write request, the processing chip 220 transmits the storage address of the target data to the exchange information. The processing chip 220 can transmit the header and payload of the data access response to the exchange chip 210 through shared memory, wherein the payload indicates that the target data was successfully written. The interaction method between the processing chip 220 and the exchange chip 210 via shared memory can be found in the foregoing description and will not be repeated here. The exchange chip 210 writes the target data to the storage address of the target data in the first storage node, encapsulates the header and payload of the data access response to obtain the data access response, and sends the data access response back to the data exchange device 200.

[0250] In the foregoing description, the successful writing of target data is taken as an example. If the switching chip 210 fails to successfully write the storage address of the target data in the first storage node to the target data, or if the target data is not persisted to the memory in the first storage node, the processing chip 220 determines that the target data writing has failed, and the payload generated by the processing chip 220 indicates that the target data writing has failed. This application embodiment does not limit the method by which the processing chip 220 determines that the target data writing has failed; any method that ensures that the processing chip 220 can determine that the target data writing has failed is applicable to this application embodiment. The subsequent operations of the processing chip 220 generating the data access response header and payload can be found in the foregoing content and will not be repeated here.

[0251] Figure 6C illustrates the processing flow of a data read request. In Figure 6C, when a data read request sent by data access device 100 is transmitted to data exchange device 200, the request first passes through the switching chip 210 of data exchange device 200. The switching chip 210 transmits the data read request to processing chip 220. Processing chip 220 receives the data read request and processes it, such as parsing the request based on the protocol stack and sending a data read response header back to switching chip 210, or obtaining the storage address of the target data to be read from the data read request and sending the storage address of the target data to switching chip 210. Switching chip 210 obtains the storage address of the target data to be read and reads the target data from storage cluster 300 based on the storage address, obtaining the payload of the data read response. Based on the header and payload of the data read response, switching chip 210 generates a data read response and sends it to data access device 100.

[0252] Figure 6D illustrates the processing flow of a data write request. In Figure 6D, when a data write request sent by data access device 100 is transmitted to data exchange device 200, the request first passes through the switching chip 210 of data exchange device 200. Switching chip 210 transmits the data write request to processing chip 220. Processing chip 220 receives the data write request and processes it. For example, processing chip 220 parses the data write request based on the protocol stack and sends a data write response header back to switching chip 210. Alternatively, it determines the storage address of the target data to be written based on the data write request and transmits the storage address of the target data to switching chip 210. Switching chip 210 obtains the storage address of the target data, reads data from storage cluster 300 based on the storage address, obtains the payload of the data write response, and generates a data write response based on the header and payload of the data write response, sending the data write response to data access device 100.

[0253] In this embodiment, the data exchange device 200 can handle the data access request instead of the storage node 310, avoiding the transmission of the data access request between different storage nodes 310 in the storage cluster 300, thus avoiding the data access request detours. Furthermore, the storage node 310 does not need to handle the data access request, reducing the computing power requirements on the storage node 310. When the data exchange device 200 handles a data write request, within the data exchange device, the processing chip 220 instructs the exchange chip 210 to perform data writing by sending the storage address of the target data. The exchange chip 210 can quickly write the data to the first storage node via the system bus, improving data write efficiency. When the data exchange device 200 handles a data read request, within the data exchange device, the processing chip 220 instructs the exchange chip 210 to perform data reading by sending the storage address of the target data. The exchange chip 210 can quickly read the target data from the first storage node via the system bus. On the one hand, the exchange chip 210 can ensure the reading efficiency of the target data by utilizing the system bus. On the other hand, after reading the target data, the data exchange device 200 can directly feed back the target data to the data exchange device 200. In this process, the target data does not need to go through the processing chip 220 before reaching the exchange chip 210. The target data does not have to go around within the data exchange device 200, which can also improve the data reading efficiency.

[0254] The data access device 100 and the data exchange device 200 interact differently, and the data exchange device 200 also processes the data access request differently. These will be described separately below. To easily distinguish between different data access requests, the data read request received by the data exchange device 200 will be referred to as the first read request, the second read request, and the third read request; the data write request received by the data exchange device 200 will be referred to as the first write request, the second write request, and the third read request. The following discussion will address each case separately.

[0255] In the first scenario, data access device 100 and data exchange device 200 interact based on RDMA.

[0256] In the first scenario, when the data access request is a data write request, there are two possible implementation methods:

[0257] The first implementation method: the data access device 100 does not need to know the storage address of the target data in the storage node 310.

[0258] Data access device 100 sends a first write request to data exchange device 200, which requests the writing of target data. The first write request carries the identifier of the data set (which can be a data block, file, or object) to which the target data is to be written, and the location information of the target data, indicating its position within the data set. The location information includes an offset and a data length. The offset describes the distance of the target data from the start of the data set, and the data length is the length of the target data. The first write request also carries the storage address of the target data within data access device 100; for ease of explanation, this storage address is referred to as the source storage address of the target data.

[0259] The data exchange device 200 receives the first write request. Inside the data exchange device 200, after obtaining the first write request, the exchange chip 210 transmits the first write request to the processing chip 220 through shared memory.

[0260] After the processing chip 220 obtains the first write request, it parses the first write request and calculates the storage address of the target data based on the identifier of the data set in the first write request and the location information of the target data. The storage address of the target data is the storage address allocated by the processing chip 220 for the target data in the storage node 310.

[0261] The processing chip 220 retrieves the target data from the data access device 100 via one-sided RDMA based on the target data's storage address in the data access device 100. The specific process is as follows: The processing chip 220 sends an RDMA READ message to the data access device 100, which carries the source storage address of the target data. After receiving the RDMA READ message, the data access device 100 (such as a network interface card in the data access device 100) sends an RDMA READ response message to the data exchange device 200, which carries the target data.

[0262] The switching chip 210 in the data switching device 200 receives the RDMA READ response message and obtains the target data from it.

[0263] Processing chip 220 transmits the storage address of the target data to switching chip 210 via shared memory. The process of processing chip 220 transmitting the storage address of the target data to switching chip 210 via shared memory is as follows: Processing chip 220 stores the storage address of the target data in a buffer, adds a descriptor to the TX ring buffer, and records the position of the target data's storage address in the buffer in the descriptor. Processing chip 220 notifies switching chip 210 to retrieve the processing result of the first write request. Switching chip 210 extracts the descriptor from the TX ring buffer and obtains the storage address of the target data based on the descriptor.

[0264] The switching chip 210 stores the target data in the storage node 310 via DMA based on the storage address of the target data.

[0265] It should be noted that if the target data volume is large enough to exceed the MTU, after the processing chip 220 sends an RDMA READ message to the data access device 100, the data access device 100 will split the target data into multiple sub-data, each sub-data having a data volume equal to the MTU. The data access device 100 will then generate and send multiple RDMA READ response messages. Each RDMA READ response message carries one sub-data.

[0266] The switching chip 210 of the data switching device 200 receives multiple RDMA READ response messages and obtains multiple sub-data from these messages. The processing chip 220 splits the storage address of the target data into the storage addresses of multiple sub-data and transmits these addresses to the switching chip 210 via shared memory. The process of transmitting the storage addresses of multiple sub-data to the switching chip 210 via shared memory is as follows: The processing chip 220 stores the storage addresses of the multiple sub-data in a buffer and sequentially adds multiple descriptors to the TX ring buffer, each descriptor recording the position of a sub-data's storage address in the buffer. The processing chip 220 notifies the switching chip 210 to retrieve the processing result of the first write request. The switching chip 210 sequentially retrieves the multiple descriptors from the TX ring buffer and obtains the storage address of the sub-data based on the retrieved descriptor. Based on the storage address of the sub-data, the sub-data is stored in the storage node 310 via DMA. For example, each time the switching device extracts a descriptor from the TX ringbuffer and obtains the storage address of a sub-data, it stores the sub-data carried in an RDMA READ response message in storage node 310 based on the storage address of the sub-data via DMA.

[0267] When the switching chip 210 writes target data to the storage cluster 300, the data switching device 200 can also send a first write response to the data access device 100 to notify the data access device 100 whether the target data write was successful or failed. The way the data switching device 200 sends the first write response to the data access device 100 is similar to the way the data switching device 200 sends the data access response mentioned above, and can be found in the foregoing description, which will not be repeated here.

[0268] The second implementation method is that the data access device 100 can know the storage address of the target data in the storage node 310.

[0269] Data access device 100 sends a second write request to data exchange device 200, the second write request being used to request the writing of target data. The second write request carries the identifier of the data set to which the target data is to be written, and the location information of the target data, which indicates the position of the target data within that data set. The location information of the target data includes an offset and a data length; the offset describes the distance of the target data's position within the data set from the start of the data set, and the data length is the length of the target data.

[0270] When the data exchange device 200 receives the second write request, the internal processing procedure of the data exchange device 200 is as follows: After obtaining the second write request, the exchange chip 210 transmits the second write request to the processing chip 220 through shared memory.

[0271] After obtaining the second write request, the processing chip 220 parses the second write request and calculates the storage address of the target data based on the identifier of the data set in the second write request and the location information of the target data. The storage address of the target data is the storage address allocated by the processing chip 220 for the target data in the storage node 310.

[0272] The processing chip 220 generates a header and payload for a second write response, the payload of which is the storage address of the target data. The header generated by the processing chip 220 is a layer 3 header and a layer 4 header, and this second write response is the response to the second write request.

[0273] The processing chip 220 transmits the header and payload of the second write response to the switching chip 210 via shared memory. The process by which the processing chip 220 transmits the header and payload of the second write response to the switching chip 210 via shared memory is as follows:

[0274] Processing chip 220 stores the header and payload of the second write response in a buffer. Processing chip 220 adds a descriptor to the TX ring buffer, which records the storage location of the header and payload of the second write response in the buffer. Processing chip 220 notifies switching chip 210 to retrieve the processing result of the second write request. Switching chip 210 retrieves the descriptor from the TX ring buffer and determines the storage location of the header and payload of the second write response in shared memory based on the descriptor.

[0275] The switching chip 210 reads the header and payload of the second write response from the buffer according to the storage location of the second write response header and payload. The switching chip 210 generates a second write response based on the second write response header and payload. When generating the second write response, the switching chip 210 generates a Layer 2 header and encapsulates the Layer 2 header, Layer 3 header, Layer 4 header, and payload together to generate the second write response.

[0276] The switching chip 210 sends a second write response to the data access device 100.

[0277] The data access device 100 receives the second write response and obtains the storage address of the target data from the second write response. The data access request sends a third write request to the data exchange device 200. The third write request is used to request the writing of the target data, and the third write request carries the target data and the storage address of the target data.

[0278] Upon receiving a third write request, the data exchange device 200 stores the target data at the storage address specified in the third write request. The internal processing of the third write request by the data exchange device 200 is as follows: the switching chip 210 receives the third write request, obtains the target data and its storage address from the request, and stores the target data in the storage node 310 via DMA based on the storage address. For example, the third write request can be a data write request sent based on one-sided RDMA. When the switching chip 210 parses the request as such, it obtains the target data and its storage address from the request.

[0279] It should be noted that if the target data is large enough to exceed the MTU, the data access device 100 will split the target data into multiple sub-data, each sub-data having a data size equal to the MTU. The data access device 100 will transmit the third write request in a message stream, which includes multiple messages, each message carrying one sub-data. The first message carries the storage address of the target data. The storage address of the target data includes the starting storage address of the target data and the data length of the target data.

[0280] When the switching chip 210 in the data switching device 200 receives the message stream, it obtains sub-data from each message and obtains the storage address of the target data from the first message. Based on the storage address of the target data, the switching device sequentially stores each sub-data in the storage node 310.

[0281] In the first implementation, the data exchange device 200 can obtain the target data from the data access device 100 via RDMA. The data exchange device 200 determines the storage address of the target data and stores the target data in the storage cluster 300. This implementation is suitable for scenarios where the data access device 100 does not need to be aware of the data distribution status in the storage cluster 300. In the second implementation, the data access device 100 has already obtained the storage address of the target data, and the data exchange device 200 can simultaneously obtain both the target data and its storage address from the data access device 100. This implementation is suitable for scenarios where the data access device 100 can be aware of the data distribution status in the storage cluster 300. From the two data writing implementations in the RDMA scenario, it can be seen that for the data exchange device 200, when it can handle data write requests, the switching chip 210 implements the writing of the target data, while the processing chip 220 only needs to complete the parsing of the message and the calculation of the storage address of the target data. In this method, the data write request will not reach the storage cluster 300, avoiding the situation where the data write request is detoured in the storage cluster 300, which can effectively ensure the writing efficiency of the target data. In addition, during the data writing process, after the target data arrives at the data exchange device 200 from the data access device 100, the switching chip 210 of the data exchange device 200 writes the target data to the storage node 310 of the storage cluster 300. The target data does not need to be actually transmitted to the processing chip 220. There is no detour in the transmission path of the target data within the data exchange device 200, which further ensures the writing efficiency of the target data.

[0282] In the first scenario, when the data access request is a data read request, it can also be implemented in the following two ways:

[0283] The first implementation method: The computing device does not need to know the storage address of the target data in storage node 310.

[0284] Data access device 100 sends a first read request to data exchange device 200, which requests to read target data. The first read request carries an identifier of the data set (which can be a data block, file, or object) to which the target data needs to be written, and location information of the target data, indicating the position of the target data within the data set. The location information includes an offset and a data length; the offset describes the distance of the target data from the start of the data set, and the data length is the length of the target data.

[0285] The data exchange device 200 receives the first read request. Inside the data exchange device 200, after obtaining the first read request, the exchange chip 210 transmits the first read request to the processing chip 220 through shared memory.

[0286] After the processing chip 220 obtains the first read request, it parses the first read request and calculates the storage address of the target data based on the identifier of the data set in the first read request and the location information of the target data. The storage address of the target data is the storage address of the target data in the storage node 310.

[0287] The processing chip 220 generates a header for the first read response and the storage address of the target data. The header generated by the processing chip 220 is a layer 3 header and a layer 4 header. The first read response is a response message prepared by the data exchange device 200 in response to the first data read request.

[0288] The processing chip 220 transmits the header of the first read response and the storage address of the target data to the switching chip 210 via shared memory. The storage address of the target data can be understood as the storage address of the payload of the first read response. The process by which the processing chip 220 transmits the header of the first read response and the storage address of the target data to the switching chip 210 via shared memory is as follows:

[0289] Processing chip 220 adds a descriptor to the TX ring buffer, which records the storage location of the first read response header in shared memory and the storage address of the target data. Processing chip 220 notifies switching chip 210 to retrieve the processing result of the first read request. Switching chip 210 retrieves the descriptor from the TX ring buffer and determines the storage location of the first read response header in shared memory and the storage address of the target data based on the descriptor.

[0290] The switching chip 210 reads the header of the first read response from the shared memory according to the storage location of the header. Furthermore, the switching chip 210 obtains the target data from the storage node 310 via DMA based on the storage address of the target data. Then, the switching chip 210 generates a first read response based on the header of the first read response and the target data read from the storage node 310. When generating the first read response, the switching chip 210 generates a Layer 2 header, and encapsulates the Layer 2 header, Layer 3 header, Layer 4 header, and target data together to generate the first read response.

[0291] The switching chip 210 sends a first read response to the data access device 100.

[0292] The data access device 100 receives the first read response and obtains the target data from the first read response.

[0293] The second implementation method: the data access device 100 has already known the storage address of the target data in the storage node 310.

[0294] Data access device 100 sends a second read request to data exchange device 200, the second read request being used to request the reading of target data. The first read request carries the storage address of the target data, which is the storage address of the target data in storage node 310.

[0295] The data exchange device 200 receives the second read request. Inside the data exchange device 200, after obtaining the second read request, the switching chip 210 transmits the second read request to the processing chip 220 through shared memory.

[0296] After the processing chip 220 obtains the second read request, it parses the second read request, obtains the storage address of the target data, and generates the header of the second read response. The header generated by the processing chip 220 is a three-layer header and a four-layer header.

[0297] The processing chip 220 transmits the header of the second read response and the storage address of the target data to the switching chip 210 via shared memory. The storage address of the target data can be understood as the storage address of the payload of the second read response. The process by which the processing chip 220 transmits the header of the second read response and the storage address of the target data to the switching chip 210 via shared memory is as follows:

[0298] Processing chip 220 adds a descriptor to the TX ring buffer, which records the storage location of the second read response header in shared memory and the storage address of the target data. Processing chip 220 notifies switching chip 210 to retrieve the processing result of the second read request. Switching chip 210 retrieves the descriptor from the TX ring buffer and determines the storage location of the second read response header in shared memory and the storage address of the target data based on the descriptor.

[0299] The switching chip 210 reads the header of the second read response from the shared memory according to the storage location of the header. The switching chip 210 obtains the target data from the storage node 310 via DMA based on the storage address of the target data. The switching chip 210 generates a second read response based on the header and the target data. When generating the second read response, the switching chip 210 generates a Layer 2 header, and then encapsulates the Layer 2 header, Layer 3 header, Layer 4 header, and target data together to generate the second read response.

[0300] The switching chip 210 sends a second read response to the data access device 100.

[0301] The data access device 100 receives the second read response and obtains the target data from the second read response.

[0302] In the first implementation, the data exchange device 200 does not need to be aware of the storage address of the target data; it only needs to issue a first read request to indicate the storage of the target data. This implementation is suitable for scenarios where the data access device 100 does not need to be aware of the data distribution status in the storage cluster 300. In the second implementation, the data access device 100 has already obtained the storage address of the target data, and the data exchange device 200 can obtain the storage address of the target data from the data access device 100. This implementation is suitable for scenarios where the data access device 100 can be aware of the data distribution status in the storage cluster 300. From the two data reading implementations in the RDMA scenario, it can be seen that for the data exchange device 200, when it can handle a data read request, the target data is read internally by the switching chip 210. The processing chip 220 only needs to provide the storage address of the target data to the switching chip 210. In this approach, data read requests do not need to reach the storage cluster 300, avoiding multiple forwardings of the data read request within the storage device and effectively ensuring the reading efficiency of the target data. Furthermore, during the data reading process, the switching chip 210 inside the data exchange device 200 reads the target data from the storage nodes 310 of the storage cluster 300 and directly feeds the target data back to the data access device 100. The processing chip 220 does not need to interact with the storage nodes 310 in the storage cluster 300 to read the target data. Within the data exchange device 200, the target data is directly transmitted to the data access device 100 via forwarding by the switching chip 210. The target data does not need to be routed within the data exchange device 200, further ensuring the reading efficiency of the target data.

[0303] In the second scenario, data access device 100 and data exchange device 200 interact based on TCP / IP.

[0304] In the second case, the data writing can be implemented in the following ways:

[0305] Data access device 100 sends a third write request to data exchange device 200, which requests the writing of target data. The third write request carries the identifier of the data set (which can be a data block, file, or object) to which the target data is to be written, and the location information of the target data, indicating its position within the data set. The location information includes an offset and a data length; the offset describes the distance of the target data from the start of the data set, and the data length is the length of the target data. The third write request also carries the target data.

[0306] The data exchange device 200 receives the third write request. Inside the data exchange device 200, after the exchange chip obtains the third write request, it transmits the third write request to the processing chip 220 through shared memory.

[0307] After obtaining the third write request, the processing chip 220 parses the third write request and calculates the storage address of the target data based on the identifier of the data set in the third write request and the location information of the target data. The storage address of the target data is the storage address allocated by the processing chip 220 for the target data in the storage node 310.

[0308] The processing chip 220 stores the target data in the storage node 310 via DMA based on the storage address of the target data.

[0309] It should be noted that if the target data is large enough to exceed the Maximum Transmission Unit (MTU), the data access device 100 will split the target data into multiple sub-data, each sub-data having a data size equal to the MTU. The data access device 100 will transmit the third write request as a message stream. This message stream includes multiple messages. The first message carries the identifier of the data set to be written and the location information of the target data. The remaining messages carry one sub-data. In some scenarios, the first message may carry a small amount of data from the target data.

[0310] The switching chip 210 of the data switching device 200 receives the message stream. If the switching chip 210 only receives the first message in the message stream when forwarding it, it transmits the first message to the processing chip 220 through shared memory. After obtaining the first message, the processing chip 220 calculates the storage address of the target data based on the identifier of the data set carried in the first message and the location information of the target data.

[0311] The processing chip 220 splits the storage address of the target data into multiple sub-data storage addresses, and transmits these sub-data storage addresses to the switching chip 210 via shared memory. The process of the processing chip 220 transmitting these sub-data storage addresses to the switching chip 210 via shared memory is as follows: The processing chip 220 sequentially adds multiple descriptors to the TX ring buffer, each descriptor recording either the storage address of a sub-data or the address of a sub-data's storage address within the buffer. The processing chip 220 notifies the switching chip 210 to retrieve the processing result of the third write request. The switching chip 210 sequentially retrieves these multiple descriptors from the TX ring buffer and obtains the storage address of the sub-data based on the retrieved descriptor. The sub-data is then stored in the storage node 310 via DMA based on the storage address of the sub-data. For example, after each descriptor retrieved from the TX ring buffer and the storage address of a sub-data, the switching device stores the sub-data carried in a packet in the storage node 310 via DMA based on the storage address of the sub-data.

[0312] Processing chip 220 transmits the storage address of the target data to switching chip 210 via shared memory. The process of processing chip 220 transmitting the storage address of the target data to switching chip 210 via shared memory is similar to the process of processing chip 220 transmitting the storage addresses of multiple sub-data items to switching chip 210 via shared memory; see the foregoing description for details. The difference is that when processing chip 220 transmits the storage address of the target data to switching chip 210 via shared memory, a descriptor can be added to the TX ring buffer. This descriptor records the storage address of the target data or the address of the target data's storage address within the buffer. Processing chip 220 notifies switching chip 210 to retrieve the processing result of the third write request. Switching chip 210 retrieves the descriptor from the TX ring buffer and obtains the storage address of the target data based on the retrieved descriptor. Based on the storage address of the target data, the sub-data carried within the packet in the packet stream is stored in storage node 310 via DMA.

[0313] If, when forwarding the packet stream, the switching chip 210 receives a portion of the packets in the packet stream, and the number of packets in this portion is greater than or equal to two, including the first packet, the switching chip 210 transmits this portion of the packets to the processing chip 220 via shared memory. After obtaining this portion of the packets, the processing chip 220 calculates the storage address of the target data based on the identifier of the data set carried in the first packet and the location information of the target data.

[0314] The operations performed by the processing chip 220 after calculating the storage address of the target data can be found in the foregoing description, and will not be repeated here.

[0315] In the case where the first message carries a small amount of target data or in the case where the processing chip 220 receives a partial message, the processing chip 220 stores the small amount of data carried in the received message in the storage node 310 through DMA based on the storage address of the target data. Then, the storage addresses of the multiple sub-data generated by the processing chip 220 or the storage addresses transmitted to the switching chip 210 are the remaining storage addresses of the target data excluding the storage address of the small amount of data.

[0316] As can be seen from the implementation of data writing in a TCP / IP scenario, when the data exchange device 200 can handle data write requests instead of the storage node 310, the data write requests will not reach the storage cluster 300, effectively improving the writing efficiency of the target data. Furthermore, during the data writing process, when the target data volume is large, after the target data is transmitted to the data exchange device 200, the switching chip 210 of the data exchange device 200 can write the target data to the storage node 310 of the storage cluster 300 under the trigger of the processing chip 220. The target data (or most of the target data) can be transmitted to the storage node 310 of the storage cluster 300 through the switching chip 210 without needing to go through the processing chip 220. That is, the target data does not need to bypass the processing information, further ensuring the writing efficiency of the target data.

[0317] In the second case, the data reading can be implemented in the following ways:

[0318] Data access device 100 sends a third read request to data exchange device 200, which requests to read target data. The third read request carries the identifier of the data set (which can be a data block, file, or object) to which the target data needs to be written, and the location information of the target data, indicating the position of the target data within the data set. The location information includes an offset and a data length; the offset describes the distance of the target data from the start of the data set, and the data length is the length of the target data.

[0319] The data exchange device 200 receives the third read request. Inside the data exchange device 200, after obtaining the third read request, the exchange chip 210 transmits the third read request to the processing chip 220 through shared memory.

[0320] After the processing chip 220 obtains the third read request, it parses the third read request and calculates the storage address of the target data based on the identifier of the data set in the third read request and the location information of the target data. The storage address of the target data is the storage address allocated by the processing chip 220 for the target data in the storage node 310.

[0321] The processing chip 220 generates the header of the third read response and the storage address of the target data. The header generated by the processing chip 220 is a three-layer header and a four-layer header.

[0322] Processing chip 220 transmits the header of the third read response and the storage address of the target data to switching chip 210 via shared memory. The storage address of the target data can be understood as the storage address of the payload of the third read response. The process by which processing chip 220 transmits the header of the third read response and the storage address of the target data to switching chip 210 via shared memory is as follows:

[0323] Processing chip 220 adds a descriptor to the TX ring buffer, which records the storage location of the third read response header in shared memory and the storage address of the target data. Processing chip 220 notifies switching chip 210 to retrieve the processing result of the third read request. Switching chip 210 retrieves the descriptor from the TX ring buffer and determines the storage location of the third read response header in shared memory and the storage address of the target data based on the descriptor.

[0324] The switching chip 210 reads the header of the third read response from the shared memory according to the storage location of the header. The switching chip 210 then retrieves the target data from the storage node 310 via DMA based on the storage address of the target data. The switching chip 210 generates a third read response based on the header and the target data. When generating the third read response, the switching chip 210 generates a Layer 2 header and encapsulates the Layer 2, Layer 3, and Layer 4 headers, along with the target data, together to generate the third read response.

[0325] The switching chip 210 sends a third read response to the data access device 100.

[0326] The data access device 100 receives the third read response and obtains the target data from the third read response.

[0327] The implementation method of data reading in the TCP / IP scenario is similar to the first data reading method in the RDMA scenario. The beneficial effects that can be achieved can be found in the previous description, and will not be repeated here.

[0328] Based on the same inventive concept as the method embodiments, this application also provides a data exchange device for executing the method performed by the data exchange device 200 in the above method embodiments. As shown in FIG7, the data exchange device 700 includes a forwarding module 701 and an access module 702. Optionally, it also includes a data module 703, wherein the forwarding module 701 and the data module can be deployed in the switching chip 210, and the access module 702 is deployed in the processing chip 220. Specifically, in the data exchange device 700, the modules are connected through a communication path.

[0329] Forwarding module 701 is used to receive data access requests, which are used to request access to target data in the storage cluster.

[0330] Access module 702 is used to determine the first storage node in the storage cluster to which the target data belongs based on the data access request, and to access the target data in the first storage node.

[0331] In one possible implementation, the data exchange device 700 is connected to the storage cluster via a system bus, and the access module 702 accesses the target data in the first storage node via the system bus.

[0332] In one possible implementation, the forwarding module 701 receives a connection establishment request, which is used to request the establishment of a connection with the storage cluster, and detects the destination address carried in the connection establishment request. If the forwarding module 701 detects that the destination address carried in the connection establishment request is the address of the storage cluster, the access module 702 generates and sends back a positive connection establishment response.

[0333] As one possible implementation, the access module 702 performs operations on the target metadata in the storage cluster.

[0334] In one possible implementation, the forwarding module 701 receives a metadata operation request, which is used to perform operations on target metadata in the storage cluster. The forwarding module 701 then sends the metadata operation request to the access module 702.

[0335] As one possible implementation, the operations performed on the target metadata are some or all of the following:

[0336] Modify, query, read, and generate operations.

[0337] As one possible implementation, the access module 702 performs operations on the target metadata cached in the data exchange device 700.

[0338] In one possible implementation, the data exchange device 700 includes a switching chip and a processing chip. A forwarding module 701 is deployed on the switching chip and transmits the received data access request to the processing chip.

[0339] As one possible implementation, the data exchange device 700 further includes a data module 703 deployed on the exchange chip and an access module 702 deployed on the processing chip. When accessing target data in the first storage node, the access module 702 obtains the storage address of the target data and transmits the storage address of the target data to the data module 703 in the exchange chip.

[0340] Data module 703 accesses the target data in the first storage node based on the storage address of the target data.

[0341] As one possible implementation, the data module 703 is connected to the storage cluster via the system bus, such as being connected to each storage node in the storage cluster via the system bus. When the data module 703 accesses the target data in the first storage node based on the storage address of the target data, it accesses the target data in the first storage node via the system bus and based on the storage address of the target data.

[0342] As one possible implementation, when the data module 703 accesses the target data in the first storage node via the system bus and based on the storage address of the target data, it reads the target data from the storage address of the target data in the first storage node via the system bus; or, it writes the target data to the storage address of the target data in the first storage node via the system bus.

[0343] As one possible implementation, when the forwarding module 701 sends a data access request to the processing chip, it removes the Layer 2 header from the data access request and transmits the data access request with the Layer 2 header removed to the processing chip.

[0344] In one possible implementation, the access module 702 generates a data access response header and transmits the data access response header to the data module 703 in the switching chip.

[0345] The data module 703 in the switching chip obtains the header of the data access response from the processing chip. The data access response is a response to the data access request. The data module 703 obtains the payload of the data access response (such as using the target data it reads as the payload or obtaining the payload from the processing chip). The transmission module completes the data access response encapsulation according to the header and payload of the data access response and feeds back the data access response.

[0346] As one possible implementation, a data access request is used to read target data, and the payload of the data access response includes the target data.

[0347] In one possible implementation, a data access request is used to write target data. The data module 703 obtains a payload from the access module 702, and the payload indicates whether the target data writing was successful or failed.

[0348] The module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0349] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a terminal device (which may be a personal computer, mobile phone, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0350] The descriptions of the processes corresponding to the above-mentioned figures each have their own emphasis. For parts of a process that are not described in detail, please refer to the relevant descriptions of other processes.

[0351] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When software implementation is used...

Claims

1. A data access method, characterized by, The method is applied to a data exchange device, and the method includes: Receive a data access request, the data access request being used to request access to target data in the storage cluster; Based on the data access request, determine the first storage node in the storage cluster to which the target data belongs, and access the target data in the first storage node.

2. The method of claim 1, wherein, The data exchange device is connected to the storage cluster via a system bus. Accessing the target data in the first storage node includes: The target data in the first storage node is accessed through the system bus.

3. The method of claim 1 or 2, wherein, Before receiving the data access request, the method further includes: Receive a connection establishment request, the connection establishment request being used to request the establishment of a connection with the storage cluster; If the destination address carried in the connection establishment request is detected to be the address of the storage cluster, a positive connection establishment response is generated and fed back.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Perform operations on the target metadata in the storage cluster.

5. The method of claim 4, wherein, Before performing operations on the target metadata in the storage cluster, the method further includes: Receive metadata operation requests, which are used to perform operations on target metadata in the storage cluster.

6. The method of claim 4 or 5, wherein, The operations performed on the target metadata are some or all of the following: Modify, query, read, and generate operations.

7. The method of claim 6, wherein, The operation performed on the target metadata in the storage cluster includes: Perform operations on the target metadata cached in the data exchange device.

8. The method according to any one of claims 1 to 7, characterized in that, The data exchange device includes a switching chip and a processing chip, and the receiving of data access requests includes: The switching chip receives the data access request; The switching chip sends the data access request to the processing chip.

9. The method of claim 8, wherein, Accessing the target data in the first storage node includes: The processing chip obtains the storage address of the target data; The processing chip transmits the storage address of the target data to the switching chip; The switching chip accesses the target data in the first storage node based on the storage address of the target data.

10. The method of claim 9, wherein, The switching chip is connected to the storage cluster via a system bus. The switching chip accesses the target data in the first storage node based on the storage address of the target data, including: The switching chip accesses the target data in the first storage node through the system bus and based on the storage address of the target data.

11. The method of claim 10, wherein, The switching chip accesses the target data in the first storage node via the system bus and based on the storage address of the target data, including: The switching chip reads the target data from the storage address of the target data in the first storage node via the system bus; or The switching chip writes the target data to the storage address of the target data in the first storage node via the system bus.

12. The method of claim 8, wherein, The switching chip sends the data access request to the processing chip, including: The switching chip removes the Layer 2 header from the data access request and transmits the data access request with the Layer 2 header removed to the processing chip.

13. The method according to any one of claims 8 to 12, characterized in that, The method further includes: The switching chip obtains the header of the data access response from the processing chip, and the data access response is a response to the data access request; The switching chip acquires the payload of the data access response; The switching chip encapsulates the data access response based on the header and payload of the data access response and then feeds back the data access response.

14. The method of claim 13, wherein, The data access request is used to read the target data, and the payload of the data access response includes the target data.

15. The method of claim 13, wherein, The data access request is used to write the target data, and the interaction chip obtains the payload of the data access response, including: The switching chip obtains the payload from the processing chip, and the payload indicates whether the target data was successfully or unsuccessfully written.

16. A data exchange device, characterized by The device includes: The forwarding module is used to receive data access requests, which are requests to access target data in the storage cluster. The access module is used to determine the first storage node in the storage cluster to which the target data belongs based on the data access request, and to access the target data in the first storage node.

17. The apparatus of claim 16, wherein, The data exchange device is connected to the storage cluster via a system bus, and the access module is used for: The target data in the first storage node is accessed through the system bus.

18. The apparatus as claimed in claim 16 or 17, characterized in that, The forwarding module is further configured to: receive a connection establishment request, the connection establishment request being used to request the establishment of a connection with the storage cluster; and detect the destination address carried in the connection establishment request. The access module is further configured to: generate and send back a positive connection establishment response when the forwarding module detects that the destination address carried in the connection establishment request is the address of the storage cluster.

19. The apparatus of any one of claims 16-18, wherein, The access module is also used for: Perform operations on the target metadata in the storage cluster.

20. The apparatus of claim 19, wherein, The operations performed on the target metadata are some or all of the following: Modify, query, read, and generate operations.

21. A data switching device, characterized by The device includes: The switching chip is used to: receive a data access request, the data access request being used to request access to target data in the storage cluster; The processing chip is used to: determine the first storage node in the storage cluster to which the target data belongs based on the data access request, and access the target data in the first storage node.

22. The apparatus of claim 21, wherein, The data exchange device is connected to the storage cluster via a system bus, and the processing chip is used for: The switching chip is triggered to access the target data in the first storage node through the system bus.

23. The device as claimed in claim 21 or 22, characterized in that, The switching chip is further configured to: receive a connection establishment request, the connection establishment request being used to request the establishment of a connection with the storage cluster, and determine that the destination address carried in the connection establishment request is the address of the storage cluster; The processing chip is also used to: generate and establish a positive response by feeding back the connection through the switching chip.

24. The apparatus of any one of claims 21-23, wherein, The processing chip is also used for: Perform operations on the target metadata in the storage cluster.

25. The device as claimed in claim 24, characterized in that, The operations performed on the target metadata are some or all of the following: Modify, query, read, and generate operations.

26. The device according to any one of claims 21 to 25, characterized in that, The switching chip is also used to send the data access request to the processing chip.

27. The device as claimed in claim 26, characterized in that, The processing chip is used to: obtain the storage address of the target data; and transmit the storage address of the target data to the switching chip. The switching chip is used to access the target data in the first storage node based on the storage address of the target data.

28. The device as claimed in claim 27, characterized in that, The switching chip is connected to the storage cluster via a system bus, and the switching chip is used for: The target data in the first storage node is accessed via the system bus and based on the storage address of the target data.

29. The device as claimed in claim 26, characterized in that, The switching chip is used for: Remove the Layer 2 header from the data access request and transmit the data access request with the Layer 2 header removed to the processing chip.

30. A computer-readable storage medium, characterized in that, When the computer-readable storage medium is executed by a computing device, the computing device performs the method according to any one of claims 1 to 15.

Images