Data processing method and related device

Abstract

The present application relates to the field of storage. Disclosed are a data processing method and a related device. The method comprises: a switch device in a distributed storage system receiving first information sent by a client, wherein the first information is used for requesting access to first data from a first server among N servers, N being a positive integer greater than 1; sending target information to a second server, wherein the target information is determined on the basis of the first information, the target information is used for requesting access to the first data from the second server, the second server is a server selected from among (N-1) servers other than the first server, the target information comprises a data packet sequence number, the data packet sequence number matches state information of the second server, and the state information of the second server is used for indicating information that the second server plans to receive next time; receiving the first data from the second server; and sending the first data to the client. The present application is used for effectively improving the operation performance of the distributed storage system.

Description

Data processing method and related device

[0001] The present application claims priority to the Chinese patent application No. 202411296436.X, filed on September 14, 2024, with the State Intellectual Property Office of China, and entitled "Data processing method and related device", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of storage, and in particular to a data processing method and related device. BACKGROUND

[0003] Distributed storage is a method for improving data storage and access reliability by dispersing data storage on multiple servers. A traditional distributed storage system often includes a client, a switch, and multiple servers (e.g., server F1, server F2, and server F3). When using a traditional protocol to access data, the client needs to send a special message (e.g., for initiating a TCP connection) to the switch. After receiving the special message, the switch selects one of the aforementioned three servers (e.g., server F1) as a server to be connected and sends the special message to server F1. When server F1 agrees to establish a connection with the client, it sends a connection notification to the switch. After the switch sends the connection notification to the client, the client and server F1 are determined to have successfully established a connection. Then, the messages generated by the client can be sent to server F1 through the connection.

[0004] However, once the switch needs to send the message sent by the client to another server (e.g., server F2), the client and server F2 need to repeat the above process to establish a connection. The entire process requires a large amount of server resources and transmission bandwidth, causing a heavy burden on the central processing unit (CPU) of the server.

[0005] How to improve the running performance of distributed storage is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0006] The present application provides a data processing method and related device, which can effectively improve the running performance of a distributed storage system.

[0007] To achieve the above-mentioned purpose, the present application adopts the following technical solutions.

[0008] In a first aspect, an embodiment of the present application provides a data processing method applied to a switching device in a distributed storage system, the distributed storage system further comprising a client and N servers, N being a positive integer greater than 1, the method comprising: receiving first information sent by the client, the first information being used to request to access first data from a first server of the N servers; sending target information to a second server, the target information being determined based on the first information, the target information being used to request to access the first data from the second server, the second server being a server selected from the (N-1) servers other than the first server, the target information comprising a packet sequence number, the packet sequence number matching state information of the second server, the state information of the second server being used to indicate information that the second server plans to receive next time; receiving the first data from the second server; and sending the first data to the client.

[0009] For example, the target information herein refers to information sent by the switching device and compatible with a network protocol of the distributed storage system, wherein the target information is determined based on the first information, and it can be understood that the target information herein can be the first information when the packet sequence number of the first information matches the state information of the second server, and the target information herein can be obtained by modifying the first information when the packet sequence number of the first information does not match the state information of the second server, for example, the packet sequence number of the first information can be modified, or the packet sequence number of the first information can be modified while at least one of some specific fields (for example, a target address) and a check parameter in the first information is modified, which will not be exemplified one by one herein.

[0010] In the above scheme, the information received by the switching device is the first information sent by the client, and the information sent by the switching device is the target information, and the packet sequence number in the target information matches the state information of the second server, which means that the switching device in the embodiment of the present application can not only provide the function of information forwarding, but also can provide the function of forwarding compatible with the network protocol, that is, the switching device can modify the received information, so that the information sent can be successfully received by the second server, and in the whole data access process, even if the access object of the client is changed from the first server to the second server, the client and the server do not need to establish a new connection through multiple transmissions, but the switching device can modify the information directly, so that the resources can be effectively saved and the transmission bandwidth can be reduced, and the performance of the distributed storage system can be improved.

[0011] In a possible implementation, the packet sequence number in the target information is obtained by modifying the packet sequence number in the first information.

[0012] In the scheme, if the data packet sequence number in the first information does not match the state information of the second server, it means that the first information cannot be successfully forwarded to the second server, at this time, the switching device changes the data packet sequence number in the first information, so that the target information finally sent can be successfully received by the second server. Therefore, the switching device not only has the function of information forwarding, but also has the function of editing and can realize the forwarding compatible with the network protocol of the distributed storage system.

[0013] In a possible implementation, the switching device stores a connection relationship, and the connection relationship is used to indicate that the client and the second server establish a remote direct memory access (RDMA) connection.

[0014] For example, the RDMA connection is an efficient network communication technology. Unlike the traditional network communication mode, the RDMA allows a machine in the network to directly read and write the memory of another machine.

[0015] In the scheme, the establishment of the RDMA connection between the client and the server means that the distributed storage system can use the new communication protocol RDMA, so as to realize the information forwarding compatible with the RDMA in the switching device, and then the load balancing can be realized on the premise of ensuring consistency. In the whole data access process, the operating system and CPU do not need to intervene too much, and high performance can still be maintained in the face of highly skewed workloads.

[0016] In a possible implementation, the switching device is configured to receive and record the state information of the second server, and the state information of the second server includes a data packet sequence number.

[0017] For example, the switching device can periodically receive the state information reported by the second server, or can actively obtain the state information of the second server. Here, it will not be limited.

[0018] In the scheme, the switching device can record the state information of the second server in advance, instead of requesting the state information of the second server when receiving the first information of the client, so that the time for the switching device to generate the target information can be effectively reduced, the target information can be quickly sent to the second server, and the access efficiency of the first data can be effectively improved.

[0019] In a possible implementation, the method further includes: obtaining a data type of the first data; if the data type of the first data is a first type, performing the step of sending the target information, and the first type is used to indicate that the data is stored in a replication mode.

[0020] In a possible implementation, the method further includes: if the data type of the first data is a second type, sending the first information to the first server, the second type being used to indicate that the data is stored in a way of erasure coding; receiving the first data from the first server, and sending the first data to the client.

[0021] In the above solution, the switching device needs to implement compatible forwarding according to the data type of the first data, which means that the access object of the client will be different if the data types are different. As can be seen, the distributed storage system includes not only data stored in a way of replication, but also data stored in a way of erasure coding. Such a mixed storage strategy of replication and erasure coding can greatly reduce the use of memory while providing fault tolerance. Implementing compatible forwarding in the switching device can effectively improve the running performance of the distributed storage system.

[0022] In a possible implementation, the first information includes first keyword information; the method further includes: performing in-network conflict detection based on the first keyword information and second keyword information of M ongoing target requests, to obtain a detection result, M being a positive integer; if the detection result indicates that no conflict request corresponding to the first information is detected, performing the step of sending the target information to the second server.

[0023] For example, the conflict request corresponding to the first information can be used to update the first data in the distributed storage system.

[0024] In the above solution, when the switching device receives the first information used to access the first data, the switching device does not directly send the target information to the second server, but first performs in-network conflict detection, thereby reducing the possibility of obtaining incorrect data. This way not only can effectively improve the running performance of the distributed storage system, but also can improve the accuracy of data access and better implement data management.

[0025] In a possible implementation, the method further includes: if the conflict request corresponding to the first information is detected, sending the first information to the first server; receiving update data from the first server, the update data being data obtained after the first data is updated based on the conflict request corresponding to the first information; and sending the update data to the client.

[0026] In the scheme, if other servers are not synchronized to the updated first data, and the exchange device forwards the first information to other servers, the exchange device will obtain incorrect data (the first data that is not updated) from other servers. Therefore, when detecting the conflict request corresponding to the first information, the exchange device directly sends the first information to the first server to receive the updated data returned by the first server, which can effectively ensure the access efficiency and improve the access accuracy.

[0027] In a possible implementation, the method further includes: receiving second information sent by the client, the second information being used to request writing second data in the first server; and storing a target request corresponding to the second information, the target request corresponding to the second information being used for in-network conflict detection.

[0028] In the scheme, the exchange device can also implement the target request (for example, a put request) of writing data by using the network protocol of the distributed storage system. The exchange device stores the target request, rather than following up in real time when performing in-network conflict detection, which can effectively improve the detection efficiency and further improve the operation performance of the distributed storage system.

[0029] In a possible implementation, the exchange device stores a state identifier corresponding to the target request, the state identifier being a first state identifier or a second state identifier, the first state identifier being used to indicate that the target request is being processed, and the second state identifier being used to indicate that the target request has been completed.

[0030] In the scheme, the state identifiers can be used to determine whether each target request is completed. The exchange device stores the state identifier corresponding to the target request, and can quickly determine the target request being processed when performing in-network conflict detection in the future, thereby improving the detection efficiency.

[0031] In a possible implementation, an RDMA connection is established between the client and the intelligent network card corresponding to the second server, the target address in the first information is the first keyword information; the target information is sent to the second server, including: the target information is sent to the intelligent network card corresponding to the second server, the target address of the target information is the first keyword information, the intelligent network card corresponding to the second server is used to send access information to the second server, the target address in the access information is index information corresponding to the first keyword information; and the first data from the second server is received, including: the first data sent by the intelligent network card corresponding to the second server is received.

[0032] For example, the index information can be used to indicate the address of the first data in the host memory of the second server, that is, the intelligent network card corresponding to the second server can directly obtain the first data from the host memory of the second server without the server CPU.

[0033] In the above solution, the first information is used to access the first request from the first server, and the RDMA connection is established between the client and the intelligent network card corresponding to the second server, which means that the embodiment of the application can use the switching device and the intelligent network card to realize the access request (for example, the get request) corresponding to the first information, and the entire access process can bypass the CPU of the second server, which can effectively reduce the CPU burden of the second server and improve the running performance of the distributed storage system.

[0034] In a possible implementation, the target address of the target information is index information corresponding to the first keyword information, and the first keyword information is the target address of the first information.

[0035] In the above solution, the target address of the target information is the index information, which means that the switching device has an editable capability and can change the target address of the first information to unload the computing task of the second server, directly obtain the first data from the memory of the second server, bypass the CPU of the second server, and effectively reduce the CPU burden of the second server and improve the running performance of the distributed storage system.

[0036] In a possible implementation, the check parameter in the target information is different from the check parameter in the first information.

[0037] For example, the check parameter in the target information and the check parameter in the first information are both used to ensure the integrity in the network transmission process.

[0038] In the above solution, the target address of the target information is the index information corresponding to the first keyword information, and the target address of the first information is the first keyword information, which means that the switching device can update the target address of the first information after receiving the first information, which will cause the content of the first information to change. At this time, the check parameter in the first information cannot ensure the integrity of the updated first information in the data transmission process, therefore, the switching device needs to recalculate the check parameter, for example, using a certain check algorithm to recalculate the check parameter of the updated first information, and finally obtain the target information sent to the second server, thereby effectively improving the reliability of the network transmission.

[0039] In a second aspect, the embodiments of the present application provide a data processing method, applied to a target intelligent network card in a distributed storage system, the distributed storage system comprising a client, a switching device, N servers, and an intelligent network card corresponding to each of the N servers, a second server corresponding to the target intelligent network card being one of the N servers, N being a positive integer greater than 1; the method comprising: receiving target information sent by the switching device, the target information being used to request access to first data from the second server, the target information comprising a data packet sequence number, the data packet sequence number matching state information of the second server, the state information of the second server being used to indicate information that the second server plans to receive next time, a target address of the target information being first keyword information; sending access information to the second server, a target address in the access information being index information corresponding to the first keyword information; receiving the first data from the second server; and sending the first data to the switching device.

[0040] In the above scheme, through the joint design of the target intelligent network card and the switching device, the performance of the distributed storage system can be improved through compatible forwarding of the switching device, and the read operation of the server can be further offloaded by the target intelligent network card to reduce the CPU burden of the second server, thereby achieving load balancing, fault tolerance, and strong consistency while saving resources.

[0041] In a possible implementation, the target intelligent network card and the client establish an RDMA connection.

[0042] In the above scheme, the target intelligent network card and the client establish an RDMA connection, which means that the distributed storage system can use the new communication protocol RDMA, so that the information forwarding compatible with RDMA can be realized in the switching device, and the operating system and CPU do not need to intervene too much in the entire data access process, thereby improving the running performance of the distributed storage system.

[0043] In a possible implementation, the method further comprises: receiving state information sent by the second server; and sending the state information to the switching device.

[0044] In the above scheme, the state information of the second server is sent to the switching device by the target intelligent network card, so that the switching device can realize compatible forwarding when changing the access object of the client to the second server, thereby improving the running performance of the distributed storage system.

[0045] In a possible implementation, the check parameter in the access information is different from the check parameter in the target information.

[0046] In the scheme, the target address of the target information is the first keyword information, and the target address of the access information is the index information corresponding to the first keyword, which means that the target intelligent network card needs to update the target address of the target information from the first keyword information to the index information corresponding to the first keyword information after receiving the target information, that is, the target intelligent network card can update the target address of the target information, which will cause the content of the target information to change. At this time, the check parameter in the target information cannot ensure the integrity of the updated target information in the data transmission process, therefore, the target intelligent network card needs to recalculate the check parameter to obtain the target information sent to the second server, thereby effectively improving the reliability of data transmission.

[0047] In a possible implementation, the target intelligent network card includes a first component and a second component, the first component is configured to update the target address of the target information, and the second component is configured to update the check parameter of the target information.

[0048] In the scheme, the target intelligent network card can offload different computing tasks of the second server through two components, that is, the first component is used to change the target address and other specific domains, and the second component is used to recalculate the check parameter. Compared with offloading by one component, two components can better improve the throughput to improve the system performance.

[0049] In a possible implementation, the target intelligent network card includes a third component, the third component is configured to update the target address of the target information, and the third component is further configured to update the check parameter of the target information.

[0050] In the scheme, the target intelligent network card can offload the computing task of the second server through one component (i.e., the third component), release more server resources, effectively reduce the CPU burden of the second server, and improve the running performance of the distributed storage system.

[0051] In a third aspect, an embodiment of the present application provides a data processing method applied to a second server in a distributed storage system, the distributed storage system including a client, a switching device, and N servers, the second server being determined by the switching device from the N servers, N being a positive integer greater than 1; the method further includes: receiving access information through the switching device, the access information including a data packet sequence number, the data packet sequence number matching state information of the second server, the state information of the second server being used to indicate information planned to be received by the second server next time; and sending first data to the switching device, the first data being obtained from a storage space of the second server based on the access information.

[0052] In the scheme, when the second server receives the access information through the switching device, the first data can be directly obtained from the storage space of the second server without too much intervention of the operating system and the CPU, so that the CPU occupation of the second server is reduced. In addition, the data packet sequence number in the access information matches the state information of the second server, which means that the access information received by the second server is reliable information, and the second server can work normally in the case of retransmission and out-of-order, so that the operation performance of the distributed storage system is effectively improved.

[0053] In a possible implementation, the RDMA connection is established between the second server and the client.

[0054] In the scheme, the RDMA connection is established between the client and the second server, which means that the distributed storage system can use the new communication protocol RDMA, and when the second server receives the access information, the CPU of the second server can be bypassed, so that the CPU burden of the second server is effectively reduced, and the operation performance of the distributed storage system is improved.

[0055] In a possible implementation, the method further includes: sending the state information to the switching device.

[0056] In the scheme, the second server sends the state information to the switching device, so that when the switching device changes the access object of the client to the second server, compatible forwarding is realized without the need for the client and the second server to establish a new connection through multiple transmissions, so that the operation performance of the distributed storage system is improved.

[0057] In a possible implementation, the RDMA connection is established between the smart NIC corresponding to the client and the smart NIC corresponding to the second server, and the smart NIC corresponding to the second server is configured to generate the access information; and the state information is sent to the switching device, including: the state information is sent to the smart NIC corresponding to the second server, and the smart NIC corresponding to the second server is configured to send the state information to the switching device.

[0058] In the scheme, the server sends the state information through the smart NIC corresponding to the second server, which means that the distributed storage system at this time also includes the smart NIC corresponding to the second server. The smart NIC is configured to generate the access information, which means that the computing task of the second server is offloaded, more server resources are released, the CPU burden of the second server is effectively reduced, and the operation performance of the distributed storage system is improved.

[0059] In a possible implementation, the target address in the access information is index information, and the index information is used to represent the storage address of the first data in the second server.

[0060] Exemplarily, the index information can be used to represent the address of the first data in the host memory of the second server, that is, the first data can be directly obtained from the storage space (for example, the host memory) of the second server, and the whole process bypasses the server CPU, effectively reducing the CPU burden of the second server and improving the operation performance of the distributed storage system.

[0061] In a possible implementation, the check parameter in the access information is different from the check parameter in the first information, and the first information is the information generated by the client for accessing the first data from the first server of the N servers.

[0062] In the above scheme, the check parameter of the access information received by the second server is different from the check parameter of the first information generated by the client, which means that the first information sent by the client has changed in the transmission process of the distributed storage system, that is, there is a network device (for example, a switching device or an intelligent network card) in the distributed storage system that can compatibly forward the first information, in other words, when the access object of the client is changed from the first server to the second server, the client and the second server do not need to transmit multiple times and re-establish a new connection, but the second server can directly and successfully receive the access information, thereby effectively saving resources and reducing transmission bandwidth and improving the operation performance of the distributed storage system.

[0063] In a fourth aspect, an embodiment of the present application provides a switching device, and a distributed storage system in which the switching device is located further includes a client and N servers, N is a positive integer greater than 1; the switching device comprises:

[0064] a transceiving unit configured to receive first information sent by the client, the first information being used to request to access first data from a first server of the N servers; the transceiving unit is configured to send target information to a second server, the target information being determined based on the first information, the target information being used to request to access the first data from the second server, the second server being a server selected from the (N-1) servers other than the first server, the target information including a data packet sequence number, the data packet sequence number matching state information of the second server, the state information of the second server being used to indicate information that the second server plans to receive next time; the transceiving unit is further configured to receive the first data from the second server; and the transceiving unit is further configured to send the first data to the client.

[0065] In a possible implementation, the data packet sequence number in the target information is obtained by modifying the data packet sequence number in the first information.

[0066] In a possible implementation, the switching device stores a connection relationship, and the connection relationship is used to indicate that a remote direct memory access (RDMA) connection is established between the client and the second server.

[0067] In a possible implementation, the exchange device is configured to receive and record the state information of the second server, and the state information of the second server includes a packet sequence number.

[0068] In a possible implementation, the exchange device further includes a processing unit configured to obtain a data type of the first data, and the processing unit is further configured to perform the step of sending the target information if the data type of the first data is a first type, and the first type is used to indicate that the data is stored in a replication manner.

[0069] In a possible implementation, the transceiver unit is further configured to send the first information to the first server if the data type of the first data is a second type, and the second type is used to indicate that the data is stored in an erasure code manner; the transceiver unit is further configured to receive the first data from the first server; and the transceiver unit is further configured to send the first data to the client.

[0070] In a possible implementation, the first information includes first keyword information, and the exchange device further includes a processing unit configured to perform in-network conflict detection based on the first keyword information and second keyword information of M ongoing target requests, to obtain a detection result, where M is a positive integer; and the processing unit is further configured to perform the step of sending the target information to the second server if the detection result indicates that no conflict request corresponding to the first information is detected.

[0071] In a possible implementation, the transceiver unit is further configured to send the first information to the first server if the conflict request corresponding to the first information is detected; the transceiver unit is further configured to receive update data from the first server, where the update data is obtained by performing update processing on the first data based on the conflict request corresponding to the first information; and the transceiver unit is further configured to send the update data to the client.

[0072] In a possible implementation, the transceiver unit is further configured to receive second information sent by the client, and the second information is used to request writing second data in the first server; and the processing unit is further configured to store a target request corresponding to the second information, and the target request corresponding to the second information is used to perform in-network conflict detection.

[0073] In a possible implementation, the exchange device stores a state identifier corresponding to the target request, and the state identifier is a first state identifier or a second state identifier, where the first state identifier is used to indicate that the target request is ongoing, and the second state identifier is used to indicate that the target request is completed.

[0074] In a possible implementation, the RDMA connection is established between the client and the intelligent network card corresponding to the second server, and the target address in the first information is the first keyword information; the transceiving unit is further configured to send target information to the intelligent network card corresponding to the second server, the target address of the target information being the first keyword information, the intelligent network card corresponding to the second server being configured to send access information to the second server, the target address in the access information being the index information corresponding to the first keyword information; and the receiving unit is further configured to receive the first data sent by the intelligent network card corresponding to the second server.

[0075] In a possible implementation, the target address of the target information is the index information corresponding to the first keyword information, and the first keyword information is the target address of the first information.

[0076] In a possible implementation, the check parameter in the target information is different from the check parameter in the first information.

[0077] In a fifth aspect, an embodiment of the present application provides a target intelligent network card, a distributed storage system in which the target intelligent network card is located including a client, a switching device, N servers, and an intelligent network card corresponding to each of the N servers, a second server corresponding to the target intelligent network card belonging to the N servers, N being a positive integer greater than 1; the target intelligent network card comprising:

[0078] The transceiving unit is configured to receive target information sent by the switching device, the target information being used to request access to first data from the second server, the target information including a data packet sequence number, the data packet sequence number matching state information of the second server, the state information of the second server being used to indicate information that the second server plans to receive next time, and a target address of the target information being first keyword information; the transceiving unit is configured to send access information to the second server, a target address in the access information being index information corresponding to the first keyword information; the transceiving unit is further configured to receive the first data from the second server; and the transceiving unit is further configured to send the first data to the switching device.

[0079] In a possible implementation, the RDMA connection is established between the target intelligent network card and the client.

[0080] In a possible implementation, the transceiving unit is further configured to receive state information sent by the second server; and the transceiving unit is further configured to send the state information to the switching device.

[0081] In a possible implementation, the check parameter in the access information is different from the check parameter in the target information.

[0082] In a possible implementation, the target intelligent network card includes a first component and a second component, the first component being configured to update the target address of the target information, and the second component being configured to update the check parameter of the target information.

[0083] In a possible implementation, the target intelligent network card comprises a third component configured to update a target address of the target information, and the third component is further configured to update a check parameter of the target information.

[0084] In a sixth aspect, an embodiment of the present application provides a second server, a distributed storage system in which the second server is located comprises a client, a switching device and N servers, the second server is determined by the switching device from the N servers, and N is a positive integer greater than 1; the second server comprises:

[0085] The transceiver is configured to receive access information from the switching device, the access information comprises a data packet sequence number, the data packet sequence number matches state information of the second server, and the state information of the second server is used to indicate information that the second server plans to receive next time; and the transceiver is further configured to send first data to the switching device, the first data being obtained from a storage space of the second server based on the access information.

[0086] In a possible implementation, the second server and the client establish an RDMA connection.

[0087] In a possible implementation, the transceiver is further configured to send the state information to the switching device.

[0088] In a possible implementation, the client and the intelligent network card corresponding to the second server establish an RDMA connection; and the transceiver is further configured to send the state information to the intelligent network card corresponding to the second server, and the intelligent network card corresponding to the second server is configured to send the state information to the switching device.

[0089] In a possible implementation, the target address in the access information is index information, and the index information is used to represent a storage address of the first data in the second server.

[0090] In a possible implementation, the check parameter in the access information is different from a check parameter in first information, and the first information is generated by the client and is used to access the first data from a first server of the N servers.

[0091] In a seventh aspect, an embodiment of the present application provides a distributed storage system, which comprises various modules used to execute the data processing method in the first aspect or any possible implementation of the first aspect, and the third aspect or any possible implementation of the third aspect.

[0092] The distributed storage system has the functions of implementing the methods in the first aspect and the third aspect. The functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions. In a possible design, the distributed storage system includes a client, a switching device, and N servers, where N is a positive integer greater than 1. The client sends first information to the switching device, where the first information is used to request to access first data from a first server of the N servers. The switching device sends target information to a second server, where the target information is determined based on the first information, the target information is used to request to access the first data from the second server, the second server is selected from (N-1) servers other than the first server, and the target information includes a packet sequence number, which matches state information of the second server, and the state information of the second server is used to indicate information that the second server plans to receive next time. The switching device receives the first data from the second server. The switching device sends the first data to the client.

[0093] In an eighth aspect, an embodiment of the present application provides a distributed storage system, which includes various modules for performing the data processing method in the first aspect or any possible implementation manner of the first aspect, the second aspect or any possible implementation manner of the second aspect, and the third aspect or any possible implementation manner of the third aspect.

[0094] The distributed storage system has the functions of implementing the methods in the first aspect, the second aspect, and the third aspect. The functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions. In a possible design, the distributed storage system includes a client, a switching device, N servers, and an intelligent network card corresponding to each server, where N is a positive integer greater than 1. The client sends first information to the switching device, where the first information is used to request to access first data from a first server of the N servers. The switching device sends target information to a target intelligent network card, where the target intelligent network card is an intelligent network card corresponding to a second server of the N servers, the target information includes a packet sequence number, which matches state information of the second server, the state information of the second server is used to indicate information that the second server plans to receive next time, and a target address of the target information and a target address of the first information are both keyword information. The target intelligent network card sends access information to the second server, where a target address in the access information is index information corresponding to the keyword information. The target intelligent network card receives the first data from the second server. The target intelligent network card sends the first data to the switching device. The switching device sends the first data to the client.

[0095] In a ninth aspect, an embodiment of the present application provides a computing device, comprising a memory and a processor, the memory being configured to store computer instructions, and the processor being configured to invoke and run the computer instructions from the memory to implement the method according to the first aspect or any implementation manner of the first aspect, or to implement the method according to the second aspect or any implementation manner of the second aspect, or to implement the method according to the third aspect or any implementation manner of the third aspect.

[0096] In a tenth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium storing instructions, when the instructions are run on a processor, to implement the method according to the first aspect or any implementation manner of the first aspect, or to implement the method according to the second aspect or any implementation manner of the second aspect, or to implement the method according to the third aspect or any implementation manner of the third aspect.

[0097] In an eleventh aspect, an embodiment of the present application provides a computer program product, the computer program product comprising instructions, when the instructions are run on a processor, to implement the method according to the first aspect or any implementation manner of the first aspect, or to implement the method according to the second aspect or any implementation manner of the second aspect, or to implement the method according to the third aspect or any implementation manner of the third aspect.

[0098] The technical effects brought by the above-mentioned second aspect to eleventh aspect and any implementation manner of each aspect can refer to the first aspect and the corresponding implementation manner of the first aspect, and the repeated parts will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0099] FIG. 1a is a system architecture diagram corresponding to a distributed storage system according to an embodiment of the present application;

[0100] FIG. 1b is an architecture diagram corresponding to the memory of a server host according to an embodiment of the present application;

[0101] FIG. 2 is a system architecture diagram corresponding to another distributed storage system according to an embodiment of the present application;

[0102] FIG. 3 is a scene schematic diagram of a switching device on a data plane according to an embodiment of the present application;

[0103] FIG. 4 is a method schematic diagram for data processing according to an embodiment of the present application;

[0104] FIG. 5 is a scene schematic diagram for compatible forwarding according to an embodiment of the present application;

[0105] FIG. 6 is an interaction diagram one for data access according to an embodiment of the present application;

[0106] FIG. 7 is an interaction diagram II for data access according to an embodiment of the present application;

[0107] FIG. 8 is an interaction diagram III for data access according to an embodiment of the present application;

[0108] FIG. 9 is an architecture diagram of a server with an intelligent network card according to an embodiment of the present application;

[0109] FIG. 10 is a structural diagram of a network device according to an embodiment of the present application;

[0110] FIG. 11 is a structural diagram of a computing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0111] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In order to clearly describe the technical solutions in the embodiments of the present application, the description mode such as "at least one of a1, a2, …, and an (or at least one)" adopted in the embodiments of the present application includes a case where any one of a1, a2, …, and an exists alone, and a case where any combination of a plurality of a1, a2, …, and an exists, and each case can exist alone. For example, the description mode of "at least one of a, b, and c" includes a case where a alone, a case where b alone, a case where c alone, a case where a and b are combined, a case where a and c are combined, a case where b and c are combined, or a case where a, b, and c are combined.

[0112] Meanwhile, in the embodiments of the present application, the words such as "exemplary" or "for example" are used to represent an example, illustration, or description. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design solutions. Rather, the words such as "exemplary" or "for example" are intended to present the relevant concept in a specific manner, for the purpose of understanding.

[0113] In various embodiments of the present application, the terms and / or descriptions of various embodiments are consistent and can be mutually referred to if there is no special description and no logical conflict. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0114] In order to facilitate understanding of the technical solutions provided in the embodiments of the present application, first, the related terms of the distributed storage system involved in the embodiments of the present application are introduced:

[0115] 1. Remote direct memory access (RDMA)

[0116] RDMA is a technology that accesses data in the memory of a remote host by bypassing the kernel of the operating system of the remote host. Since it does not pass through the operating system, it not only saves a large amount of resources of a central processing unit (CPU), but also improves system throughput and reduces system network communication delay, and is particularly suitable for wide application in a large-scale parallel computer cluster.

[0117] RDMA has several characteristics. (1) Data is transmitted between the network and a remote machine; (2) the operating system kernel is not involved, and all contents related to transmission are offloaded to other acceleration devices (network devices with acceleration functions, such as intelligent network cards or switching devices); (3) data transmission is directly performed between the user space virtual memory and the acceleration device without involving the operating system kernel, and there is no additional data movement and copying.

[0118] 2. Hybrid storage mode

[0119] The hybrid storage mode refers to that in the storage space of the server, both data stored by using the replication mode and data stored by using the erasure code mode are included, so that fault tolerance capability can be provided while the memory occupation is reduced. The replication mode refers to that a data is copied into multiple identical copies, which are respectively stored in different servers. The erasure code mode refers to that the original data is encoded by using an erasure code algorithm to obtain redundancy, and the data and the redundancy are stored together to achieve the fault tolerance purpose. The calculation of the erasure code can occur within the data, for example, a to-be-stored data is divided into multiple shards, and the erasure code of the multiple shards is determined according to a check algorithm. Alternatively, the calculation of the erasure code can also occur between data, for example, the erasure code of multiple to-be-stored data can be directly determined according to the check algorithm, which will not be limited here.

[0120] The data processing method provided by the embodiment of the present application can be applied to a distributed storage system, which can be a file storage system, a key-value storage system, or other types of distributed storage systems, which will not be limited here. Before introducing the specific embodiments of the present application, the distributed storage system related to the embodiments of the present application will be introduced. The distributed storage system can store data on multiple servers to share the storage load, which not only improves the reliability, availability and access efficiency of the system, but also is easy to expand.

[0121] Please refer to FIG. 1a, which is a system architecture diagram of a distributed storage system according to an embodiment of the present application. The distributed storage system 100 can include a server cluster, a switching device B and a client cluster.

[0122] The server cluster can include N servers, where N is a positive integer greater than 1. As shown in FIG. 1a, the server cluster can specifically include server F1, server F2, and server F N Each server can communicate with each other. Each server is a device with both computing and storage capabilities, for example, the server can also be a machine with a data processing unit (DPU), which will not be limited here.

[0123] It should be understood that each server in the embodiments of the present application can store data in a replication manner, or in an erasure code manner, or in a hybrid storage manner, which will not be limited here. For example, if the server uses a hybrid storage manner, it means that the server can store data of the first type (for example, hot data) in a replication manner, and store data of the second type (for example, cold data) in an erasure code manner. Hot data refers to data with high access frequency and key role to business and application, and cold data refers to data with low access frequency and less important to business and application.

[0124] For ease of description, the embodiments of the present application can take 3 as an example, which can specifically include server F1, server F2, and server F3. The memory organization of data in the three servers can refer to FIG. 1b, which is an architecture diagram of the memory corresponding to a server host according to an embodiment of the present application. The distributed storage system corresponding to FIG. 1b can store 6 data, which can specifically include data H1, data H2, data H3, data H4, data S5, and data S6.

[0125] Among them, data H1, data H2, data H3, and data H4 are referred to as hot data, which is stored in a replication manner, that is, each server of the three servers stores a backup of the hot data, thereby allowing the client to read the data from any one server without destroying consistency, thereby effectively guaranteeing load balancing.

[0126] Data S5 and data S6 are referred to as cold data, which can be stored in an erasure code manner, thereby saving storage space while providing fault tolerance capability. For example, for cold data S5, the master server (for example, server F1) in the embodiments of the present application can split data S5 into two equal-size shards (specifically, data C1 and data C2), and generate a check shard through a check algorithm, that is, the erasure code P(1, 2) corresponding to data S5, and then when the three data shards are generated, the three shards can be randomly stored in three different servers.

[0127] Similarly, for the cold data S6, the master server (for example, server F1) in the embodiment of the application can split the data S6 into two equal-size shards (specifically, data C3 and data C4), and generate a check shard through a check algorithm, that is, the erasure code P(3, 4) corresponding to the data S6, and then, when the three data shards are generated, the three shards can be randomly stored on three different servers.

[0128] The client cluster of the distributed storage system 100 can include K clients, where K is a positive integer, that is, the client cluster can include one client or multiple clients, and the number of clients will not be limited here. As shown in FIG. 1a, the client cluster can specifically include client A1, …, and client AK. K For any client, a request (an access request or a write request) can be sent to the switching device B by using a network protocol adopted by the distributed storage system 100.

[0129] It can be understood that the client here can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle terminal, a smart television, or the like, which is a smart terminal with a data processing function. Alternatively, if the network protocol adopted by the distributed storage system 100 is an RDMA protocol, the client here can also be a network device with an RDMA network card. The RDMA network card is a network communication protocol card supporting remote direct memory access, and the RDMA network card can bypass the operating system kernel and CPU of the network device to directly read or write data from the memory space of an application program.

[0130] The switching device B (for example, a programmable switch) is an important component of building a high-performance, safe and reliable network infrastructure, and plays an important function in the distributed storage system 100, for example, implements request forwarding compatible with a network protocol. Of course, in order to improve the running performance of the distributed storage system 100, the switching device B also has an in-network conflict detection function or an offload acceleration function, where the in-network conflict detection refers to obtaining each ongoing target request (that is, a put request for writing data) on the switching device, so that when an access request (a get request) arrives, it is determined whether there is an ongoing target request for the same key information in the distributed storage system, if there is, it means that a conflict request is detected, and if there is not, it means that no conflict request is detected.

[0131] To facilitate subsequent data access process, the switching device B can be used to store the data type corresponding to each data. For example, refer to Table 1, which is a data record table associated with a storage mode provided by an embodiment of the present application. The data record table can include a plurality of mapping relationships corresponding to the data. For example, the six data shown in FIG. 1b can include a mapping relationship corresponding to data H1, a mapping relationship corresponding to data H2, a mapping relationship corresponding to data H3, a mapping relationship corresponding to data H4, a mapping relationship corresponding to data S5, and a mapping relationship corresponding to data S6. As shown in Table 1:

[0132] Table 1

[0133] For example, the mapping relationship corresponding to data H1 can be used to indicate that the data type of data H1 is hot data, and the storage mode of data H1 in the distributed storage system 100 is a replication mode. Similarly, the mapping relationship corresponding to data S5 can be used to indicate that the data type of data S5 is cold data, and the storage mode of data S5 in the distributed storage system 100 is an erasure code mode. The same applies to other data.

[0134] In a possible implementation, if the network protocol used by the distributed storage system 100 is the RDMA protocol, any client can establish an RDMA connection with each server. To support reliable transmission, each connection can maintain a packet sequence number (psn). Under normal circumstances, the psn in each new packet can be considered to be incremented by 1 compared to the psn in the last sent packet. If the packet sequence number of the information sent by the sending end matches the packet sequence number of the information planned to be received by the receiving end next time, the packet can be normally received. If the packet sequence number of the information sent by the sending end does not match the packet sequence number of the information planned to be received by the receiving end next time, the packet cannot be received by the receiving end.

[0135] To implement RDMA-compatible request forwarding and work normally in the case of retransmission and out-of-order, the switching device B in the embodiment of the present application can also save each RDMA connection and the psn state corresponding to each RDMA connection. In other words, the switching device can record the state information of each server. For example, each server can periodically report its state information to the switching device, or the switching device can actively obtain the state information of the server, which will not be limited by the embodiment of the present application.

[0136] For convenience of distinction, the data packet sequence number obtained by the client after increment processing according to the last sent data packet sequence number can be referred to as a first data packet sequence number, and the data packet sequence number in the state information of the server can be referred to as a second data packet sequence number, which is used to indicate the data packet sequence number of information planned to be received by the server next time.

[0137] For example, referring to Table 2, which is a state diagram table associated with an RDMA connection provided by the embodiment of the present application. If the client in the embodiment of the present application can be taken as an example of two clients (for example, client A1 and client A2) and three servers (for example, server F1, server F2 and server F3), the connection relationship stored by the switching device B can include six kinds, as shown in Table 2 below:

[0138] Table 2

[0139] Suppose that the initial data packet sequence number of the server and the client is 2000, according to Table 2 above, the first data packet sequence number corresponding to the client A2 is still the initial data packet sequence number, which means that the client A2 has not sent a request to the server in the distributed storage system 100 temporarily, and the first data packet sequence number corresponding to the client A1 is 2002, which means that the client A1 has sent two requests, one of which is sent to the server F1 and the other is sent to the server F3.

[0140] Optionally, the psn state stored by the switching device B can also refer to Table 3, which is another state diagram table associated with an RDMA connection provided by the embodiment of the present application. The state diagram table can include four flags, specifically including identification 1, identification 2, identification 3 and identification 4. As shown in Table 3:

[0141] Table 3

[0142] In other words, according to the state diagram shown in Table 3, it can be obviously known that the data packet corresponding to a request for accessing data can be changed before and after passing through the switching device. For example, if a message originally wants to access data from the server F1, the switching device B can need to forward the message to other servers (for example, the server F3) in order to achieve load balancing. In order to be compatible with forwarding, the switching device B can change the psn in the message to improve the success rate of message receiving. In a possible implementation, the switching device B can also store index information corresponding to each data in order to quickly obtain data from the storage space of a certain server. For example, the switching device B can store a hash index table, which is a data structure for directly accessing according to keyword information, that is, a hash algorithm is used to convert keyword information into a new hash value, and the hash value can be used to quickly locate the data storage location.

[0143] It should be noted that the items shown in Table 1, Table 2, Table 3 and Table 4 are only a form for reference, and other items can also be established according to requirements in an actual business scenario, and the specific form of the foregoing tables is not limited in the embodiments of the present application.

[0144] Optionally, the servers in the distributed storage system can also be machines with intelligent network cards, that is, the distributed storage system herein can include the switching device, the plurality of clients, the plurality of servers, and the intelligent network card corresponding to each server. In the embodiments of the present application, the client in the distributed storage system can be taken as an example to describe the architecture of the distributed storage system with the intelligent network card, wherein the intelligent network card is used to implement the offload acceleration function, and the switching device is mainly used to implement the request forwarding compatible with the network protocol, and of course, the switching device can also implement the in-network conflict detection function. For details, please refer to FIG. 2, which is a system architecture diagram of another distributed storage system according to an embodiment of the present application. As shown in FIG. 2, the distributed storage system 200 can include a client A, a switching device B, N servers, and an intelligent network card corresponding to each server, wherein N is a positive integer greater than 1.

[0145] The client A can be any one of the client clusters shown in FIG. 1a, and the client A can send a request (an access request or a write request) to the switching device B by using the network protocol adopted by the distributed storage system 200. For example, the client A can send information to the switching device B by using the RDMA protocol, and the information can be a data packet for accessing data or a data packet for writing data. The data packet can include a frame header, an IP header, a UDP header, an original message, a frame check sequence (a check parameter), and a data packet sequence number.

[0146] The switching device B here can include two modules, specifically can include a detection module 21 and a load balancing module 22. The detection module 21 is configured to perform in-network conflict detection, and the load balancing module 22 is configured to change the sequence number of the data packet to achieve load balancing forwarding compatible with the network protocol. It can be understood that the switching device B can support users to customize the data plane and the control plane through a specific programming language, to provide relatively flexible programmable capability while maintaining line speed.

[0147] The server in the embodiment of the present application has the same function as the server shown in FIG. 1a, that is, each server can adopt a hybrid storage management strategy to reduce the occupation of server memory while providing fault tolerance capability. In other words, each server can store data belonging to the first type (for example, hot data) in a replication manner and store data belonging to the second type (for example, cold data) in an erasure code manner.

[0148] For ease of description, the i-th server in the N servers can be denoted as server F i The server F i The corresponding intelligent network card is denoted as intelligent network card C i , and i is a positive integer less than or equal to N. It can be understood that the switching device B mainly forwards the information sent by the client into information matching the state information of the corresponding server. The state information of the server can be reported by the server regularly or pulled by the switching device B actively, which will not be limited here.

[0149] For example, the intelligent network card C i When receiving the state information of the server F i , the intelligent network card C i can send the state information to the switching device B.

[0150] As shown in FIG. 2, a block of memory can be pre-allocated on the host of the server F i , and the block of memory can include memory R1 and memory R2, wherein the memory R1 can be used to store hot data, and the memory R2 can be used to store cold data.

[0151] It can be understood that the server F i may also include a notification module 23, a management module 24 and a statistical module 25, which are specifically introduced as follows:

[0152] The notification module 23 can be used to inform the switching device B of the data type of each data or the storage manner of each data, so that the switching device B maintains and updates the data record table (such as the table 1 shown above) associated with the storage manner.

[0153] The management module 24 can be configured to inform the switching device B of the current network connection (e.g., RDMA connection) so that the switching device B can update the state information associated with the network protocol (e.g., the state information in the state diagram shown in Table 2 or Table 3).

[0154] The statistics module 25 can be configured to count the currently stored data and evaluate the data type of each data, i.e., evaluate the hotness parameter according to the access frequency of each data. When the access frequency is greater than a first frequency threshold (e.g., 50 times / day), the data can be determined as hot data. When the access frequency is less than a second frequency threshold (e.g., 1 time / week), the data can be determined as cold data.

[0155] It should be understood that the intelligent network card C i adds a programmable module to the original network card function, allowing users to offload specified functions (e.g., read operation corresponding tasks) and reduce the processing burden of the CPU. In the distributed storage system 200, the intelligent network card C i is configured to process the request sent by the switching device B, which can specifically include reading and writing host memory, synchronizing with other servers, returning a response, etc.

[0156] The intelligent network card C i may also store the hash index table shown in FIG. 2, which can include index information corresponding to each data and a storage address corresponding to each data. The index information here can be a hash value obtained by calculating the keyword information corresponding to each data using a hash algorithm.

[0157] When the distributed storage system 200 uses the RDMA protocol for data management, the client A can establish an RDMA connection with each server. For example, the client A and the server F i establish an RDMA connection, which means that the client A can establish an RDMA connection with the host of the server, or with the intelligent network card C i of the server F i .

[0158] It can be understood that the embodiments of the present application support in-network conflict detection and network protocol compatible forwarding by designing a custom data plane on the switching device shown in FIG. 1a or FIG. 2. For details, please refer to FIG. 3, which is a scene diagram of a data plane of a switching device according to an embodiment of the present application. As shown in FIG. 3, the information received by the switching device from the client can be roughly divided into three categories, which can specifically include a write request for writing data, a read request for accessing data, and a read response for feeding back the access.

[0159] When the switching device receives information X from any client, it can classify information X. Here, information X is used to perform corresponding operations on the first server (e.g., the master server among N servers).

[0160] If the request corresponding to information X is a write request for writing data, the switching device can directly send information X to the first server (e.g., tracker) to write the data in information X into the storage space of the first server.

[0161] If the request corresponding to information X is a read response for feedback access, then the switching device can directly send information X to the first server. At the same time, the switching device can also update the data packet sequence number corresponding to the first server.

[0162] If the request corresponding to information X is a read request for accessing data, the switching device first determines whether intranet conflict detection is required.

[0163] For example, if the data accessed by information X is stored in erasure coding mode, the switching device does not need to perform intra-network conflict detection and can directly send information X to the first server. At the same time, the switching device can also update the data packet sequence number corresponding to the first server.

[0164] For example, if the data accessed by information X is stored in a replicated manner, the switching device needs to perform intra-network conflict detection. For instance, based on the keyword information in information X and the ongoing write requests in the distributed storage system, it determines whether a conflicting request corresponding to information X exists in the distributed storage system. If it does, information X needs to be sent to the first server. If it does not exist, the switching device can use its load balancing module to select one backup server from (N-1) backup servers as the second server and send information X to the second server. Simultaneously, the switching device needs to update the data packet sequence number corresponding to the second server.

[0165] When the client's access object changes from the first server to the second server, the entire process does not require the client to re-establish a connection with the second server. That is, the client does not need to perform multiple transmissions with the second server, which can effectively reduce the transmission bandwidth between distributed storage systems, reduce transmission overhead, and improve the performance of distributed storage systems.

[0166] In the distributed storage system, the specific implementation of compatible forwarding using a new communication protocol (e.g., RDMA protocol) and based on the editability of the switching device can be seen in the embodiment methods corresponding to Figures 4-9 below.

[0167] Further, please refer to FIG. 4, which is a schematic diagram of a method for data processing provided by the embodiments of the present application. As shown in FIG. 4, the method can be performed by the switching device in the distributed storage system shown in FIG. 1a, which will not be limited herein. The method can include at least steps S401-S404:

[0168] In step S401, first information sent by a client is received.

[0169] The first information is used to request access to first data from a first server of N servers. The first information includes first keyword information (for example, key), a first check parameter, and a packet sequence number. The packet sequence number (for example, 2002) in the first information can be obtained by incrementing the packet sequence number sent by the client last time.

[0170] It should be understood that the first check parameter is used to verify the integrity of the message during network transmission. For example, the first check parameter can be a checksum obtained by calculating the header or message data of the message according to a check algorithm during generation of the first information.

[0171] For convenience of distinction, the packet sequence number of the information sent by the client this time can be referred to as a first packet sequence number, and the packet sequence number of the information that the server plans to receive next time can be referred to as a second packet sequence number. It can be understood that when the distributed storage system is a master-slave architecture, the N servers can include one master server and (N-1) backup servers. Generally, the client will select the master server as the preferred target, that is, the first server can be the master server (for example, server F1 shown in FIG. 1a) of the N servers.

[0172] In step S402, target information is sent to a second server.

[0173] The second server can establish an RDMA connection with the client. The second server can be any one of the (N-1) servers, or the second server can be the server with the strongest computing resources among the (N-1) servers to better achieve load balancing. The target information is determined based on the first information, and the target information can be used to request access to the first data from the second server. The second server is selected from the (N-1) servers except the first server. The target information includes a packet sequence number, and the packet sequence number in the target information matches the state information of the second server, which indicates the information that the second server plans to receive next time.

[0174] Exemplarily, when the data packet sequence number of the first information matches the state information of the second server, the target information here can be the first information. When the data packet sequence number of the first information does not match the state information of the second server, the target information here can be the data packet sequence number indicated based on the state information of the second server, after the first data packet sequence of the first information is changed.

[0175] Specifically, when the exchange device receives the first information sent by the client, the exchange device can select one server as the second server from the (N-1) servers other than the first server, and then generate the third information based on the state information of the second server and the first data packet sequence number in the first information. Then, the exchange device can determine the target information based on the third information. Here, the data packet sequence number in the third information matches the state information of the second server, i.e., the data packet sequence number in the third information is the same as the second data packet sequence number corresponding to the second server. In addition, the target address in the third information is the same as the target address in the first information, which is the first keyword information.

[0176] Exemplarily, after the second server is determined, the exchange device can determine the second data packet sequence number corresponding to the second server based on the state information of the second server (i.e., the sequence number of the data packet that the second server plans to receive next), and then compare the first data packet sequence number in the first information with the second data packet sequence number corresponding to the second server. If the first data packet sequence number in the first information is the same as the second data packet sequence number corresponding to the second server, the exchange device does not need to change the first data packet sequence number in the first information, and can directly take the first information as the third information. If the first data packet sequence number in the first information is different from the second data packet sequence number corresponding to the second server, the exchange device can change the first data packet sequence number in the first information to the second data packet sequence number, and then take the changed first information as the third information. It can be understood that the target address in the third information is still the first keyword information, and the check parameter in the third information is still the check parameter in the first information.

[0177] In a possible implementation, the exchange device can directly determine the third information as the target information, and send the third information to the second server. At this time, the target address of the target information here is still the first keyword information.

[0178] In another possible implementation, in order to reduce the burden of the server and save computing resources, the exchange device can also update the third information, and determine the updated third information as the target information. For example, the target address of the target information here can be the index information corresponding to the first keyword, and the check parameter in the target information is different from the check parameter in the first information.

[0179] For example, the switching device can change a specific field of the third information according to the logic of the access operation (for example, change the target address in the third information from the first keyword information to the index information corresponding to the first keyword information), since the content of the third information has changed, to ensure the integrity of the third information during network transmission, the switching device also needs to recalculate the check parameter based on the check algorithm, and then generate the target information based on the recalculated check parameter and the third information after changing the specific field, and send it to the second server.

[0180] To facilitate understanding of the compatible forwarding function of the switching device, further, please refer to FIG. 5, which is a scene diagram for compatible forwarding provided by an embodiment of the present application. Wherein, the distributed storage system corresponding to FIG. 5 can include a client A, a switching device B and three servers, specifically including a server F1, a server F2 and a server F3. The client A can establish an RDMA connection with the three servers respectively. Wherein, when the distributed storage system belongs to a master-slave architecture, the server F1 here is the master server, and the server F2 and the server F3 can be backup servers.

[0181] Wherein, npsn can be used to represent the next packet sequence number generated by the client, and epsn can be used to represent the expected packet sequence number (epsm) of the server.

[0182] As shown in FIG. 5, the initial packet sequence numbers of the client A and the three servers can all be taken as 2000, the client A can sequentially send two requests, wherein the packet sequence number in the first request message can be 2000, and the packet sequence number in the second request message can be 2001, which is obtained by incrementing the packet sequence number in the first request message by the client. Wherein, the first request message and the second request message can both be called the first information, which is used to request data from the server F1.

[0183] When the client A sends the first request message to the switching device B, the switching device B determines that the server accessed by the first request message is the server F1, and since the state information of the server F1 indicates that the packet sequence number of the information to be received by the server F1 next time is 2000, the switching device B does not need to change the first request message and can directly forward the first request message to the server F1. In other words, the psn of the first request message does not change before and after the first request message passes through the switching device B. Alternatively, after the switching device B sends the first request message to the server F1, the embodiment of the present application can update the second packet sequence number corresponding to the server F1 to 2001.

[0184] When the client A sends the second request message to the switching device B, the switching device B can perceive the load balancing of the three servers to determine to compatibly forward the second request message to the backup server 2 (i.e., the server F3). Since the packet sequence number in the second request message is 2001, and the state information of the server F3 indicates that the packet sequence number of the information to be received by the server F3 next time is 2000, the switching device needs to change the packet sequence number in the second request message from 2001 to 2000, and then can forward the changed second request message to the server F3. In other words, the psn of the second request message changes before and after the second request message passes through the switching device B.

[0185] Step S403, receiving the first data from the second server.

[0186] Step S404, sending the first data to the client.

[0187] It is worth noting that the response information corresponding to the forwarded request reaches the switching device, and the response information is restored to the original packet sequence number (i.e., the packet sequence number corresponding to the first information), so that the client can normally receive the response information. In other words, if the access object of the client changes from the first server to the second server, the switching device needs to restore the packet sequence number in the response information to the packet sequence number matched with the state information of the client when the switching device receives the response information of the second server, so as to improve the success rate of the response information.

[0188] In the embodiment of the present application, the information received by the switching device is the first information sent by the client, and the information sent out is the target information, the sequence number of which matches the state information of the second server, which means that the switching device in the embodiment of the present application not only provides the function of information forwarding, but also provides the function of compatible network protocol forwarding, that is, the switching device can change the received information so that the sent-out information can be successfully received by the second server. In the whole data access process, even if the access object of the client is changed from the first server to the second server, the client and the server do not need to establish a new connection through multiple transmissions, but the switching device changes directly, thereby effectively saving resources and reducing transmission bandwidth, and effectively improving the performance of the distributed storage system.

[0189] In a possible implementation, after receiving the first information, the switching device can further perform in-network conflict detection, so as to improve the performance of the distributed storage system while ensuring the accuracy of data access. In order to facilitate the understanding of the above process, further, please refer to FIG. 6, which is an interaction diagram one for data access provided by the embodiment of the present application. As shown in FIG. 6, the method can be performed by the client, the switching device, the first server and the second server. The client can be the client A shown in FIG. 5, the switching device can be the switching device B shown in FIG. 5, the first server can be the master server in the distributed storage system, for example, the server F1 shown in FIG. 5, and the second server can be the backup server selected by the switching device B from the N servers in the distributed storage system, for example, the server F3 shown in FIG. 5. The method can at least include steps S601-S610:

[0190] Step S601, the client sends the first information to the switching device.

[0191] The first information is used to request the first data from the first server.

[0192] Step S602, the switching device performs in-network conflict detection to obtain a detection result.

[0193] Wherein, M is a positive integer, specifically, the switching device can perform in-network conflict detection based on the first keyword information in the first information and the second keyword information of the M target requests in progress to obtain the detection result.

[0194] Exemplarily, the switching device can acquire M target requests being processed in the current distributed storage system, and then find second keyword information matching the first keyword information from the second keyword information corresponding to the M target requests respectively. If no second keyword information is found, it means that there is no conflict request corresponding to the first information in the distributed storage system. If second keyword information is found, the target request corresponding to the found second keyword information is determined as the conflict request corresponding to the first information.

[0195] In a possible implementation, the switching device can record the state identifier of the target request (i.e., the request corresponding to the write operation) when receiving any target request sent by any client, so as to improve the detection efficiency when performing the in-network conflict detection subsequently. The state identifier herein is a first state identifier or a second state identifier. The first state identifier is used to indicate that the target request is being processed, and the second state identifier is used to indicate that the target request has been completed.

[0196] For ease of understanding, exemplarily, Table 4 is provided, which is a state diagram table associated with a target request according to an embodiment of the present application. Table 4 can be seen as follows:

[0197] Table 4

[0198] For example, the switching device can acquire the target requests with the first state identifier, i.e., request 1 and request 2, from the state diagram table shown in Table 4 when performing the in-network conflict detection. The second keyword information of request 1 is denoted by key1, and the second keyword information of request 2 is denoted by key2. Then, the switching device can find second keyword information matching the first keyword information in the first information from key1 and key2. If the first keyword information matches one of the two second keyword information, a first detection result is generated (the result is used to indicate that the conflict request corresponding to the first information is detected); if the first keyword information does not match any of the two second keyword information, a second detection result is generated (the result is used to indicate that the conflict request corresponding to the first information is not detected).

[0199] If the detection result indicates that the conflict request corresponding to the first information is detected, it means that the master server in the distributed storage system is updating the first data. In order to avoid that the switching device acquires the data before the update from the backup servers of the N servers, the switching device cannot send the first information to the backup servers, but needs to continue to perform step S603 to acquire the new first data from the storage space of the first server.

[0200] In step S603, the switching device sends the first information to the first server.

[0201] Step S604, the switching device receives the update data from the first server.

[0202] The update data here is the data obtained after updating the first data based on the conflict request corresponding to the first information.

[0203] Step S605, the switching device sends the update data to the client.

[0204] Optionally, if the detection result indicates that the conflict request corresponding to the first information is not detected, in order to better achieve load balancing and reduce the burden of the first server, the switching device can jump to step S606 to obtain data from the storage space of the second server after step S602 is executed.

[0205] Step S606, the switching device determines the second server from the (N-1) servers other than the first server.

[0206] Step S607, the switching device generates target information based on the state information of the second server and the first information.

[0207] Step S608, the switching device sends the target information to the second server.

[0208] Step S609, the switching device receives the first data sent by the second server.

[0209] Step S610, the switching device sends the first data to the client.

[0210] The specific implementation of steps S606-S610 can refer to the description of steps S402-S404 in the embodiment corresponding to FIG. 4, which will not be repeated here.

[0211] In the embodiment of the present application, the switching device needs to perform in-network conflict detection when receiving the first information for accessing the first data, thereby reducing the possibility of obtaining incorrect data. In other words, by using the data processing method provided in the embodiment of the present application, not only the running performance of the distributed storage system can be effectively improved, but also the accuracy of data access can be improved, and better data management can be achieved.

[0212] In another possible implementation, if the distributed storage system adopts the hybrid storage mode, the switching device, after receiving the first information, can not only perform the intra-network conflict detection, but also implement compatible forwarding according to the data type of the data accessed by the first information. For the convenience of understanding this process, further, please refer to FIG. 7, which is an interaction diagram two for data access provided by an embodiment of the present application. As shown in FIG. 7, the method can be performed by the client, the switching device, the first server and the second server. The client can be the client A shown in FIG. 5, the switching device can be the switching device B shown in FIG. 5, the first server can be the master server in the distributed storage system, for example, the server F1 shown in FIG. 5, and the second server can be the backup server selected by the switching device B from the N servers in the distributed storage system, for example, the server F3 shown in FIG. 5. The method can at least include steps S701-S706:

[0213] In step S701, the client sends the first information to the switching device.

[0214] The first information is used to request the first data from the first server.

[0215] In step S702, the switching device acquires the data type of the first data.

[0216] If the data type of the first data is the second type (used to indicate that the data is stored in the erasure code mode), the switching device can continue to perform step S703:

[0217] In step S703, the switching device sends the first information to the first server.

[0218] In step S704, the switching device receives the first data from the first server.

[0219] In step S705, the switching device sends the first data to the client.

[0220] Alternatively, if the data type of the first data is the first type (used to indicate that the data is stored in the replication mode), the switching device, after performing step S702, can jump to perform step S706:

[0221] In step S706, the switching device performs steps S402-S404 shown in FIG. 4 or steps S602-S610 shown in FIG. 6, which will not be repeated here.

[0222] The data processing method provided in the embodiments of the present application can be applied to a distributed storage system using a hybrid storage mode for storage. The distributed storage system can reduce memory occupation and save storage resources. When the exchange device receives first information for accessing first data, the exchange device needs to determine a server to be finally accessed by the client according to a data type of the first data, and through compatible forwarding, the performance of the distributed storage system is better improved.

[0223] In a possible implementation, the distributed storage system can further include N intelligent network cards corresponding to the N servers respectively, which can be used to further offload and accelerate read operations of the distributed storage system to reduce the burden of the CPU, so as to save computing resources and storage resources while realizing load balancing, fault tolerance and strong consistency. In other words, this implementation can realize resource saving and good scalability at the same time.

[0224] The intelligent network card can include a plurality of components, and specifically can include a first component and a second component. The first component can change a specific field (for example, a target address, etc.) of the received information according to the logic of the access operation, and the second component can recalculate the check parameter for the information changed by the first component.

[0225] For example, the first component in the intelligent network card can be an on-path multi-core component (for example, a DPA), and the second component can be a system-level chip (for example, a soc with an ARM core) of an off-path component. That is, using such an intelligent network card can further offload the computing task of the second server from the ARM core to the multi-core component, thereby effectively improving the system performance.

[0226] For example, the first component in the intelligent network card can be a network processor (NP), and the second component can be an application-specific integrated circuit (for example, an ASIC).

[0227] Optionally, the intelligent network card can include a third component, which is used not only to change a specific field (for example, a target address, etc.) of the received information according to the logic of the access operation, but also to recalculate the check parameter of the changed information.

[0228] Further, for the convenience of understanding the compatible forwarding process of the distributed storage system shown in FIG. 2, please refer to FIG. 8, which is an interactive diagram three for data access provided by an embodiment of the present application. As shown in FIG. 8, the method can be jointly executed by a client, a switching device, a second server, and an intelligent network card corresponding to the second server (i.e., a target intelligent network card). The method can at least include steps S801-S806:

[0229] In step S801, the client sends first information to the switching device.

[0230] The first information is used to request first data from a first server of N servers. The first information includes first keyword information.

[0231] In step S802, the switching device sends target information to the target intelligent network card.

[0232] The data packet sequence number in the target information is used to request access to the first data from the second server. The data packet sequence number in the target information matches the state information of the second server. The state information of the second server is used to indicate the information that the second server plans to receive next time. The target address of the target information is the keyword information (i.e., the first keyword information in the first information).

[0233] For example, the switching device can select one server as the second server from the (N-1) servers other than the first server, and determine the second data packet sequence number corresponding to the second server (i.e., the sequence number of the data packet that the second server plans to receive next time) based on the state information of the second server. Further, the switching device can compare the first data packet sequence number in the first information with the second data packet sequence number corresponding to the second server. If the first data packet sequence number in the first information is the same as the second data packet sequence number corresponding to the second server, the switching device does not need to change the first data packet sequence number in the first information, and can directly use the first information as the target information. If the first data packet sequence number in the first information is different from the second data packet sequence number corresponding to the second server, the switching device can change the first data packet sequence number in the first information to the second data packet sequence number, and then use the changed first information as the target information.

[0234] Optionally, the switching device can also perform in-network conflict detection before sending the target information to the target intelligent network card. The specific process can be referred to the description of step S602 shown in FIG. 6. When the detection result indicates that no conflict request corresponding to the first information is detected, step S802 can be continued.

[0235] Optionally, when the distributed storage system adopts the hybrid storage mode for storage, the exchange device also needs to obtain the data type of the first data before sending the target information to the target intelligent network card. When the data type of the first data is the first type, step S802 is executed, which will not be described here. In the embodiments of the present application, compatible forwarding can be performed through the exchange device to ensure load balancing, read operation can be offloaded through the intelligent network card to reduce CPU usage, and the hybrid storage mode can be used to reduce the use of memory resources, thereby more effectively improving the running performance of the distributed storage system.

[0236] In step S803, the target intelligent network card sends access information to the second server.

[0237] In the access information, the target address is the index information corresponding to the first keyword information, and the check parameter in the access information can be different from the check parameter in the first information.

[0238] It can be understood that the target intelligent network card can be a flexible programmable network card, which is equivalent to adding an editable module to the function of a traditional network card. The traditional network card has a check function, so the target intelligent network card can be regarded as a combination of a check module and an editable module.

[0239] For example, when the target information is received, the editable module of the target intelligent network card can first change a specific field of the target information according to the logic of the access operation (for example, changing the target address in the target information from the first keyword information to the index information corresponding to the first keyword information). Since the content of the target information has changed, the editable module can also recalculate the check parameter (i.e., the first check parameter) based on the check algorithm, and then generate the access information based on the recalculated check parameter and the target information after changing the specific field. Then, the check module can check the access information. For example, the check module can calculate the access information according to the same check algorithm as the editable module to obtain a second check parameter. If the first check parameter is the same as the second check parameter, it means that the check is successful, and the target intelligent network card can send the access information to the second server.

[0240] In step S804, the target intelligent network card obtains the first data from the memory of the second server.

[0241] In step S805, the target intelligent network card sends the first data to the exchange device.

[0242] For the convenience of understanding the process of obtaining the first data by the target intelligent network card, please refer to FIG. 9, which is an architecture diagram of a server with an intelligent network card according to an embodiment of the present application. As shown in FIG. 9, the intelligent network card 900 can include a network card receiving port 91, a first component 92, a switching device 93 (for example, an on-board switching device E-switch), a second component 94, and a network card sending port 96. It is worth noting that the switching device 93 here is a component integrated in the intelligent network card, and is not the switching device used to implement the compatible forwarding function as described above.

[0243] It should be understood that when the network card receiving port 91 shown in FIG. 9 receives the target information, the target information can be forwarded to the first component 92 according to a specific routing rule, and the first component can change the specific domain of the target information according to the logic of the access operation (for example, changing the target address in the target information from the first keyword information to the index information corresponding to the first keyword information).

[0244] Further, the first component 92 can send the target information with the changed specific domain to the second component 94 through the switching device 93, so that the second component 94 recalculates its check parameter and generates access information based on the recalculated check parameter and the target information with the changed specific domain. At this time, the second component 94 can send the access information to the host 95 of the server F shown in FIG. 9 to obtain the first data accessed by the access information. After obtaining the data in the memory of the host 95, the intelligent network card 900 can return the first data to the switching device through the network card sending port 96.

[0245] In step S806, the switching device sends the first data to the client.

[0246] In the embodiments of the present application, through the joint design of the target intelligent network card and the switching device, not only can the performance of the distributed storage system be improved through the compatible forwarding of the switching device, but also the read operation of the distributed storage system can be further offloaded and accelerated through the target intelligent network card to reduce the CPU burden of the second server, thereby saving computing resources and storage resources while achieving load balancing, fault tolerance and strong consistency.

[0247] In another possible implementation, the embodiments of the present application can not only use the target intelligent network card to implement the read request, but also use the target intelligent network card to implement the write request, so that the CPU of the server can be bypassed throughout the process, thereby reducing the burden of the server CPU and improving the running performance of the distributed storage system.

[0248] For example, the client can send second information to the switching device, where the second information can be used to request writing of second data in the first server. Upon receiving the second information, the switching device can store a target request corresponding to the second information, where the target request corresponding to the second information can be used for in-network conflict detection. Meanwhile, the second data is successfully written into the storage space of the first server by the intelligent network card corresponding to the first server. The first server can also synchronize the second data to other servers according to a storage mode corresponding to the data type of the second data, and when the first server successfully receives a notification sent by (N-1) servers indicating synchronization completion, the first server can return a feedback information to the switching device through the intelligent network card corresponding to the first server, where the feedback information can be used to indicate that the second data has been successfully written into N servers.

[0249] For ease of understanding, please continue to refer to the above Figure 9. If the server shown in Figure 9 is the first server, the intelligent network card 900 can store a target request (for example, a put request) corresponding to the second information in the on-chip memory of the second component 94 when receiving the second information sent by the client through the network card receiving port 91. The work thread running in the second component 94 can be responsible for polling the on-chip memory, and once the target request corresponding to the second information is read, the second data can be written into the storage space of the server F, and synchronization can also be performed with other servers.

[0250] After successfully writing the second data into N servers, the switching device can send response information (for example, an ACK response) corresponding to the second information to the client, where the response can be used to indicate that the target request corresponding to the second information has been completed.

[0251] Further, please refer to Figure 10, which is a structural schematic diagram of a network device provided by an embodiment of the present application. As shown in Figure 10, the network device 10 can include at least one of a transceiver unit 1001 and a processing unit 1002. These units can perform the corresponding functions of the devices in the above method embodiments.

[0252] In a possible implementation, the network device 10 can be used to implement the functions of the switching device (for example, the switching device B in Figure 1a or Figure 2) in the distributed storage system.

[0253] Specifically, the transceiver 1001 is configured to receive first information sent by the client, the first information being used to request to access the first data from a first server of the N servers; the transceiver 1001 is configured to send target information to a second server, the target information being determined based on the first information, the target information being used to request to access the first data from the second server, the second server being a server selected from the (N-1) servers other than the first server, and the target information including a data packet sequence number, the data packet sequence number matching state information of the second server, the state information of the second server being used to indicate information that the second server plans to receive next time; the transceiver 1001 is further configured to receive the first data from the second server; and the transceiver 1001 is further configured to send the first data to the client.

[0254] In a possible implementation, the data packet sequence number in the target information is obtained by modifying the data packet sequence number in the first information.

[0255] In a possible implementation, the switching device stores a connection relationship, the connection relationship being used to indicate that the client and the second server establish a remote direct memory access (RDMA) connection.

[0256] In a possible implementation, the switching device is configured to receive and record the state information of the second server, the state information of the second server including the data packet sequence number.

[0257] In a possible implementation, the switching device further includes a processing unit 1002, which is configured to obtain a data type of the first data; and the processing unit 1002 is further configured to perform the step of sending the target information if the data type of the first data is a first type, the first type being used to indicate that the data is stored in a replication manner.

[0258] In a possible implementation, the transceiver 1001 is further configured to send the first information to the first server if the data type of the first data is a second type, the second type being used to indicate that the data is stored in an erasure code manner; the transceiver 1001 is further configured to receive the first data from the first server; and the transceiver 1001 is further configured to send the first data to the client.

[0259] In a possible implementation, the first information includes first keyword information; the switching device further includes a processing unit 1002, which is further configured to perform in-network conflict detection based on the first keyword information and second keyword information of M target requests in progress, to obtain a detection result, M being a positive integer; and the processing unit 1002 is further configured to perform the step of sending the target information to the second server if the detection result indicates that no conflict request corresponding to the first information is detected.

[0260] In a possible implementation, the transceiver 1001 is further configured to send the first information to the first server if it is detected that the first information corresponds to a conflict request; the transceiver 1001 is further configured to receive update data from the first server, the update data being data obtained by performing update processing on the first data based on the conflict request corresponding to the first information; and the transceiver 1001 is further configured to send the update data to the client.

[0261] In a possible implementation, the transceiver 1001 is further configured to receive second information sent by the client, the second information being used to request writing of second data in the first server; and the processing unit 1002 is further configured to store a target request corresponding to the second information, the target request corresponding to the second information being used to perform in-network conflict detection.

[0262] In a possible implementation, the exchange device stores a state identifier corresponding to the target request, the state identifier being a first state identifier or a second state identifier, the first state identifier being used to indicate that the target request is being processed, and the second state identifier being used to indicate that the target request has been completed.

[0263] In a possible implementation, an RDMA connection is established between the smart NIC corresponding to the second server and the client, the target address in the first information is first keyword information; the transceiver 1001 is further configured to send target information to the smart NIC corresponding to the second server, the target address of the target information being the first keyword information, the smart NIC corresponding to the second server being configured to send access information to the second server, the target address in the access information being index information corresponding to the first keyword information; and the receiving unit is further configured to receive the first data sent by the smart NIC corresponding to the second server.

[0264] In a possible implementation, the target address of the target information is index information corresponding to the first keyword information, and the first keyword information is the target address of the first information.

[0265] In a possible implementation, the check parameter in the target information is different from the check parameter in the first information.

[0266] The specific implementation of the transceiver 1001 and the processing unit 1002 can be referred to the description of steps S401-S404 in the above-described embodiments corresponding to FIG. 4, and will not be described here. In addition, the beneficial effects of using the same method will not be described here.

[0267] In another possible implementation, the network device 10 can be used to implement the function of a target smart NIC (for example, the target smart NIC shown in FIG. 8) in a distributed storage system.

[0268] Specifically, the transceiver 1001 is configured to receive target information sent by the switching device, the target information being used to request to access the first data from the second server, the target information comprising a data packet sequence number, the data packet sequence number matching state information of the second server, the state information of the second server being used to indicate information that the second server plans to receive next time, and a target address of the target information being first keyword information; the transceiver 1001 is configured to send access information to the second server, a target address in the access information being index information corresponding to the first keyword information; the transceiver 1001 is further configured to receive the first data from the second server; and the transceiver 1001 is further configured to send the first data to the switching device.

[0269] In a possible implementation, the RDMA connection is established between the target intelligent network card and the client.

[0270] In a possible implementation, the transceiver 1001 is further configured to receive state information sent by the second server, and send the state information to the switching device.

[0271] In a possible implementation, the check parameter in the access information is different from the check parameter in the target information.

[0272] In a possible implementation, the target intelligent network card comprises a first component and a second component, the first component is configured to update the target address of the target information, and the second component is configured to update the check parameter of the target information.

[0273] In a possible implementation, the target intelligent network card comprises a third component, the third component is configured to update the target address of the target information, and the third component is further configured to update the check parameter of the target information.

[0274] The specific implementation of the transceiver 1001 and the processing unit 1002 can refer to the descriptions of steps S401-S404 in the above-described embodiments corresponding to FIG. 9, and will not be described here again. In addition, the beneficial effects of using the same method will not be described again.

[0275] In another possible implementation, the network device 10 can be configured to implement the function of the second server in the distributed storage system, the second server being determined by the switching device in the distributed storage system from N servers, for example, the second server in the above-described FIG. 6, FIG. 7 or FIG. 8.

[0276] Specifically, the transceiver 1001 is configured to receive, from the switch device, access information, the access information comprising a data packet sequence number, the data packet sequence number matching second server state information, the second server state information being used to indicate information that the second server plans to receive next time; and the transceiver 1001 is further configured to send, to the switch device, first data, the first data being obtained from a storage space of the second server based on the access information.

[0277] In a possible implementation, the RDMA connection is established between the second server and the client.

[0278] In a possible implementation, the transceiver 1001 is further configured to send, to the switch device, the state information.

[0279] In a possible implementation, the RDMA connection is established between the client and a smart NIC corresponding to the second server, the smart NIC corresponding to the second server being configured to generate the access information; and the transceiver 1001 is further configured to send, to the smart NIC corresponding to the second server, the state information, the smart NIC corresponding to the second server being configured to send, to the switch device, the state information.

[0280] In a possible implementation, the target address in the access information is index information, the index information being used to indicate a storage address of the first data in the second server.

[0281] In a possible implementation, the check parameter in the access information is different from a check parameter in first information, the first information being generated by the client and being used to access the first data from a first server of the N servers.

[0282] The specific implementation of the transceiver 1001 and the processing unit 1002 can be referred to the descriptions of steps S401-S404 in the above-described embodiments corresponding to FIG. 9, and will not be repeated here. In addition, the beneficial effects of using the same method will not be repeated here.

[0283] Further, please refer to FIG. 11, which is a structural schematic diagram of a computing device according to an embodiment of the present application. As shown in FIG. 11, specifically, the computing device 11 can be configured to implement the function of the switch device (for example, the switch device B shown in FIG. 5) in the distributed storage system, or the computing device 11 can be configured to implement the function of the target smart NIC (for example, the target smart NIC shown in FIG. 8) in the distributed storage system, or the computing device 11 can be configured to implement the function of the second server (for example, the server F3 shown in FIG. 5) in the distributed storage system.

[0284] Referring to FIG. 11, the computing device 11 includes all or part of hardware in a processor 1101, a communication interface 1102 and a memory 1103. The number of processors 1101 in the computing device 11 can be one or more, and FIG. 11 takes one processor as an example. In an embodiment of the present application, the processor 1101, the communication interface 1102 and the memory 1103 can be connected through a bus system or other means, and FIG. 11 takes the connection through the bus system 1104 as an example.

[0285] The processor 1101 can be a CPU, an NP or a combination of the CPU and the NP. The processor 1101 can also include a hardware chip. The hardware chip can be an ASIC, a programmable logic device (PLD) or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0286] The communication interface 1102 is used to receive and send data. Specifically, the communication interface 1102 can include a receiving interface and a sending interface. The receiving interface can be used to receive data, and the sending interface can be used to send data. The number of communication interfaces 1102 can be one or more.

[0287] The memory 1103 can include a volatile memory such as a random-access memory (RAM), and can also include a non-volatile memory such as a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), and can also include a combination of the above kinds of memories.

[0288] Optionally, the memory 1103 stores an operating system and programs, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the programs can include various operation instructions for implementing various operations. The operating system can include various system programs for implementing various basic services and processing hardware-based tasks. The processor 1101 can read the programs in the memory 1103 to implement the method provided in the embodiments of the present application.

[0289] The memory 1103 can be a memory device in the computing device 11 or a storage device independent of the computing device 11.

[0290] The bus system 1104 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus system 1104 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is shown in FIG. 11, but it does not mean that there is only one bus or only one type of bus.

[0291] In some possible embodiments, the network device described above can be implemented as a virtualized device. For example, the virtualized device can be a virtual machine (VM) running a program for sending a message function, and the virtual machine is deployed on a hardware device (for example, a physical server). The virtual machine refers to a complete computer system that is simulated by software, has complete hardware system functions, and runs in a completely isolated environment. The virtual machine can be configured as a network device. For example, the network device can be implemented based on a general-purpose physical server combined with network function virtualization (NFV) technology. Those skilled in the art can virtualize the network device with the above functions on the general-purpose physical server by reading this application in combination with the NFV technology, and details are not described herein.

[0292] It should be noted that the network device mentioned in the embodiments of the present application can be a switching device, an intelligent network card, a server, or a chip for implementing the method of the present application, and the embodiments of the present application do not make specific limitations. When the network device is a chip, the interface circuit in the chip can be used to perform the receiving or sending operation, and the processor in the chip can be used to perform the processing operation.

[0293] In one specific implementation, the embodiments of the present application also provide a chip including a processor and an interface circuit. The interface circuit is configured to receive an instruction and transmit the instruction to the processor. The processor can be configured to perform the operations of the network devices in the above data processing method. The processor is coupled with a memory, and the memory is configured to store a program or an instruction. When the program or the instruction is executed by the processor, the chip implements the method in any method embodiment described above.

[0294] Optionally, the processor in the chip can be one or more. The processor can be implemented by hardware and / or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor, which is implemented by reading software codes stored in a memory.

[0295] Optionally, the memory in the chip can also be one or more. The memory can be integrated with the processor, or can be arranged separately from the processor, which is not limited in the application. For example, the memory can be a non-transient processor, such as a read-only memory (ROM), which can be integrated on the same chip as the processor, or can be arranged on different chips respectively, and the type of the memory and the arrangement manner of the memory and the processor are not limited in the application.

[0296] For example, the chip can be a field programmable gate array (FPGA), can be an application-specific integrated circuit (ASIC), can be a system on chip (SoC), can be a central processor unit (CPU), can be a network processor (NP), can be a digital signal processor (DSP), can be a micro controller unit (MCU), can be a programmable logic device (PLD), or can be other integrated chips.

[0297] The embodiment of the application further provides a computer readable storage medium, which includes instructions or a computer program, and when the instructions or the computer program are run on a processor, the processor executes the data processing method provided in the above embodiment.

[0298] The embodiment of the application further provides a computer program product including instructions or a computer program, and when the instructions or the computer program are run on a processor, the data processing device executes the data processing method provided in the above embodiment.

[0299] The terms "first", "second", "third", "fourth", and the like in the description and in the claims of the present application, and above-described drawings (if there are) are used to distinguish similar objects, and do not necessarily have to be used to describe a particular sequential or chronological order. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products, or apparatuses.

[0300] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0301] In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical business division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0302] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0303] Those skilled in the art should realize that, in one or more of the above examples, the business described in the present application can be implemented by hardware, software, firmware or any combination thereof. When implemented by software, these businesses can be stored in a computer readable medium or transmitted as one or more instructions or codes on a computer readable medium. The computer readable medium includes computer storage medium and communication medium, wherein the communication medium includes any medium that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0304] The above detailed description of the application is only for the purpose of illustrating the technical solutions of the application, and is not intended to limit the application.

[0305] The above examples are only used to illustrate the technical solutions of the application, and are not intended to limit the application; although the application has been described in detail with reference to the foregoing examples, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing examples can still be changed, or some technical features thereof can be replaced by equivalents; and these changes or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the application.

Claims

1. A data processing method, characterized by, The application discloses a switching device applied to a distributed storage system, wherein the distributed storage system further comprises a client and N servers, N is a positive integer greater than 1. The method comprises: receiving first information sent by the client, wherein the first information is used for requesting to access first data from a first server of the N servers; sending target information to a second server, wherein the target information is determined based on the first information, the target information is used for requesting to access the first data from the second server, the second server is selected from (N-1) servers except the first server, and the target information comprises a packet sequence number, the packet sequence number matches state information of the second server, and the state information of the second server is used for indicating information planned to be received next time by the second server; receiving the first data from the second server; sending the first data to the client.

2. The method of claim 1, wherein, The packet sequence number in the target information is obtained by modifying the packet sequence number in the first information.

3. The method according to claim 1 or 2, characterized in that, The switching device stores a connection relationship, wherein the connection relationship is used for indicating that a remote direct memory access (RDMA) connection is established between the client and the second server.

4. The method according to any one of claims 1 to 3, characterized in that, The switching device is used for receiving and recording the state information of the second server, and the state information of the second server comprises the packet sequence number.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: obtaining a data type of the first data; if the data type of the first data is a first type, performing the step of sending the target information, and the first type is used for indicating that data is stored in a replication mode.

6. The method of claim 5, wherein, The method further comprises: if the data type of the first data is a second type, sending the first information to the first server, the second type is used for indicating that data is stored in an erasure code mode; receiving the first data from the first server; sending the first data to the client.

7. The method according to any one of claims 1 to 6, characterized in that, The first information comprises first keyword information. The method further comprises: performing in-network conflict detection based on the first keyword information and second keyword information of M ongoing target requests, obtaining a detection result, and M is a positive integer; if the detection result indicates that no conflict request corresponding to the first information is detected, performing the step of sending the target information to the second server.

8. The method of claim 7, wherein, The method further comprises: if the conflict request corresponding to the first information is detected, sending the first information to the first server; receiving update data from the first server, wherein the update data is obtained by updating the first data based on the conflict request corresponding to the first information; sending the update data to the client.

9. The method according to claim 7 or 8, characterized in that, The method further comprises: receiving second information sent by the client, wherein the second information is used for requesting to write second data in the first server; storing a target request corresponding to the second information, wherein the target request corresponding to the second information is used for performing in-network conflict detection.

10. The method according to any one of claims 7-9, characterized in that, The exchange device stores a state identifier corresponding to the target request, the state identifier being a first state identifier or a second state identifier, the first state identifier indicating that the target request is in progress, and the second state identifier indicating that the target request has been completed.

11. The method according to any one of claims 1 to 10, characterized in that, The client establishes an RDMA connection between the intelligent network card corresponding to the second server, and the target address in the first information is first keyword information. The method further includes: The method further includes: The method further includes: The method further includes:

12. The method according to any one of claims 1 to 11, characterized in that, The method further includes:

13. The method of claim 12, wherein, The target address of the target information is index information corresponding to the first keyword information, and the first keyword information is the target address of the first information.

14. A data processing method, characterized by, The check parameter in the target information is different from the check parameter in the first information. The target intelligent network card is applied to a distributed storage system, the distributed storage system including a client, an exchange device, N servers, and an intelligent network card corresponding to each server, a second server corresponding to the target intelligent network card belonging to the N servers, and N being a positive integer greater than 1. The method includes: The method includes: The method includes: The method includes:

15. The method of claim 14, wherein, The target intelligent network card establishes an RDMA connection with the client.

16. The method according to claim 14 or 15, characterized in that The method further includes: The method further includes: The check parameter in the access information is different from the check parameter in the target information.

17. The method according to any one of claims 14-16, characterized by, The target intelligent network card includes a first component and a second component, the first component being configured to update the target address of the target information, and the second component being configured to update the check parameter of the target information.

18. The method according to any one of claims 14-17, characterized by, The target intelligent network card includes a third component, the third component being configured to update the target address of the target information, and the third component being further configured to update the check parameter of the target information.

19. The method according to any one of claims 14-18, characterized by, ​ 20. A data processing method, characterized by, A second server applied to a distributed storage system, the distributed storage system comprising a client, a switching device and N servers, the second server being determined by the switching device from the N servers, N being a positive integer greater than 1; The method further comprises: receiving, by the switching device, access information, the access information comprising a packet sequence number, the packet sequence number matching state information of the second server, the state information of the second server being used to indicate information that the second server plans to receive next time; sending, to the switching device, the first data, the first data being obtained from a storage space of the second server based on the access information.

21. The method of claim 20, wherein, An RDMA connection is established between the second server and the client.

22. The method of claim 20 or 21, wherein, The method further comprises: sending, to the switching device, the state information.

23. The method of claim 22, wherein, An RDMA connection is established between the second server and the client. The method further comprises: sending, to the switching device, the state information.

24. The method of any one of claims 20-23, wherein, The access information comprises an index information, the index information being used to indicate a storage address of the first data in the second server.

25. The method of claim 24, wherein, The access information comprises a check parameter, the check parameter being different from a check parameter in first information, the first information being generated by the client and used to access the first data from a first server of the N servers.

26. A switching device, characterized by The distributed storage system further comprises the client and N servers, N being a positive integer greater than 1; The switching device comprises: a transceiver unit, configured to receive first information sent by the client, the first information being used to request to access first data from a first server of the N servers; The transceiver unit is further configured to send, to a second server, target information, the target information being determined based on the first information, the target information being used to request to access the first data from the second server, the second server being selected from (N-1) servers other than the first server, the target information comprising a packet sequence number, the packet sequence number matching state information of the second server, the state information of the second server being used to indicate information that the second server plans to receive next time; The transceiver unit is further configured to receive the first data from the second server; The transceiver unit is further configured to send, to the client, the first data.

27. A target intelligent network card, comprising: The distributed storage system further comprises a client, a switching device, N servers and an intelligent network card corresponding to each server, the second server corresponding to the target intelligent network card belonging to the N servers, N being a positive integer greater than 1; The target intelligent network card comprises: The transceiver unit is configured to receive target information sent by the switching device, the target information being used to request to access the first data from the second server, the target information including a data packet sequence number, the data packet sequence number matching state information of the second server, the state information of the second server being used to indicate information that the second server plans to receive next time, and a target address of the target information being first keyword information; The transceiver unit is further configured to send access information to the second server, a target address in the access information being index information corresponding to the first keyword information; The transceiver unit is further configured to receive the first data from the second server; The transceiver unit is further configured to send the first data to the switching device.

28. A second server, wherein the second server is configured to: The distributed storage system in which the second server is located further includes a client, a switching device and N servers, the second server belonging to the N servers, N being a positive integer greater than 1; The second server includes: A transceiver unit configured to receive access information from the switching device, the access information including a data packet sequence number, the data packet sequence number matching state information of the second server, the state information of the second server being used to indicate information that the second server plans to receive next time; The transceiver unit is further configured to send the first data to the switching device, the first data being obtained from a storage space of the second server based on the access information.

29. A distributed storage system, characterized by The distributed storage system includes a client, a switching device and N servers, N being a positive integer greater than 1; The client sends first information to the switching device, the first information being used to request to access first data from a first server of the N servers; The switching device sends target information to a second server, the target information being determined based on the first information, the target information being used to request to access the first data from the second server, the second server being a server selected from (N-1) servers other than the first server, the target information including a data packet sequence number, the data packet sequence number matching state information of the second server, the state information of the second server being used to indicate information that the second server plans to receive next time; The switching device receives the first data from the second server; The switching device sends the first data to the client.

30. A distributed storage system, characterized by The distributed storage system includes a client, a switching device, N servers and an intelligent network card corresponding to each server, N being a positive integer greater than 1; The client sends first information to the switching device, the first information being used to request to access first data from a first server of the N servers; The exchange device sends target information to a target intelligent network card, the target intelligent network card being an intelligent network card corresponding to a second server in the N servers, the target information including a data packet sequence number, the data packet sequence number matching state information of the second server, the state information of the second server being used to indicate information that the second server plans to receive next time, and a target address of the target information being keyword information; The target intelligent network card sends access information to the second server, a target address in the access information being index information corresponding to the keyword information; The target intelligent network card receives the first data from the second server; The target intelligent network card sends the first data to the exchange device; The exchange device sends the first data to the client.

31. A computing device, comprising: A computer readable storage medium stores instructions, when the instructions are run on a processor, to implement the method in any one of claims 1-13, or to implement the method in any one of claims 14-19, or to implement the method in any one of claims 20-25.

32. A computer-readable storage medium, comprising: The computer readable storage medium stores instructions, when the instructions are run on a processor, to implement the method in any one of claims 1-13, or to implement the method in any one of claims 14-19, or to implement the method in any one of claims 20-25.

33. A computer program product, characterised in that, The computer program product includes instructions, when the instructions are run on a computer, to cause the computer to implement the method in any one of claims 1-13, or to implement the method in any one of claims 14-19, or to implement the method in any one of claims 20-25.

Images