Data processing method and related devices

Abstract

Provided in the embodiments of the present application are a data processing method and related devices, which can be used in the field of cloud technology. The method comprises: a first device receives an RPC message sent by a second device, the RPC message comprising first address information of each piece of data to be processed among at least one piece of data to be processed; and then the first device uses a remote direct memory access mode to acquire one by one from the second device each piece of data to be processed corresponding to the first address information, and, on the basis of the acquired data to be processed, performs a data processing operation. In this way, an RPC message no longer needs to carry complete data to be processed, and can achieve the transmission of a large amount of data to be processed by simply carrying short address information; using the remote direct memory access mode to acquire data to be processed can avoid time overheads brought by procedures such as CPU resource scheduling and kernel trapping, thereby greatly improving data transmission efficiency and reducing delay during data transmission.

Description

A data processing method and related device

[0001] The present application claims priority from the Chinese patent application No. 202411378066.4 filed on September 29, 2024, and entitled "A data processing method and related device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Embodiments of the present application relate to the technical field of computer, and in particular, to a data processing method and related device. BACKGROUND

[0003] The remote procedure call (RPC) protocol is a computer communication protocol, and the RPC protocol allows a device to send an RPC message to a remote device to call an RPC service provided by the remote device. For example, the types of the RPC service can include data writing, data querying, and data computing. In the process of using the RPC message to call the RPC service between different devices, data transmission is often involved. In an application scenario where the amount of data to be processed is large, a large amount of data to be processed needs to be carried in the RPC message, which causes the problem of high latency in the data transmission process. SUMMARY

[0004] The present application provides a data processing method and related device for improving the efficiency of data transmission of data to be processed, so as to reduce the latency of the data transmission process.

[0005] The present application provides the following technical solutions:

[0006] In a first aspect, the present application provides a data processing method, which can be used in the field of cloud technology. The method comprises: a first device receives a first RPC message sent by a second device, wherein the first RPC message comprises first address information of each data to be processed in at least one data to be processed. Then, the first device can obtain the first address information of each data to be processed in the at least one data to be processed from the first RPC message, that is, at least one first address information is obtained. Further, the first device can obtain each data to be processed corresponding to the first address information in the at least one first address information from the second device in a remote direct memory access manner, and perform a data processing operation based on the obtained data to be processed. It should be noted that the first device can start to perform the data processing operation after obtaining all the data to be processed included in the at least one data to be processed. Alternatively, the data obtaining operation and the data processing operation can be performed alternately, which can be determined according to actual application scenarios.

[0007] Exemplarily, the first address information of the to-be-processed data can be a storage address of the to-be-processed data in the memory of the second device.

[0008] Exemplarily, obtaining the to-be-processed data corresponding to the first address information can be understood as obtaining the to-be-processed data at the storage location in the memory pointed by the first address information. The to-be-processed data corresponding to the first address information is obtained in a manner of remote direct memory access, which can bypass the operating system of the first device and be executed by a special hardware group. When the to-be-processed data corresponding to the first address information is obtained in the manner of remote direct memory access, the second processor of the first device can not be occupied, so that the operating system of the first device is also bypassed, and the time overhead caused by CPU resource scheduling and inherent processes such as kernel trap of the first device is avoided.

[0009] For example, the first device can call a first module in the first device to obtain the to-be-processed data corresponding to the first address information from the second device in a manner of remote direct memory access. Exemplarily, the first module can be a remote direct memory access (RDMA) module, and the first module can be an RDMA network card. Optionally, the RDMA network card can be integrated into a network communication module of the first device. Alternatively, the first module can also be a hardware group with a remote direct memory access capability or the like.

[0010] In the present implementation, the RPC message can include address information of the to-be-processed data. After receiving the RPC message, the first device can directly obtain the to-be-processed data corresponding to the address information by using the remote direct memory access mode, and then perform a data processing operation based on the obtained to-be-processed data. Therefore, the RPC message no longer needs to carry complete to-be-processed data. The RPC message only needs to carry short address information to realize the transmission of a large amount of to-be-processed data, greatly reducing the amount of data carried in the RPC message, thereby reducing the transmission delay of the RPC message. If the complete to-be-processed data is carried in the RPC message and then transmitted by transmitting the RPC message, because there are inherent processes outside data transmission such as CPU resource scheduling and kernel trapping when using system calls provided by the operating system in the process of obtaining the to-be-processed data, the transmission delay of the to-be-processed data is high. In the present application, the to-be-processed data corresponding to the first address information is directly obtained from the second device by using a dedicated hardware group, avoiding the time overhead caused by processes such as CPU resource scheduling and kernel trapping, thereby greatly improving the efficiency of the data transmission of the to-be-processed data and reducing the time delay of the data transmission process.

[0011] In a possible implementation, the first address information is contained in a header of the first RPC message and / or a data body of the first RPC message. In other words, the at least one first address information can be contained in the header of the first RPC message; or the at least one first address information can be contained in the data body of the first RPC message; or part of the at least one first address information is contained in both the header and the data body of the first RPC message.

[0012] For example, if the first RPC message is a packet, the header of the first RPC message can also be referred to as a packet header of the first RPC message, and the data body of the first RPC message can also be referred to as a packet body of the first RPC message. If the first RPC message is a data packet, the header of the first RPC message can also be referred to as a packet header of the first RPC message, and the data body of the first RPC message can also be referred to as a data payload of the first RPC message.

[0013] In the present implementation, since the speed of parsing the header of the RPC message is faster when the RPC message is parsed, the first address information is included in the header of the first RPC message, which is beneficial for the receiver (i.e., the first device) of the first RPC message to quickly obtain the first address information, and further beneficial for more quickly obtaining the to-be-processed data corresponding to the first address information; and the storage space of the data body of the RPC message is larger, which is beneficial for carrying more first address information and for meeting the scenario of needing to transmit multiple first address information.

[0014] In a possible implementation, in the case where the first address information is included in the header of the first RPC message, the header of the first RPC message further includes first information, and the first information indicates the positions of the first address information in the header of the first RPC message; for example, the positions of the different first address information in the at least one first address information in the header of the first RPC message are continuous, and the first information can indicate the position of the first first address information in the at least one first address information in the header of the first RPC message.

[0015] For example, in the case where the first address information is included in the data body of the first RPC message, the header of the first RPC message further includes second information, and the second information indicates the positions of the first address information in the data body of the first RPC message; for example, the positions of the different first address information in the at least one first address information in the data body of the first RPC message are continuous, and the first information can indicate the position of the first first address information in the at least one first address information in the data body of the first RPC message.

[0016] In the present implementation, the header of the first RPC message can further carry first information indicating the positions of the first address information in the header of the first RPC message, and / or second information indicating the positions of the first address information in the data body of the first RPC message, so that the first device can quickly find the first address information by means of the first information and / or the second information, and further beneficial for more quickly obtaining the to-be-processed data corresponding to the first address information.

[0017] In a possible implementation, the header of the first RPC message further comprises at least one of the following: a first quantity of first address information contained in the header of the first RPC message, the first quantity referring to a quantity of at least one first address information contained in the header of the first RPC message; a second quantity of first address information contained in a data body of the first RPC message, the second quantity referring to a quantity of at least one first address information contained in the header of the first RPC message; a third quantity of at least one first address information corresponding to the at least one to-be-processed data, the third quantity referring to a total quantity of all first address information corresponding to the at least one to-be-processed data; or a data quantity of each to-be-processed data in the at least one to-be-processed data, which can also be referred to as a size of each to-be-processed data.

[0018] In the implementation, the header of the first RPC message can further comprise the quantity of first address information and / or the data quantity of to-be-processed data. The quantity information facilitates the first device to comprehensively understand the first address information in the first RPC message, and the first device can check the first address information in the first RPC message by means of the quantity information in the header of the first RPC message, so as to avoid omission of the first address information, and improve accuracy of the to-be-processed data acquisition process. The data quantity of to-be-processed data also facilitates the first device to more quickly and accurately acquire to-be-processed data.

[0019] In a possible implementation, the type of the application scenario corresponding to the first RPC message is at least one of data writing, data querying, or data computing. For example, the type of the first RPC message is an RPC request, and the RPC service requested by the first RPC message can be data writing, that is, the type of the application scenario corresponding to the first RPC message is data writing, and each to-be-processed data can be understood as data to be written, and the data processing operation can comprise data disk writing of each to-be-processed data.

[0020] For another example, the type of the application scenario corresponding to the first RPC message is data querying, and the first RPC message can be an RPC response (ack), an RPC reply (response), or an RPC response+RPC reply, each to-be-processed data can be understood as queried data, and the data processing operation can comprise storing each to-be-processed data.

[0021] For another example, the type of the first RPC message is an RPC request, and the RPC service requested by the first RPC message can also be data computing, that is, the type of the application scenario corresponding to the first RPC message is data computing, and each to-be-processed data can be understood as data to be used in the computing process, and the data processing operation can comprise data computing based on each to-be-processed data.

[0022] In a possible implementation, the first device comprises RPC software and RPC hardware, wherein the RPC software and the RPC hardware both have processing capability defined by the RPC protocol. The RPC software is deployed on an operating system of the first device. The RPC hardware is hardware for providing the RPC service; for example, the RPC hardware can comprise a separate first processor, i.e., does not need to occupy a second processor of the first device, thereby being able to process the RPC message independently of the second processor of the first device. The RPC software and the RPC hardware intercommunicate, i.e., the RPC software and the RPC hardware can realize mutual communication within the first device.

[0023] In the implementation, the RPC hardware independently provides the RPC service outside the operating system of the first device, thereby avoiding time consumption caused by the kernel (i.e., the kernel in the operating system of the first device) when the RPC hardware provides the RPC service, having the advantage of fast processing speed, the RPC software is more convenient to provide complex RPC services, and the RPC software has stronger scalability. The first device simultaneously deploys the RPC software and the RPC hardware, the RPC software and the RPC hardware intercommunicate, has the synergy of the RPC software and the RPC hardware, and is beneficial to providing more efficient RPC services, supporting more complex RPC services, and having stronger scalability.

[0024] In a possible implementation, the first device further comprises a network communication module, and functions of the network communication module include receiving and sending the RPC message. Functions of the RPC hardware can include: obtaining the received RPC message from the network communication module, transmitting the first RPC request corresponding to the received RPC message to the RPC software, executing the processing operation corresponding to the second RPC request corresponding to the received RPC message, transmitting the RPC message generated by the RPC software to the network communication module, and transmitting the RPC message generated by the RPC hardware to the network communication module.

[0025] In the implementation, the RPC hardware reads the RPC message faster, and the RPC hardware is deployed between the RPC software and the network communication module, thereby being convenient to understand the RPC message faster. The RPC hardware can transfer part of the RPC request (e.g., the first RPC request) to the RPC software for execution, and the RPC hardware can process part of the RPC request (e.g., the second RPC request), thereby refining the specific synergy mode of the RPC software and the RPC hardware, and being beneficial to providing more efficient RPC services, supporting more complex RPC services, and having stronger scalability.

[0026] In a possible implementation, in the case that the data processing operation is to write at least one to-be-processed data into the memory of the first device, the method is executed by the RPC hardware in the first device, that is, the receiving of the first RPC message, the obtaining of the to-be-processed data, and the data processing operation are all executed by the RPC hardware in the first device.

[0027] In the present implementation, in the case that the data processing operation corresponding to the first RPC message is data writing, the first RPC message is received by the RPC hardware, the to-be-processed data corresponding to the first address information is obtained from the second device in a manner of remote direct memory access, and the to-be-processed data is written into the memory of the first device. Since data writing is a logically simple RPC service, it is easy to implement the foregoing steps by using the RPC hardware completely, and using the RPC hardware completely to execute the foregoing steps is conducive to avoiding the operating system of the first device in the entire process of data writing, to avoid the time overhead caused by trapping into the kernel, and to further improve the efficiency of the process of data writing.

[0028] In a possible implementation, the method further includes: the first device receives a second RPC message, the type of the RPC request corresponding to the second RPC message is data query, and the second RPC message includes a logical table and / or an index of each to-be-queried data in at least one to-be-queried data; second address information of each to-be-queried data is determined according to the logical table and / or the index of each to-be-queried data, the second address information of each to-be-queried data can also be referred to as second address information of the queried data, and the second address information of each to-be-queried data can also be understood as a storage address of the queried data in the memory of the first device; the first device can write at least one second address information corresponding to at least one to-be-queried data into a third RPC message, and send the third RPC message corresponding to the second RPC message to a third device, where the third RPC message includes the second address information of each to-be-queried data, and the second address information is used for the third device to obtain data corresponding to the second address information from the first device in a manner of remote direct memory access.

[0029] In the present implementation, after receiving the RPC request of data query, the first device can no longer carry the complete queried data in the RPC reply message (that is, the third RPC message), but carry the second address information of the queried data, and then the third device obtains data corresponding to the second address information from the first device in a manner of remote direct memory access, so that the third device avoids the time overhead caused by CPU resource scheduling and the process of trapping into the kernel in the process of obtaining the data to be queried, and is conducive to quickly obtaining the data to be queried.

[0030] In a possible implementation, when the type of the RPC request corresponding to the second RPC message received by the first device is a data query, the second RPC message is obtained by the RPC hardware in the first device from the network communication module of the first device, the operation of determining the second address information of each data to be queried is performed by the RPC software in the first device, and the third RPC message corresponding to the second RPC message is transmitted by the RPC hardware in the first device through the network communication module of the first device.

[0031] In the implementation, the second RPC message corresponding to the data query is obtained by the RPC hardware from the network communication module, the RPC software is used to perform the data query to obtain the second address information of the data to be queried, and then the third RPC message corresponding to the second RPC message is transmitted by the RPC hardware to the network communication module, so that the third RPC message is transmitted, the capability of the RPC hardware to quickly read the RPC message is utilized, the RPC software is used to quickly determine the second address information of each data to be queried, and the data to be queried can be quickly obtained by the third device.

[0032] In a possible implementation, when the data processing operation is a calculation operation based on at least one data to be processed, the operation of obtaining the data to be processed is performed by the RPC hardware, and the data calculation operation based on the data to be processed is performed by the RPC software.

[0033] In the implementation, the data processing operation is the calculation operation based on at least one data to be processed, the operation of obtaining the data to be processed is performed by the RPC hardware, and the calculation operation is performed by the RPC software, so that the RPC hardware is used to quickly obtain the data to be processed, and the RPC software is used to implement complex data calculation, thereby providing better RPC service.

[0034] In a possible implementation, the first device is a storage node, and the second device is a calculation node, or the first device is a calculation node, and the second device is a storage node. In the implementation, the first device and the second device are specifically the calculation node and the storage node, that is, the specific forms of the first device and the second device in the method provided in the application are determined, and the implementability of the scheme is improved.

[0035] In a second aspect, the application provides a data processing method, which can be applied to the field of cloud technology and includes the following steps: a second device obtains a first RPC message, wherein the first RPC message includes first address information of each data to be processed in at least one data to be processed; and the second device sends the first RPC message to a first device, and the first address information is used to enable the first device to obtain the data to be processed corresponding to the first address information in a manner of remote direct memory access.

[0036] In a possible implementation, the first address information is contained in a header of the first RPC message and / or a data body of the first RPC message.

[0037] In a possible implementation, when the first address information is contained in the header of the first RPC message, the header of the first RPC message further includes first information indicating a position of the first address information in the header of the first RPC message; and / or, when the first address information is contained in the data body of the first RPC message, the header of the first RPC message further includes second information indicating a position of the first address information in the data body of the first RPC message.

[0038] In a third aspect, the present application provides a device, which can be used in the field of cloud technology, and the device includes an RPC message processing system, which is configured to: receive a first remote procedure call RPC message sent by a second device, wherein the first RPC message includes first address information of each of at least one to-be-processed data; acquire the to-be-processed data corresponding to the first address information from the second device in a manner of remote direct memory access; and perform a data processing operation based on the to-be-processed data.

[0039] In a possible implementation, the first address information is contained in a header of the first RPC message and / or a data body of the first RPC message.

[0040] In a possible implementation, when the first address information is contained in the header of the first RPC message, the header of the first RPC message further includes first information indicating a position of the first address information in the header of the first RPC message; and / or, when the first address information is contained in the data body of the first RPC message, the header of the first RPC message further includes second information indicating a position of the first address information in the data body of the first RPC message.

[0041] In a possible implementation, the header of the first RPC message further includes at least one of the following information: a quantity of the first address information contained in the header of the first RPC message, a quantity of the first address information contained in the data body of the first RPC message, a quantity of at least one first address information corresponding to the at least one to-be-processed data, or a data quantity of each to-be-processed data.

[0042] In a possible implementation, the RPC message processing system includes an RPC software and an RPC hardware, wherein the RPC software and the RPC hardware both have processing capabilities defined by an RPC protocol, the RPC software is deployed on an operating system of the device, the RPC hardware is hardware for providing an RPC service, and the RPC software and the RPC hardware intercommunicate.

[0043] In a possible implementation, the device further includes a network communication module, functions of the network communication module include receiving and sending RPC messages, functions of the RPC hardware include: obtaining the received RPC messages from the network communication module, transmitting the first RPC request corresponding to the received RPC message to the RPC software, performing the processing operation corresponding to the second RPC request corresponding to the received RPC message, transmitting the RPC message generated by the RPC software to the network communication module, and transmitting the RPC message generated by the RPC hardware to the network communication module.

[0044] In a possible implementation, the data processing operation is writing the at least one to-be-processed data into the memory of the first device, and the method is performed by the RPC hardware in the first device.

[0045] In a possible implementation, the processing system of the RPC message is further configured to: receive a second RPC message, a type of an RPC request corresponding to the second RPC message is a data query, and the second RPC message includes a logical table and / or an index of each to-be-queried data in the at least one to-be-queried data; the first device determines second address information of each to-be-queried data according to the logical table and / or the index of each to-be-queried data; and the first device sends a third RPC message corresponding to the second RPC message to the third device, the third RPC message including the second address information of each to-be-queried data, and the second address information is used for the third device to obtain data corresponding to the second address information from the first device in a manner of remote direct memory access.

[0046] In a possible implementation, the second RPC message is obtained by the RPC hardware from the network communication module of the first device, the determination of the second address information of each to-be-queried data is performed by the RPC software, and the third RPC message is sent by the RPC hardware through the network communication module.

[0047] In a possible implementation, the data processing operation is a calculation operation based on the at least one to-be-processed data, the obtaining of the to-be-processed data is performed by the RPC hardware, and the calculation operation is performed by the RPC software.

[0048] In a possible implementation, the first device and the second device are devices in a same database system, where the first device is a storage node in the database system, and the second device is a computing node in the database system; or the first device is a computing node in the database system, and the second device is a storage node in the database system.

[0049] In a fourth aspect, the present application provides a device, which can be applied to the technical field of cloud technology. The device comprises an RPC message processing system, which is configured to: obtain a first RPC message, wherein the first RPC message comprises first address information of each piece of to-be-processed data; and send the first RPC message to a first device, wherein the first address information is used by the first device to obtain the to-be-processed data corresponding to the first address information by using a remote direct memory access mode.

[0050] In a fifth aspect, the present application provides a computing device cluster, which comprises at least one computing device. Each computing device comprises a processor and a memory. The memory is configured to store instructions. The processor is configured to execute the method according to the first aspect or the second aspect according to the instructions.

[0051] In a sixth aspect, the present application provides a computer storage medium, which stores one or more instructions. When the instructions are executed by one or more computers, the one or more computers execute the method according to the first aspect or the second aspect.

[0052] In a seventh aspect, the present application provides a computer program product, which stores instructions. When the instructions are executed by a computer, the computer executes the method according to the first aspect or the second aspect.

[0053] The second aspect to the seventh aspect of the present application correspond to the first aspect or various possible manners of the first aspect, and have corresponding beneficial effects. BRIEF DESCRIPTION OF DRAWINGS

[0054] FIG. 1 is a structural schematic diagram of a cloud service system according to an embodiment of the present application;

[0055] FIG. 2 is a flow schematic diagram of a data processing method according to an embodiment of the present application;

[0056] FIG. 3 is a schematic diagram of a header of an RPC message according to an embodiment of the present application;

[0057] FIG. 4 is a schematic diagram of RPC software and RPC hardware according to an embodiment of the present application;

[0058] FIG. 5 is another flow schematic diagram of a data processing method according to an embodiment of the present application;

[0059] FIG. 6 is a flow schematic diagram of data writing to a disk according to an embodiment of the present application;

[0060] FIG. 7 is a flow schematic diagram of data query according to an embodiment of the present application;

[0061] FIG. 8 is a flow schematic diagram of data calculation according to an embodiment of the present application;

[0062] FIG. 9 is a structural schematic diagram of an apparatus provided by an embodiment of the present application;

[0063] FIG. 10 is a structural schematic diagram of a computing apparatus provided by an embodiment of the present application;

[0064] FIG. 11 is a structural schematic diagram of a computing apparatus cluster provided by an embodiment of the present application;

[0065] FIG. 12 is a schematic diagram of a network connection of computer apparatuses in a computer cluster provided by an embodiment of the present application. DETAILED DESCRIPTION

[0066] The terms "first", "second", and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the terms so used are interchangeable under appropriate circumstances, and that the embodiments of the present application are capable of functioning in other sequences than described or otherwise illustrated herein. In addition, the terms "comprise", "comprising", "include", "including", and the like are meant to encompass non-exclusive inclusions, so that a process, method, system, product, or apparatus that comprises a list of elements is not necessarily limited to those elements, but can include other elements not expressly listed or inherent to such process, method, system, product, or apparatus.

[0067] In the present application, "sending" and "receiving" refer to the direction of signal transmission. For example, "sending information to XX apparatus" can be understood as that the destination of the information is the XX apparatus, which can include direct transmission through the air interface, or indirect transmission through the air interface by other units or modules. "Receiving information from YY apparatus" can be understood as that the source of the information is the YY apparatus, which can include direct reception from the YY apparatus through the air interface, or indirect reception from the YY apparatus through the air interface by other units or modules. "Sending" can also be understood as "output" of a chip interface, and "receiving" can also be understood as "input" of a chip interface. In other words, sending and receiving can be between devices, or within a device, such as between components, modules, chips, software modules, or hardware modules within a device through a bus, wire, or interface. It can be understood that the information between the source and the destination of the information transmission can be processed as necessary, such as encoding and modulation, but the destination can understand the valid information from the source. Similar expressions in the present application can be similarly understood, and will not be repeated here.

[0068] The indication in the present application can include direct indication and indirect indication, and can also include explicit indication and implicit indication. The information indicated by certain information (indication information described below) is referred to as to-be-indicated information. In the implementation process, there are many ways to indicate the to-be-indicated information, for example, the to-be-indicated information can be directly indicated, such as the to-be-indicated information itself or the index of the to-be-indicated information. The to-be-indicated information can also be indirectly indicated by indicating other information, where the other information and the to-be-indicated information have an association relationship. The to-be-indicated information can also be indicated only by a part, and the other part of the to-be-indicated information is known or agreed in advance. For example, the arrangement order of each information agreed in advance (for example, protocol predefined) can be used to indicate a specific information, thereby reducing the indication overhead to a certain extent. The present application does not limit the specific manner of indication. It can be understood that the indication information can be used to indicate the to-be-indicated information for the sender of the indication information, and the indication information can be used to determine the to-be-indicated information for the receiver of the indication information.

[0069] The method provided by the present application can be applied to an application scenario using a remote procedure call (RPC) technology, where the RPC protocol is a computer communication protocol, and the RPC protocol allows a device to send an RPC message to a remote device to call an RPC service provided by the remote device, for example, the types of the RPC service can include data landing, data query and data calculation, and the data landing in the present application can also be referred to as data writing. The process of calling the RPC service between different devices using the RPC message often involves data transmission. In the related technology, the RPC message often carries complete to-be-processed data. However, in the scenario where the data amount of the to-be-processed data is large, the transmission delay of the RPC message carrying the complete to-be-processed data is high, that is, the data transmission process of the RPC message has a high delay problem. In order to solve the foregoing problem, the present application discloses that the RPC message can carry first address information of each to-be-processed data in at least one to-be-processed data. After receiving the RPC message sent by the second device, the first device can acquire the to-be-processed data corresponding to the first address information from the second device in a remote direct memory access manner, and then perform a data processing operation based on the acquired to-be-processed data. Exemplarily, the first address information can be understood as a storage address of the to-be-processed data in a memory, and acquiring the to-be-processed data corresponding to the first address information can be understood as acquiring the to-be-processed data at the storage position of the memory pointed by the first address information.

[0070] In the present implementation, since the first device can directly obtain the to-be-processed data corresponding to the first address information in a remote direct memory access manner after receiving the RPC message, the RPC message no longer needs to carry complete to-be-processed data, and the RPC message can realize the transmission of a large amount of to-be-processed data by carrying only short address information, greatly reducing the amount of data carried in the RPC message, thereby reducing the transmission delay of the RPC message; if complete to-be-processed data is carried in the RPC message and then the to-be-processed data is transmitted by transmitting the RPC message, since there are inherent processes outside data transmission such as CPU resource scheduling and kernel trapping when a system call provided by an operating system is used in the foregoing process of obtaining to-be-processed data, the transmission delay of the to-be-processed data is high, and in the present application, the to-be-processed data corresponding to the first address information is directly obtained from the second device in a remote direct memory access manner by using a dedicated hardware group, avoiding the time overhead caused by CPU resource scheduling and kernel trapping and the like inherent processes, thereby greatly improving the efficiency of the foregoing data transmission of to-be-processed data to reduce the time delay of the data transmission process.

[0071] Optionally, in an application scenario, the above RPC technology can be applied to a database system. Optionally, the database system can be a cloud service system. To more intuitively understand the present solution, the cloud service system will first be introduced in combination with FIG. 1. FIG. 1 is a structural schematic diagram of a cloud service system provided by an embodiment of the present application. As shown in FIG. 1, the cloud service system includes infrastructure for providing cloud services, which includes a plurality of computing nodes and a plurality of storage nodes. Any one computing node is in communication connection with a plurality of storage nodes, that is, any one computing node can access a plurality of storage nodes. For example, the computing node can send an RPC message to the storage node, and the storage node can also send an RPC message to the computing node. In other words, the first device can be a computing node in the cloud service system, and the second device can be a storage node in the cloud service system. Alternatively, the first device can be a storage node in the cloud service system, and the second device can be a computing node in the cloud service system. The specific application scenario can be determined according to actual needs. The computing node and the storage node will be introduced below.

[0072] The computing node, which can also be referred to as a distributed database middleware (DDM) node, multiple DDM nodes can be as a whole, and present to the user's client as a DDM. When the user's client accesses the DDM, it can randomly access any one of the multiple DDM nodes contained in the DDM. The storage node, which can also be referred to as a database node, multiple database nodes can be used to store the user's data.

[0073] Exemplarily, for any one of the multiple computing nodes or any one of the multiple storage nodes, it can be understood in essence as a cloud instance in the cloud service system, and the cloud instance can be presented in various ways, for example, the cloud instance can be a physical server specified by the cloud management platform, for example, the cloud instance can also be a virtual machine (VM) created by the cloud management platform on the physical server through virtualization technology, for example, the cloud instance can also be a container (docker) created by the cloud management platform on the physical server through virtualization technology, for example, the cloud instance can also be a micro-VM created by the cloud management platform on the physical server through virtualization technology, for example, the cloud instance can also be a container (docker) created by the cloud management platform on the physical server through virtualization technology, for example, the cloud instance can also be a bare metal server specified by the cloud management platform, and the like.

[0074] Further, the multiple computing nodes or the multiple storage nodes can be generally arranged in one site or multiple sites, and the site can be presented in various forms, for example, one or more sites can be multiple regions in the infrastructure, for example, one or more sites can be multiple availability zones in the infrastructure, for example, one or more sites can be at least one edge site and at least one central cloud site in the infrastructure, and the like.

[0075] In the embodiments of the present application, it is clear that the first device and the second device are specifically the computing node and the storage node, that is, the specific form of the first device and the second device in the method provided by the present application is clear, and the realizability of the present scheme is improved.

[0076] Alternatively, in another application scenario, the transmission of the RPC message from the sending end to the receiving end can pass through the forwarding of the gateway, and the first device can also be a gateway, and the second device can be a sending end of the RPC message.

[0077] Or, in another application scenario, RPC messages can also be transmitted between the client and the cloud server, for example, the client can send an RPC message to the cloud server, then the first device can be the cloud server and the second device can be the client; for another example, the cloud server can send an RPC message to the client, and then the first device can be the client and the second device can be the cloud server, and the application scenarios of the present application are not exhausted here, and the specific conditions can be determined according to the actual application scenario.

[0078] In combination with the above description, the specific implementation process of the data processing method provided by the present application will be described below. Please refer to FIG. 2, which is a flowchart of the data processing method provided by an embodiment of the present application. The data processing method provided by the present application can include:

[0079] 201. The second device acquires a first RPC message, wherein the first RPC message includes first address information of each of the at least one to-be-processed data.

[0080] Exemplarily, the first RPC message (message) can be a packet or a data packet. The type of the application scenario corresponding to the first RPC message is at least one of data writing, data querying, or data computing.

[0081] For example, the type of the first RPC message is an RPC request, and the RPC service requested by the first RPC message can be data writing, that is, the type of the application scenario corresponding to the first RPC message is data writing, and each to-be-processed data can be understood as data to be written. When the second device wants to perform data writing on the at least one to-be-processed data, the second device can acquire the first address information of each of the at least one to-be-processed data, and then generate the first RPC message. In the present application, the first address information of the to-be-processed data can be the storage address of the to-be-processed data in the memory of the second device.

[0082] For another example, the type of the first RPC message is an RPC request, and the RPC service requested by the first RPC message can also be data computing, that is, the type of the application scenario corresponding to the first RPC message is data computing, and each to-be-processed data can be understood as data used in the computing process. When the second device wants to perform data computing based on the at least one to-be-processed data, the second device can acquire the first address information of each of the at least one to-be-processed data, and then generate the first RPC message.

[0083] Optionally, the type of the first RPC message can also be any one of the following: RPC response (ack), RPC response (response), RPC response + RPC response, or the type of other RPC messages, etc. For example, in the case of the type of the application scenario corresponding to the first RPC message being data query, after the second device receives the RPC request of data query, the second device can obtain at least one logical table and / or index carried in the RPC request, perform data query based on the at least one logical table and / or index, obtain the first address information of each of the at least one queried data, and then reply and / or respond by using the first RPC message. Each to-be-processed data can be understood as each of the at least one queried data, and the first address information of each to-be-processed data can be the storage address of each queried data in the storage.

[0084] For example, the at least one logical table and / or index includes table 1 + key 1, table 2 + key 2, and table 3 + key 3, the first address information of queried data 1 (i.e., to-be-processed data 1) can be obtained by performing data query based on table 1 + key 1, the first address information of queried data 2 (i.e., to-be-processed data 2) can be obtained by performing data query based on table 2 + key 2, and the first address information of queried data 3 (i.e., to-be-processed data 3) can be obtained by performing data query based on table 3 + key 3. The first RPC message can include the first address information of to-be-processed data 1, the first address information of to-be-processed data 2, and the first address information of to-be-processed data 3. It should be understood that the examples herein are only for the convenience of understanding the present solution and do not limit the present solution.

[0085] For example, the header of the first RPC message can carry the type of the first RPC message. For example, when the type of a certain RPC message is 0, it represents that the type of the RPC message is RPC request. For example, when the type of a certain RPC message is 1, it represents that the type of the RPC message is RPC response. For example, when the type of a certain RPC message is 2, it represents that the type of the RPC message is RPC response. For example, when the type of a certain RPC message is 3, it represents that the type of the RPC message is RPC response + RPC response, etc. The examples herein are only for the convenience of understanding the present solution, and the specific type can be determined in combination with the actual application scenario.

[0086] The at least one first address information corresponding to the at least one to-be-processed data can be contained in a header of the first RPC message and / or a data body of the first RPC message. In other words, the at least one first address information can be contained in the header of the first RPC message; or the at least one first address information can be contained in the data body of the first RPC message; or part of the at least one first address information can be contained in the header of the first RPC message and the data body of the first RPC message.

[0087] If the first RPC message is a packet, the header of the first RPC message can also be referred to as a packet header of the first RPC message, and the data body of the first RPC message can also be referred to as a data payload of the first RPC message.

[0088] Optionally, in the case where the first address information is contained in the header of the first RPC message, the first address information can be carried in an extension header of the header of the first RPC message. In other words, the field for carrying the first address information in the header of the first RPC message can be understood as a field in the extension header of the first RPC message.

[0089] Optionally, the at least one first address information corresponding to the at least one to-be-processed data can be preferentially carried in the header of the first RPC message. In the case where the storage space of the header of the first RPC message is not enough to write all the first address information in the at least one first address information corresponding to the at least one to-be-processed data, the first address information is further carried in the data body of the first RPC message. For example, in the process of writing the at least one first address information corresponding to the at least one to-be-processed data into the first RPC message, the second device can preferentially start writing the first address information in the header of the first RPC message. In the case where the space of the header of the first RPC message is not enough, the remaining first address information is continuously written in the data body of the first RPC message.

[0090] In the embodiments of the present application, since the speed of parsing the header of the RPC message is faster when parsing the RPC message, the first address information is contained in the header of the first RPC message, which is beneficial for the receiver (i.e., the first device) of the first RPC message to quickly obtain the first address information, and further beneficial for more quickly obtaining the to-be-processed data corresponding to the first address information; and the storage space of the data body of the RPC message is larger, which is beneficial for carrying more first address information and for meeting the scenario of needing to transmit multiple first address information.

[0091] Optionally, in the case that the first address information is contained in the header of the first RPC message, the header of the first RPC message further includes first information, the first information indicates the positions of the at least one first address information in the header of the first RPC message, and the first information is used for the second device to locate the at least one first address information in the header of the first RPC message according to the first information; for example, the positions of different first address information in the at least one first address information in the header of the first RPC message are continuous, and the first information can indicate the position of the first first address information in the at least one first address information in the header of the first RPC message.

[0092] And / or, in the case that the first address information is contained in the data body of the first RPC message, the header of the first RPC message further includes second information, the second information indicates the positions of the first address information in the data body of the first RPC message, and the second information is used for the second device to locate the at least one first address information in the header of the first RPC message according to the first information; for example, the positions of different first address information in the at least one first address information in the data body of the first RPC message are continuous, and the first information can indicate the position of the first first address information in the at least one first address information in the data body of the first RPC message.

[0093] Optionally, the first information can indicate the offset of the position of the first first address information in the header of the first RPC message relative to the starting position of the extension header of the RPC message; and the second information can indicate the offset of the position of the first first address information in the data body of the first RPC message relative to the starting position of the extension header of the first RPC message.

[0094] In the embodiments of the present application, the header of the first RPC message can further carry the first information indicating the positions of the first address information in the header of the first RPC message, and / or the second information indicating the positions of the first address information in the data body of the first RPC message, so that the first device can quickly find the first address information by means of the first information and / or the second information, and further beneficial for more quickly obtaining the to-be-processed data corresponding to the first address information.

[0095] Optionally, the header of the first RPC message further comprises at least one of the following information: a first quantity of first address information contained in the header of the first RPC message, in other words, the first quantity refers to the quantity of at least one first address information contained in the header of the first RPC message; a second quantity of first address information contained in the data body of the first RPC message, in other words, the second quantity refers to the quantity of at least one first address information contained in the header of the first RPC message; a third quantity of at least one first address information corresponding to the at least one to-be-processed data, in other words, the third quantity refers to the total quantity of all first address information corresponding to the at least one to-be-processed data; a data quantity of each to-be-processed data, the data quantity of each to-be-processed data can also be alternatively referred to as the size of each to-be-processed data; or other information related to the first address information, and the like. Optionally, the unit of measurement of the size of each to-be-processed data can be byte or other units, and the like.

[0096] In the embodiments of the present application, the header of the first RPC message can further comprise the quantity of first address information and / or the data quantity of to-be-processed data. The aforementioned quantity information facilitates the first device to more comprehensively understand the first address information in the first RPC message, and then the first device can check the first address information in the first RPC message by means of the quantity information in the header of the first RPC message, so as to avoid omission of the first address information, and is conducive to improving the accuracy of the to-be-processed data acquisition process. The data quantity of to-be-processed data also facilitates the first device to more quickly and accurately acquire the to-be-processed data.

[0097] It should be noted that the header of the first RPC message can have more or less information, which can be determined in combination with actual application scenarios. In order to understand the present scheme more intuitively, please refer to FIG. 3, which is a schematic diagram of a header of an RPC message provided by an embodiment of the present application. FIG. 3 shows the structure of the header of the RPC message, and the structure of the extension header in the header of the RPC message. The meaning of each field in the header of the RPC message shown in FIG. 3 is introduced as follows. The first version (Version) field is used to store the version number of the RPC; the type (Type) field is used to indicate the type of the RPC message; the response (Ack) field indicates whether the RPC message needs to return an RPC response, and the receiving end of the RPC message can autonomously decide whether the RPC Ack and the RPC response are returned in combination; the content summary (Argument) direct memory access (DMA) quantity (Count) field indicates the number of Argument DMAs performed by the receiving end of the RPC message, and when the value in the Argument DMA Count field is 0, it means that the Argument DMA table does not exist; the function (Function) field indicates the RPC service function required to be called by the RPC message; the request total size (Request Total Size) field indicates the size of the complete RPC request, and the aforementioned size can include not only the size of the entire RPC message, but also the size of at least one to-be-processed data; the request identity (ID) field is the unique RPC call identity for calling the RPC service by using the RPC request, and the value in the Request ID field is used to match the RPC request with the RPC response and / or the RPC reply; the client’s RPC channel (Client’s RPC Channel) field indicates the RPC Channel adopted by the client (Client) side when sending the RPC request and receiving the RPC response and / or the RPC reply, and the client side in the present application can also be understood as the sending end of the RPC request; the function defined (Function Defined) field indicates the type of the service provided by the extension header of the RPC message, and the Function Defined field can be used to indicate how to parse the content of the extension header, for example, the type of the service provided by the extension header of the RPC message in the present application is to carry the first address information corresponding to the to-be-processed data, so that the complete to-be-processed data no longer needs to be carried in the RPC message.

[0098] The contents of the fields in the extension header in the header of the RPC message are described as follows. The Version field in the extension header indicates the version number of the extension header. The Data Trans Mode field indicates the supported data transmission mode. The User Hdr Offset field indicates the offset of the first address information in the header of the RPC message relative to the start position of the extension header of the RPC message. The User Data Offset field indicates the offset of the first address information in the data body of the RPC message relative to the start position of the extension header of the RPC message. The DMA's Count In User Data Zone field indicates the second number of all the first address information contained in the data body of the RPC message. The Local Queue ID field is used as an index to associate the context information in the queue (Queue) of the local (i.e., the sending end of the RPC message) when the receiving end of the RPC message replies to the message. The Server's RPC Channel field indicates a channel identification (channel id) allocated by a server (Server) for a client side. Again, in the example of the message structure shown in FIG. 3, the client can be understood as the sending end of the RPC message, and the Server can be understood as the receiving end of the RPC message.

[0099] The Count In Hdr Zone field of the DMA indicates the first quantity of the first address information contained in the header of the RPC message; the Local Queue Token Value field is used for legitimacy check when the fast response mode is adopted; the Data DMA's Address field indicates the first address information of the data to be processed; the Data DMA's Size field indicates the size of the data to be processed obtained by the remote direct memory access; the Data DMA's Token ID field indicates the value in the token ID field of the data to be processed obtained by the remote direct memory access, which is used for legitimacy check in the process of obtaining the data to be processed by the remote direct memory access; the value in the Data DMA's Token value field of the data to be processed obtained by the remote direct memory access is used for legitimacy check in the process of obtaining the data to be processed by the remote direct memory access; the Data DMA's Flag field is the flag of the first address information, for example, when the value in the Data DMA's Flag field is 0, it can indicate reading the data at this address, and when the value in the Data DMA's Flag field is 1, it can indicate writing data to this address; the Rsvd field represents the reserved field; the Security Hdr field is used to store the information used when the encryption is started, the decryption is started, and the integrity check is performed, and it should be understood that the example in FIG. 3 is only for the convenience of understanding the scheme, and the field values of other fields can also be included in the extension header in the header of the RPC message, for example, the Early Response field can also be included in the extension header of the RPC message, the Early Response field indicates that the receiving end of the RPC message adopts the Early Response mode and no longer needs to return the RPC reply; for another example, the Flow Control Flag field can also be included in the extension header of the RPC message, the Flow Control Flag field indicates whether the sending of the RPC message is scheduled or unscheduled, etc., and the specific fields contained in the header of the RPC message can be determined in combination with the actual application scenario, and are not used to limit the scheme.

[0100] Optionally, in step 201, the second device can further determine whether the total data amount of the at least one to-be-processed data is less than or equal to a preset threshold. If the total data amount of the at least one to-be-processed data is less than or equal to the preset threshold, the second device can directly write the complete at least one to-be-processed data in the first RPC message, and no longer carry the first address information of the to-be-processed data in the first RPC message to obtain the first RPC message. Correspondingly, the first device (i.e., the receiving device of the first RPC message) can directly obtain the complete at least one to-be-processed data from the first RPC message. Optionally, the value in the field in the header of the first RPC message indicating the third number of the first address information of the at least one to-be-processed data can be set to 0, so as to indicate that 0 first address information is carried in the first RPC message. Alternatively, the value in the field in the header of the first RPC message indicating the first number of the first address information contained in the header of the first RPC message can be set to 0, and the value in the field in the header of the first RPC message indicating the second number of the first address information contained in the data body of the first RPC message can be set to 0, so as to indicate that 0 first address information is carried in the first RPC message.

[0101] If the total data amount of the at least one to-be-processed data is greater than the preset threshold, the second device can write the first address information of each to-be-processed data in the at least one to-be-processed data in the first RPC message to obtain the first RPC message.

[0102] 202, the first device receives the first RPC message sent by the second device.

[0103] After obtaining the first RPC message, the second device can send the first RPC message to the first device. Correspondingly, the first device can receive the first RPC message sent by the second device, and then can process the first RPC message. Illustratively, the first device can obtain the first address information of each to-be-processed data in the at least one to-be-processed data from the first RPC message.

[0104] Illustratively, the first device can include a network communication module, and the function of the network communication module can include receiving and sending RPC messages. Therefore, the network communication module in the first device can receive the first RPC message.

[0105] Exemplarily, the first device can deploy a processing system of the RPC message, the processing system of the RPC message can include RPC software and / or RPC hardware, in detail, the first device can only deploy the RPC software, or the first device can also only deploy the RPC hardware, and the first device can also deploy the RPC software and the RPC hardware simultaneously. The RPC software and the RPC hardware both have the processing capability defined by the RPC protocol, the RPC software is deployed on the operating system of the first device, and the RPC software can provide the first type of RPC service; exemplarily, the RPC software can provide the first type of RPC service by calling the RPC service function provided by the operating system of the first device. Optionally, the RPC software can also be used to process the RPC message of any one of the following types: RPC response (ack), RPC reply (response), or RPC response+RPC reply.

[0106] The RPC hardware is hardware for providing the RPC service, the RPC hardware can include a first processor independent of the second processor of the first device, so that the RPC message can be processed independently of the second processor of the first device; exemplarily, the RPC hardware can be hardware for providing the RPC service in a hardware offload manner, the RPC hardware can be understood as a special hardware group for providing the second type of RPC service, and the RPC hardware can provide the second type of RPC service by calling the RPC service function in the RPC hardware. Optionally, the RPC software can also be used to process the RPC message of any one of the following types: RPC response (ack), RPC reply (response), or RPC response+RPC reply.

[0107] In a case, if the processing system of the RPC message only includes the RPC software or the processing system of the RPC message only includes the RPC hardware, the first type of RPC service and the second type of RPC service can be the same, and the first type of RPC service and the second type of RPC service can both include: data writing, data query, data calculation, or other types of RPC services.

[0108] Exemplarily, in an implementation mode, if the processing system of the RPC message only includes the RPC software, after the network communication module in the first device receives the first RPC message, the first RPC message can be transmitted to the RPC software by the operating system of the first device, and the first address information of each of the at least one to-be-processed data in the first RPC message can be obtained by the RPC software.

[0109] In another implementation, if the processing system of the RPC message only includes the RPC hardware, after the network communication module in the first device receives the first RPC message, the RPC hardware in the first device can directly obtain the first RPC message from the aforementioned network communication module, and the RPC hardware obtains the first address information of each of the at least one to-be-processed data from the first RPC message.

[0110] In another case, if the processing system of the RPC message includes the RPC software and the RPC hardware, for example, implementation 1: there is an intersection but not the same between the first type of RPC service and the second type of RPC service, for example, the first type of RPC service can include: data writing, data query and data calculation, the second type of RPC service can include data writing, and the data writing type of RPC service can be preferentially provided by the RPC hardware, and in the case that the RPC hardware does not meet the execution condition, the data writing type of RPC service can be provided by the RPC software. For example, the "RPC hardware does not meet the execution condition" can be understood as that the idle hardware resources of the RPC hardware are insufficient to support the provision of a new second type of RPC service.

[0111] For example, the execution condition can include at least one of: the number of RPC services being provided by the RPC hardware is less than or equal to a first threshold value; the usage rate of the processor in the RPC hardware is less than or equal to a second threshold value; the usage rate of the storage space in the RPC hardware is less than or equal to a third threshold value; the first queue has a free space, the first queue can be a queue maintained in the RPC hardware, and the first queue can include all RPC services being provided by the RPC hardware; or other types of execution conditions, etc., which can be determined in combination with actual application scenarios.

[0112] Optionally, step 202 can include: if the first RPC message is an RPC message that can be processed by the RPC hardware, and the RPC hardware satisfies the execution condition, it can be determined that the data processing operation corresponding to the first RPC message is executed by the RPC hardware in the subsequent steps, and the RPC hardware obtains the first address information of each of the at least one to-be-processed data from the first RPC message; for example, the RPC message that can be processed by the RPC hardware includes: a second type of RPC request, an RPC response (ack), an RPC response (response), or an RPC response+RPC response RPC message. If the first RPC message is not an RPC message that can be processed by the RPC hardware, or the RPC hardware does not satisfy the execution condition, it can be determined that the data processing operation corresponding to the first RPC message is executed by the RPC software in the subsequent steps, and the RPC hardware can transmit the first RPC message to the RPC software, and the RPC software obtains the first address information of each of the at least one to-be-processed data from the first RPC message; or, the RPC hardware transmits the information extracted from the first RPC message to the RPC software, and the information extracted from the first RPC message includes the first address information of each of the at least one to-be-processed data.

[0113] Implementation 2: The first type of RPC service is different from the second type of RPC service, for example, the first type of RPC service can include: data query and data calculation, and the second type of RPC service can include data writing, and the RPC software and the RPC hardware can provide different types of RPC services respectively. Optionally, step 202 can include: if the first RPC message is an RPC message that can be processed by the RPC hardware, it can be determined that the data processing operation corresponding to the first RPC message is executed by the RPC hardware in the subsequent steps, and the RPC hardware obtains the first address information of each of the at least one to-be-processed data from the first RPC message. If the first RPC message is not an RPC message that can be processed by the RPC hardware, it can be determined that the data processing operation corresponding to the first RPC message is executed by the RPC software in the subsequent steps, and the RPC hardware can transmit the first RPC message to the RPC software, or the RPC hardware transmits the information extracted from the first RPC message to the RPC software.

[0114] Implementation 3: The first type of RPC service and the second type of RPC service can be the same, and the first type of RPC service and the second type of RPC service can include: data writing, data query, data calculation, or other types of RPC services. Optionally, the RPC hardware can be preferred to provide the RPC service, and the RPC software can provide the RPC service in the case that the RPC hardware does not satisfy the execution condition.

[0115] Optionally, step 202 can comprise: if the RPC hardware satisfies the execution condition, it can be determined that the data processing operation corresponding to the first RPC message is executed by the RPC hardware in the subsequent step, and the RPC hardware obtains the first address information of each to-be-processed data in the at least one to-be-processed data from the first RPC message. If the RPC hardware does not satisfy the execution condition, it can be determined that the data processing operation corresponding to the first RPC message is executed by the RPC software in the subsequent step, and the RPC hardware can transmit the first RPC message to the RPC software, or the RPC hardware transmits the information extracted from the first RPC message to the RPC software.

[0116] Optionally, in the case that the processing system of the RPC message comprises the RPC software and the RPC hardware, the RPC software and the RPC hardware intercommunicate, in other words, the RPC software and the RPC hardware can realize mutual communication within the first device. For example, the first device can comprise a storage space shared by the RPC software and the RPC hardware, and the RPC hardware and the RPC software have access authority of the aforementioned storage space, so as to realize intercommunication between the RPC software and the RPC hardware; optionally, the storage space shared by the RPC software and the RPC hardware can be deployed in the interior of the RPC hardware. For another example, the RPC software and the RPC hardware can realize intercommunication through the operating system in the first device, when the RPC hardware needs to transmit a message to the RPC software, the message can be transmitted to the RPC software through the operating system by initiating a hardware interrupt; when the RPC software needs to transmit a message to the RPC hardware, the message can be transmitted to the RPC hardware through the operating system. Alternatively, the RPC software and the RPC hardware can also realize intercommunication in other ways, which can be determined in combination with the actual product form.

[0117] In the embodiment of the application, since the RPC hardware is a dedicated hardware providing RPC service, that is, the RPC service is provided independently from the operating system of the first device, when the RPC service is provided by the RPC hardware, the time consumption caused by the kernel (that is, the kernel in the operating system of the first device) can be avoided, and the processing speed is fast. The RPC software is more convenient to provide complex RPC service, and the RPC software has stronger scalability. The first device simultaneously deploys the RPC software and the RPC hardware, the RPC software and the RPC hardware intercommunicate, and the first device has the synergy of the RPC software and the RPC hardware, which is beneficial to provide more efficient RPC service, support more complex RPC service, and have stronger scalability.

[0118] Optionally, the RPC hardware is located between the RPC software and the network communication module, in other words, the RPC hardware in the first device can directly communicate with the network communication module in the first device, the RPC software can acquire the received RPC message from the network communication module through the RPC hardware, and the RPC software can transmit the RPC message to be sent to the network communication module through the RPC hardware.

[0119] Exemplarily, the functions of the RPC hardware can include: acquiring the received RPC message from the network communication module; transmitting the first RPC request corresponding to the received RPC message to the RPC software; performing the processing operation corresponding to the second RPC request corresponding to the received RPC message; transmitting the RPC message generated by the RPC software to the network communication module; or transmitting the RPC message generated by the RPC hardware to the network communication module. Optionally, the functions of the RPC hardware further include: processing the received RPC response, RPC reply or RPC response+RPC reply, and transmitting the received RPC response, RPC reply or RPC response+RPC reply to the RPC software.

[0120] Exemplarily, a second queue can be maintained between the RPC software and the RPC hardware, and the second queue is used to store the message transmitted between the RPC software and the RPC hardware. In the present application, the second queue can also be referred to as a message queue for transmitting the message. For example, the second queue can be stored in the storage space shared between the RPC software and the RPC hardware, and for another example, the second queue can be maintained by the operating system of the first device. For a more intuitive understanding of the present scheme, please refer to FIG. 4, which is a schematic diagram of the RPC software and the RPC hardware provided by an embodiment of the present application. As shown in FIG. 4, the first device includes the RPC software, the RPC hardware and the network transmission module, the RPC hardware is located between the RPC software and the network transmission module, and a queue can be maintained between the RPC software and the RPC hardware, which is used to store the message transmitted between the RPC software and the RPC hardware. In other words, the RPC software and the RPC hardware realize the intercommunication of the message through the queue. It should be understood that the example in FIG. 4 is only for facilitating the understanding of the present scheme, and is not used to limit the present scheme.

[0121] Further, when the RPC hardware transmits the first RPC request corresponding to the received RPC message to the RPC software, in one case, the RPC hardware in the first device can transmit the received RPC message to the RPC software in the first device, for example, the RPC hardware in the first device can put the RPC message into the second queue to transmit the received RPC message to the RPC software in the first device; in another case, after the RPC hardware in the first device obtains the RPC message from the network transmission module, the RPC hardware can analyze the RPC, extract the information required for executing the first RPC request from the RPC message, and transmit the information required for executing the first RPC request to the RPC software in the first device, for example, the RPC hardware in the first device can put the information required for executing the first RPC request extracted from the RPC message into the second queue to transmit to the RPC software in the first device.

[0122] In the process in which the RPC software transmits the RPC message generated by the RPC software to the network communication module, in one case, the RPC software in the first device can transmit the encapsulated RPC message to the RPC hardware in the first device, and the RPC hardware transmits the RPC message to be sent to the network communication module, for example, the RPC software can put the RPC message to be sent into the second queue to transmit to the RPC hardware. In another case, the RPC software in the first device can also transmit the data carried in the RPC message to the RPC hardware in the first device, and the RPC hardware encapsulates the RPC message based on the data carried in the aforementioned RPC message, and then transmits the RPC message to be sent to the network communication module, for example, the RPC software can put the aforementioned data to be carried in the RPC message into the second queue to transmit to the RPC hardware.

[0123] In the embodiments of the present application, because the RPC hardware reads the RPC message faster, the RPC hardware is deployed between the RPC software and the network transmission module, which facilitates faster understanding of the RPC message; and the RPC hardware can transfer part of the RPC requests (for example, the first RPC request) to the RPC software for execution, and the RPC hardware can process part of the RPC requests (for example, the second RPC request), which refines the specific cooperation mode of the RPC software and the RPC hardware, and is beneficial to provide more efficient RPC services, support more complex RPC services, and have stronger expansibility.

[0124] 203、The first device obtains the to-be-processed data corresponding to the first address information from the second device in a manner of remote direct memory access.

[0125] After the first device obtains the first address information of each of the at least one to-be-processed data from the first RPC message, the first device can obtain each of the to-be-processed data corresponding to the first address information from the second device one by one in a remote direct memory access manner according to the first address information of each of the to-be-processed data. For example, the first address information can be understood as a storage address of the to-be-processed data in a memory, and obtaining the to-be-processed data corresponding to the first address information can be understood as obtaining the to-be-processed data at a storage location in the memory pointed by the first address information.

[0126] For example, the remote direct memory access is a technology for bypassing an operating system of a remote device (for example, the second device) to access data stored in the remote device. Obtaining the to-be-processed data corresponding to the first address information in the remote direct memory access manner can be understood as a hardware offload-based manner, which bypasses the operating system of the first device and uses a special hardware group to perform the operation of obtaining the to-be-processed data corresponding to the first address information from the second device. When the to-be-processed data corresponding to the first address information is obtained in the remote direct memory access manner, the second processor of the first device can not be occupied, so that the operating system of the first device is bypassed, and the time overhead caused by CPU resource scheduling and kernel trapping of the first device is avoided.

[0127] Step 203 can include that the RPC hardware or the RPC software in the first device calls a first module in the first device to implement obtaining the to-be-processed data corresponding to the first address information from the second device in the remote direct memory access manner. For example, the first module can be a remote direct memory access (RDMA) module, and the first module can be an RDMA network card. Optionally, the RDMA network card can be integrated into a network communication module of the first device. Alternatively, the first module can be another hardware group with a remote direct memory access capability, and the specific implementation can be determined according to actual application products.

[0128] For example, if the processing system of the RPC message only includes the RPC software, step 203 can be performed by the RPC software in the first device. If the processing system of the RPC message only includes the RPC hardware, step 203 can be performed by the RPC hardware in the first device. If the processing system of the RPC message includes the RPC software and the RPC hardware, whether the RPC software or the RPC hardware is used to perform the data processing operation corresponding to the first RPC message is determined in step 202, and then step 203 can be performed by the RPC software or the RPC hardware determined in step 202.

[0129] 204. The first device performs a data processing operation based on the to-be-processed data.

[0130] Exemplarily, the type of the application scenario corresponding to the first RPC message is at least one of data writing, data querying or data computing. Further, in the case that the type of the application scenario corresponding to the first RPC message is data writing, each to-be-processed data can be understood as data to be written, and the data processing operation can include: writing each to-be-processed data into the memory of the first device, in other words, performing data disk for each to-be-processed data. In the case that the type of the application scenario corresponding to the first RPC message is data querying, each to-be-processed data can be understood as each queried data, and the data processing operation can include: storing each to-be-processed data. In the case that the type of the application scenario corresponding to the first RPC message is data computing, each to-be-processed data can be understood as data to be used in the computing process, and the data processing operation can include: performing data computing based on each to-be-processed data.

[0131] Exemplarily, if the processing system of the RPC message only includes the RPC software, step 204 can be executed by the RPC software in the first device. If the processing system of the RPC message only includes the RPC hardware, step 204 can be executed by the RPC hardware in the first device. If the processing system of the RPC message includes the RPC software and the RPC hardware, whether the data processing operation is executed by the RPC software or the RPC hardware has been determined in step 202, then step 204 can be executed by the RPC software or the RPC hardware determined in step 202.

[0132] It should be noted that the first device can start to execute the data processing operation after obtaining all to-be-processed data included in the at least one to-be-processed data, that is, can start to execute step 204 after executing step 203 once or multiple times; or the to-be-processed data obtaining operation and the data processing operation can be performed in cross, that is, step 203 and step 204 can be executed in cross, and the specific implementation can be determined according to the actual application scenario.

[0133] In the embodiment of the present application, the RPC message can include address information of the to-be-processed data. After receiving the RPC message, the first device can directly obtain the to-be-processed data corresponding to the address information by using the remote direct memory access mode, and then perform the data processing operation based on the obtained to-be-processed data. Therefore, the RPC message no longer needs to carry complete to-be-processed data. The RPC message only needs to carry short address information to realize the transmission of a large amount of to-be-processed data, greatly reducing the amount of data carried in the RPC message, thereby reducing the transmission delay of the RPC message. If complete to-be-processed data is carried in the RPC message and then transmitted by transmitting the RPC message, the transmission delay of the to-be-processed data is high due to the inherent process other than data transmission, such as CPU resource scheduling and kernel trapping when using the system call provided by the operating system, in the process of obtaining the to-be-processed data. In the present application, the to-be-processed data corresponding to the first address information is directly obtained from the second device by using the remote direct memory access mode, which avoids the time consumption caused by the processes such as CPU resource scheduling and kernel trapping, thereby greatly improving the efficiency of the data transmission of the to-be-processed data and reducing the time delay in the data transmission process.

[0134] Optionally, based on the above-mentioned embodiment corresponding to FIG. 2, please refer to FIG. 5, which is another flowchart of the data processing method provided by the embodiment of the present application. The data processing method provided by the embodiment of the present application can include:

[0135] 501. The second device obtains the first RPC message, wherein the first RPC message includes first address information of each to-be-processed data in at least one to-be-processed data.

[0136] 502. The first device receives the first RPC message sent by the second device.

[0137] 503. The first device obtains the to-be-processed data corresponding to the first address information from the second device by using the remote direct memory access mode.

[0138] 504. The first device performs a data processing operation based on the to-be-processed data.

[0139] The specific implementation modes of steps 501 to 504 and the specific meanings of the terms in steps 501 to 504 can be described in the above-mentioned embodiment corresponding to FIG. 2, which will not be repeated here.

[0140] 505、The first device receives a second RPC message, a type of an RPC request corresponding to the second RPC message is data query, and the second RPC message comprises a logical table and / or an index of each of the at least one to-be-queried data.

[0141] The step 505 is an optional step. The first device can receive the second RPC message sent by the third device, the type of an RPC request corresponding to the second RPC message is data query, in other words, the second RPC message is used to request the first device to provide the RPC service of data query, and the second RPC message is a kind of RPC message, which can be understood in combination with the description of the first RPC message above, and the specific form of the second RPC message and the logical table and / or the index will not be further described here.

[0142] 506、The first device determines second address information of each of the at least one to-be-queried data according to the logical table and / or the index of each of the at least one to-be-queried data.

[0143] The step 506 is an optional step. The second address information of each of the at least one to-be-queried data can be understood as a storage address of each of the at least one to-be-queried data in a memory of the first device.

[0144] 507、The first device sends a third RPC message corresponding to the second RPC message to the third device, the third RPC message comprising the second address information of each of the at least one to-be-queried data, and the second address information is used for the third device to obtain data corresponding to the second address information from the first device in a manner of remote direct memory access.

[0145] The step 507 is an optional step. After the first device determines the second address information of each of the at least one to-be-queried data, the first device can write at least one second address information corresponding to the at least one to-be-queried data into the third RPC message, the third RPC message comprising the second address information of each of the at least one to-be-queried data, and then send the third RPC message corresponding to the second RPC message to the third device. Correspondingly, the third device can receive the second RPC message, obtain the second address information of each of the at least one to-be-queried data from the second RPC message, and obtain data corresponding to the second address information from the first device in a manner of remote direct memory access; for example, the second address information can be understood as a storage address of the to-be-queried data in a memory of the first device, and obtaining data corresponding to the second address information can be understood as obtaining data at a storage location pointed by the second address information in the memory.

[0146] Exemplarily, the third device can invoke the second module in the first device to implement obtaining the data corresponding to the second address information from the first device in a manner of remote direct memory access, the second module is a hardware group with a remote direct memory access capability, and a specific form of the second module can refer to the description of the first module.

[0147] Exemplarily, the third RPC message can be an RPC reply message, and a specific implementation manner of "writing the at least one second address information corresponding to the at least one to-be-queried data into the third RPC message" can refer to the description in the corresponding embodiment of FIG. 2, and the difference lies in that the first address information in the corresponding embodiment of FIG. 2 is replaced by the second address information, and the first PRC message in the corresponding embodiment of FIG. 2 is replaced by the third PRC message, and no repeated description is made herein.

[0148] In the embodiment of the application, after receiving the RPC request of data query, the first device can also not carry the complete queried data in the RPC reply message (that is, the third RPC message), but carry the second address information of the queried data, and then the third device obtains the data corresponding to the second address information from the first device in a manner of remote direct memory access, so that the third device avoids time overheads caused by CPU resource scheduling and processes such as falling into the kernel in the process of obtaining the data to be queried, and is beneficial to quickly obtaining the data to be queried.

[0149] Alternatively, the first device can also determine each to-be-queried data according to a logical table and / or an index of each to-be-queried data, and then write each to-be-queried data into the RPC reply message and send the RPC reply message to the third device, so that the third device can directly obtain the data to be queried from the RPC reply message.

[0150] Optionally, on the basis of the above description, the specific implementation manners of the three application scenarios of data landing, data query and data calculation are described as follows in combination with specific examples.

[0151] I. Data Landing

[0152] Optionally, when the data processing operation corresponding to the first RPC message received by the first device is to write the at least one to-be-processed data into the memory of the first device, the receiving of the first RPC message, the obtaining of the to-be-processed data, and the data processing operation are all performed by the RPC hardware in the first device. Referring to FIG. 6, FIG. 6 is a flow diagram of data writing provided by an embodiment of the present application. The initiator of the data writing RPC request includes to-be-written data (which is an example of to-be-processed data), RPC software, and a network transmission module. The receiver of the data writing RPC request (which is an example of the first device) includes a network transmission module, RPC software, and RPC hardware, and the foregoing RPC software and RPC hardware can both call a data writing function; the RPC hardware is located between the RPC software and the network transmission module, and the foregoing network transmission module can be integrated with an RDMA network card; for example, the RPC hardware internally includes a shared storage space between the RPC software and the RPC hardware, which is used to maintain a queue when the RPC software and the RPC hardware perform message transmission. In this embodiment, the value of the Function field corresponding to the data writing is taken as 1, that is, func 1 represents the data writing RPC request.

[0153] The specific implementation steps can be as follows:

[0154] The RPC software of the initiator determines the first address information and the length of each to-be-written data in the two to-be-written data, for example, the address (Addr) is 0x0100000, the length (Size) is 1 MB, the address is 0x0400000, and the length is 2 MB in FIG. 6.

[0155] The RPC software of the initiator encapsulates an RPC message, for example, encapsulates into the RPC request (req) message in FIG. 6, the func in the RPC request message is 1, the number of first address information (Hdr DMACNT) in the header of the RPC request message is 2, the first address information is the storage address of the to-be-written data in the memory of the initiator obtained in a remote direct memory access manner, the information of the first to-be-written data in the RPC request message is the address (DMA Addr) 0x0100000 and the length (DMA Size) 1 MB, and the information of the second to-be-written data in the RPC request message is the address (DMA Addr) 0x0400000 and the length (DMA Size) 2 MB.

[0156] The network transmission module of the initiator sends the RPC request message to the receiver.

[0157] The network transmission module of the receiver receives the RPC request message.

[0158] The RPC hardware of the receiving end reads the RPC request message from the network transmission module of the receiving end, reads the header of the RPC request message, determines that the data landing service requested by the RPC request message is the RPC service supported by the RPC hardware, and obtains the address (Addr) 0x0100000, length (Size) 1MB and address 0x0400000, length 2MB in the RPC request message. The RPC hardware can call the network transmission module to perform the RDMA operation to the initiating end according to the first address information of each to-be-written data. The memory address of the receiving end is the landing address of each to-be-written data, that is, the landing address of each to-be-written data is the memory address of the receiving end. The receiving end can acquire each to-be-written data in the RDMA mode one by one, perform the data landing function, and write each to-be-written data into the memory address of the receiving end, so as to realize the data landing of the two to-be-written data.

[0159] It should be understood that the example in FIG. 6 is only for facilitating understanding of the present scheme and does not limit the present scheme.

[0160] In the embodiment of the present application, when the data processing operation corresponding to the first RPC message is data writing, the first RPC message is received by the RPC hardware, the to-be-processed data corresponding to the first address information is acquired from the second device in the remote direct memory access mode, and the to-be-processed data is written into the memory of the first device. Since data writing is a logically simple RPC service, it is easy to implement the foregoing steps by using the RPC hardware to execute the foregoing steps, and using the RPC hardware to execute the foregoing steps is advantageous to avoiding the operating system of the first device in the entire process of data writing, so as to avoid the time overhead caused by falling into the kernel, and further improve the efficiency of the data writing process.

[0161] II. Data query

[0162] Optionally, when the type of the RPC request corresponding to the second RPC message received by the first device is a data query, the second RPC message is obtained by the RPC hardware in the first device from the network communication module of the first device, the operation of determining the second address information of each data to be queried is performed by the RPC software in the first device, and the third RPC message corresponding to the second RPC message is sent by the RPC hardware in the first device through the network communication module of the first device. Referring to FIG. 7, FIG. 7 is a flowchart of a data query provided in an embodiment of the present application. The initiator of the RPC request of the data query includes the RPC software and the network transmission module. The receiver of the RPC request of the data query (that is, an example of the first device) includes the network transmission module, the RPC software and the RPC hardware. The RPC software can call the data query function, and the RPC hardware cannot call the data query function. The specific structure between the RPC software and the RPC hardware is the same as that in FIG. 6, which will not be described again. In this embodiment, the value of the Function field corresponding to the data query is taken as 2, that is, func 2 represents the RPC request of the data query.

[0163] The specific implementation steps can be as follows:

[0164] The RPC software of the initiator determines that the logical table and the index of the three data to be queried (that is, an example of the data to be processed) are logical table Table 1 and index Key 1, logical table Table 2 and index Key 2, and logical table Table 3 and index Key 3.

[0165] The RPC software of the initiator encapsulates the RPC message, for example, encapsulates the RPC request (req) message in FIG. 7. In the RPC request message, func is 2, the number of the first address information in the header of the RPC request message (Hdr DMACNT) is 0, and the content in the data body of the RPC request message is logical table Table 1 and index Key 1, logical table Table 2 and index Key 2, and logical table Table 3 and index Key 3.

[0166] The network transmission module of the initiator sends the RPC request message to the receiver.

[0167] The network transmission module of the receiver receives the RPC request message.

[0168] The RPC hardware of the receiving end reads the RPC request message from the network transmission module of the receiving end, reads the header of the RPC request message, determines that the data query service requested by the RPC request message is not the RPC service supported by the RPC hardware, and finds that the number of the first address information in the RPC request message is 0, and the RPC hardware puts the aforementioned RPC request message into the queue between the RPC software and the RPC hardware.

[0169] The RPC software of the receiving end reads the RPC request message from the aforementioned queue, executes the data query function according to the logical table Table 1 and the index Key 1, the logical table Table 2 and the index Key 2, and the logical table Table 3 and the index Key 3, and determines the second address information corresponding to each of the three to-be-queried data, wherein the three second address information refers to the storage address of the data pointed to by the logical table Table 1 and the index Key 1, the logical table Table 2 and the index Key 2, and the logical table Table 3 and the index Key 3 in the storage of the receiving end.

[0170] The RPC software of the receiving end encapsulates the RPC reply message based on the second address information corresponding to each of the three to-be-queried data, wherein the RPC reply message includes the second address information corresponding to each of the three to-be-queried data, and the RPC software of the receiving end puts the RPC reply message into the queue between the RPC software and the RPC hardware, so as to transmit the RPC reply message to the RPC hardware of the receiving end.

[0171] The RPC hardware of the receiving end calls the network transmission module to send the RPC reply message to the initiating end.

[0172] The RPC software of the initiating end obtains the RPC reply message from the network transmission module, and obtains the data corresponding to the second address information from the receiving end in a manner of remote direct memory access, so as to realize data query.

[0173] It should be understood that the example in FIG. 7 is only for facilitating understanding of the scheme, and is not used to limit the scheme.

[0174] In the embodiment of the application, the second RPC message corresponding to the data query RPC request is obtained from the network communication module by the RPC hardware, is transferred to the RPC software to execute data query to obtain the second address information of the to-be-queried data, and then the third RPC message corresponding to the second RPC message is transmitted to the network communication module by the RPC hardware, so as to realize sending out of the third RPC message, which not only utilizes the ability of the RPC hardware to quickly read the RPC message, but also quickly determines the second address information of each to-be-queried data by the RPC software, and is beneficial to the third device to obtain the data to be queried more quickly.

[0175] III. Data computation

[0176] Optionally, when the data processing operation is a computation operation based on at least one to-be-processed data, the obtaining operation of the to-be-processed data is performed by the RPC hardware, and the data computation operation based on the to-be-processed data is performed by the RPC software. Referring to FIG. 8, FIG. 8 is a flow diagram of data computation provided by an embodiment of the present application. The initiator of the RPC request of data computation includes to-be-computed data (which is an example of to-be-processed data), RPC software, and a network transmission module. The receiver (which is an example of the first device) of the RPC request of data computation includes a network transmission module, RPC software, and RPC hardware. The RPC software can call a data computation function, and the RPC hardware cannot call the data computation function. The specific structure between the RPC software and the RPC hardware is consistent with the specific structure in FIG. 6, which is not described again. In this embodiment, the value of the Function field corresponding to data computation is taken as 3, that is, func 3 represents the RPC request of data computation.

[0177] The specific implementation steps can be as follows:

[0178] The RPC software of the initiator determines the first address information and the length of each to-be-computed data in the two to-be-computed data, for example, the address (Addr) is 0x0100000, the length (Size) is 1 MB, the address is 0x0400000, and the length is 2 MB in FIG. 8.

[0179] The RPC software of the initiator encapsulates an RPC message, for example, encapsulates into the RPC request (req) message in FIG. 8. In the RPC request message, func is 3, the number of first address information (Hdr DMACNT) in the header of the RPC request message is 2, the first address information is the storage address of the to-be-written data in the storage of the initiator obtained by using the remote direct memory access mode, the information of the first to-be-written data in the RPC request message is the address (DMA Addr) 0x0100000 and the length (DMA Size) 1 MB, and the information of the second to-be-written data in the RPC request message is the address (DMA Addr) 0x0400000 and the length (DMA Size) 2 MB.

[0180] The network transmission module of the initiator sends the RPC request message to the receiver.

[0181] The network transmission module of the receiver receives the RPC request message.

[0182] The RPC hardware of the receiving end reads the RPC request message from the network transmission module of the receiving end, reads the header of the RPC request message, determines that the data computing service requested by the RPC request message is not the RPC service supported by the RPC hardware, and obtains the address (Addr) : 0x0100000, length (Size) : 1MB and address: 0x0400000, length: 2MB in the RPC request message. The RPC hardware of the receiving end can obtain each to-be-computed data in turn based on the address (Addr) : 0x0100000, length (Size) : 1MB and address: 0x0400000, length: 2MB in the RDMA mode.

[0183] Optionally, if each to-be-computed data in the two to-be-computed data can be independently computed, the RPC hardware of the receiving end can put the RPC request message and one to-be-computed data into the queue between the RPC software and the RPC hardware after obtaining one to-be-computed data, to submit to the RPC software of the receiving end. The RPC software of the receiving end reads the RPC request message and one to-be-computed data from the above queue, and executes the data computing function based on the one to-be-computed data to obtain one computing result. Meanwhile, the RPC hardware of the receiving end can continue to obtain another to-be-computed data in the RDMA mode, and put the another to-be-computed data into the queue between the RPC software and the RPC hardware, to submit to the RPC software of the receiving end. The RPC software of the receiving end reads the another to-be-computed data from the above queue, and executes the data computing function based on the one to-be-computed data to obtain another computing result.

[0184] Alternatively, the RPC hardware of the receiving end can put the RPC request message and two to-be-computed data into the queue between the RPC software and the RPC hardware after obtaining the two to-be-computed data, and the RPC software of the receiving end reads the RPC request message and the two to-be-computed data from the above queue, and executes the data computing function based on the two to-be-computed data to obtain the computing result.

[0185] The RPC software of the receiving end encapsulates the RPC reply message based on the second address information corresponding to each to-be-queried data in the three to-be-queried data, and the RPC reply message includes the computing result generated by the RPC software. The RPC software of the receiving end puts the RPC reply message into the queue between the RPC software and the RPC hardware, to transmit the RPC reply message to the RPC hardware of the receiving end.

[0186] The RPC hardware of the receiving end calls the network transmission module to send the RPC reply message to the initiating end.

[0187] It should be understood that the examples in FIG. 8 are only for the convenience of understanding the scheme, and are not used to limit the scheme.

[0188] In the embodiments of the present application, the data processing operation is a calculation operation based on the at least one to-be-processed data, the obtaining operation of the to-be-processed data is performed by the RPC hardware, and the calculation operation is performed by the RPC software, so that the to-be-processed data can be quickly obtained by using the RPC hardware, and complex data calculation can be implemented by using the RPC software, which is beneficial to providing an RPC service with better performance.

[0189] On the basis of the embodiments corresponding to FIG. 1 to FIG. 8, in order to better implement the above scheme of the embodiments of the present application, the following further provides a related device for implementing the above scheme. Please refer to FIG. 9, which is a structural schematic diagram of a device provided by the embodiments of the present application. Exemplarily, the device can be the first device in each of the embodiments corresponding to FIG. 1 to FIG. 8. The device 900 comprises an RPC message processing system 901, which is configured to: receive a first remote procedure call (RPC) message sent by a second device, wherein the first RPC message comprises first address information of each to-be-processed data in at least one to-be-processed data; obtain the to-be-processed data corresponding to the first address information from the second device in a manner of remote direct memory access (RDMA); and perform a data processing operation based on the to-be-processed data.

[0190] Optionally, the first address information is contained in a header of the first RPC message and / or a data body of the first RPC message.

[0191] Optionally, in the case that the first address information is contained in the header of the first RPC message, the header of the first RPC message further comprises first information, and the first information indicates a position of the first address information in the header of the first RPC message; and / or, in the case that the first address information is contained in the data body of the first RPC message, the header of the first RPC message further comprises second information, and the second information indicates a position of the first address information in the data body of the first RPC message.

[0192] Optionally, the header of the first RPC message further comprises at least one of the following information: a quantity of the first address information contained in the header of the first RPC message, a quantity of the first address information contained in the data body of the first RPC message, a quantity of at least one first address information corresponding to the at least one to-be-processed data, or a data quantity of each to-be-processed data.

[0193] Optionally, the RPC message processing system comprises an RPC software and an RPC hardware, wherein the RPC software and the RPC hardware both have processing capabilities defined by an RPC protocol, the RPC software is deployed on an operating system of the device, the RPC hardware is hardware for providing an RPC service, and the RPC software and the RPC hardware intercommunicate.

[0194] Optionally, the device 900 further includes a network communication module 902, functions of the network communication module 902 include receiving and sending RPC messages, functions of the RPC hardware include: obtaining the received RPC messages from the network communication module, transmitting the first RPC request corresponding to the received RPC message to the RPC software, performing the processing operation corresponding to the second RPC request corresponding to the received RPC message, transmitting the RPC message generated by the RPC software to the network communication module, and transmitting the RPC message generated by the RPC hardware to the network communication module.

[0195] Optionally, the data processing operation is writing the at least one to-be-processed data into the memory of the first device, and the method is performed by the RPC hardware in the first device.

[0196] Optionally, the processing system 901 of the RPC message is further configured to: receive a second RPC message, a type of an RPC request corresponding to the second RPC message is a data query, and the second RPC message includes a logical table and / or an index of each to-be-queried data in the at least one to-be-queried data; the first device determines second address information of each to-be-queried data according to the logical table and / or the index of each to-be-queried data; and the first device sends a third RPC message corresponding to the second RPC message to the third device, the third RPC message including the second address information of each to-be-queried data, and the second address information is used for the third device to obtain data corresponding to the second address information from the first device in a manner of remote direct memory access.

[0197] Optionally, the second RPC message is obtained by the RPC hardware from the network communication module of the first device, the determination of the second address information of each to-be-queried data is performed by the RPC software, and the third RPC message is sent by the RPC hardware through the network communication module.

[0198] Optionally, the data processing operation is a calculation operation based on the at least one to-be-processed data, the obtaining of the to-be-processed data is performed by the RPC hardware, and the calculation operation is performed by the RPC software.

[0199] Optionally, the first device is a storage node, and the second device is a computing node; or the first device is a computing node, and the second device is a storage node.

[0200] It should be noted that the information interaction and execution process between the modules / units in the device 900 are based on the same concept as the various method embodiments corresponding to FIGS. 1 to 8 of the present application, and the specific content can be referred to the description of the method embodiments in the foregoing description of the present application, which will not be repeated here.

[0201] The embodiment of the present application further provides another device, which can be the second device in each of the embodiments corresponding to FIG. 1 to FIG. 8. The device comprises a processing system of an RPC message, and the processing system of the RPC message is configured to: acquire a first RPC message, wherein the first RPC message comprises first address information of each of at least one to-be-processed data; and send the first RPC message to a first device, and the first address information is used for the first device to acquire the to-be-processed data corresponding to the first address information in a manner of remote direct memory access.

[0202] Optionally, the first address information is contained in a header of the first RPC message and / or a data body of the first RPC message.

[0203] Optionally, in the case that the first address information is contained in the header of the first RPC message, the header of the first RPC message further comprises first information, and the first information indicates a position of the first address information in the header of the first RPC message; and / or, in the case that the first address information is contained in the data body of the first RPC message, the header of the first RPC message further comprises second information, and the second information indicates a position of the first address information in the data body of the first RPC message.

[0204] Optionally, the header of the first RPC message further comprises at least one of the following information: a quantity of the first address information contained in the header of the first RPC message, a quantity of the first address information contained in the data body of the first RPC message, a quantity of at least one first address information corresponding to the at least one to-be-processed data, or a data quantity of each to-be-processed data.

[0205] It should be noted that the information interaction and execution process between the modules / units in the device are based on the same concept as the method embodiments corresponding to FIG. 1 to FIG. 8, and the specific content can be referred to the description of the method embodiments in the foregoing method embodiments.

[0206] The embodiment of the present application further provides a data processing apparatus, which is arranged in a first device, and the apparatus comprises: a receiving module configured to receive a first remote procedure call (RPC) message sent by a second device, wherein the first RPC message comprises first address information of each of at least one to-be-processed data; an acquiring module configured to acquire the to-be-processed data corresponding to the first address information from the second device in a manner of remote direct memory access; and a processing module configured to perform a data processing operation based on the to-be-processed data.

[0207] Optionally, the first address information is contained in a header of the first RPC message and / or a data body of the first RPC message.

[0208] Optionally, in the case that the first address information is contained in the header of the first RPC message, the header of the first RPC message further comprises first information indicating the position of the first address information in the header of the first RPC message; and / or, in the case that the first address information is contained in the data body of the first RPC message, the header of the first RPC message further comprises second information indicating the position of the first address information in the data body of the first RPC message.

[0209] Optionally, the header of the first RPC message further comprises at least one of the following information: the number of the first address information contained in the header of the first RPC message, the number of the first address information contained in the data body of the first RPC message, the number of the at least one first address information corresponding to the at least one to-be-processed data, or the data amount of each to-be-processed data.

[0210] Optionally, the first device comprises an RPC software and an RPC hardware, wherein the RPC software and the RPC hardware both have the processing capability defined by the RPC protocol, the RPC software is deployed on an operating system of the first device, the RPC hardware is hardware for providing the RPC service, the RPC software and the RPC hardware intercommunicate, and the data processing apparatus is deployed in the RPC software and the RPC hardware.

[0211] Optionally, the first device further comprises a network communication module, and the functions of the network communication module include receiving and sending the RPC message, and the functions of the RPC hardware include: obtaining the received RPC message from the network communication module, transmitting the first RPC request corresponding to the received RPC message to the RPC software, executing the processing operation corresponding to the second RPC request corresponding to the received RPC message, transmitting the RPC message generated by the RPC software to the network communication module, and transmitting the RPC message generated by the RPC hardware to the network communication module.

[0212] Optionally, the data processing operation is to write the at least one to-be-processed data into the memory of the first device, and the method is executed by the RPC hardware in the first device.

[0213] Optionally, the receiving module is further configured to receive a second RPC message, the type of the RPC request corresponding to the second RPC message is data query, and the second RPC message comprises a logical table and / or an index of each to-be-queried data in at least one to-be-queried data; and the data processing apparatus further comprises: a determining module configured to determine second address information of each to-be-queried data according to the logical table and / or the index of each to-be-queried data; and a sending module configured to send a third RPC message corresponding to the second RPC message to a third device, wherein the third RPC message comprises the second address information of each to-be-queried data, and the second address information is used for the third device to obtain data corresponding to the second address information from the first device in a manner of remote direct memory access.

[0214] Optionally, the second RPC message is acquired by the RPC hardware from the network communication module of the first device, the determining operation of the second address information of each to-be-inquired data is performed by the RPC software, and the third RPC message is sent by the RPC hardware through the network communication module.

[0215] Optionally, the data processing operation is a calculation operation based on the at least one to-be-processed data, and the acquiring operation of the to-be-processed data is performed by the RPC hardware, and the calculation operation is performed by the RPC software.

[0216] Optionally, the first device is a storage node, and the second device is a computing node; or the first device is a computing node, and the second device is a storage node.

[0217] It should be noted that the information interaction, execution process and the like between the modules / units in the data processing apparatus described above are based on the same concept as the various method embodiments corresponding to FIGS. 1 to 8 of the present application, and the specific content can be referred to the description of the method embodiments described above, which will not be repeated here.

[0218] The embodiment of the present application further provides another data processing apparatus, which is deployed in the second device, and the apparatus comprises: an acquiring module configured to acquire a first RPC message, wherein the first RPC message comprises first address information of each to-be-processed data in at least one to-be-processed data; and a sending module configured to send the first RPC message to the first device, and the first address information is used for the first device to acquire the to-be-processed data corresponding to the first address information in a manner of remote direct memory access.

[0219] Optionally, the first address information is contained in a header of the first RPC message and / or a data body of the first RPC message.

[0220] Optionally, in the case that the first address information is contained in the header of the first RPC message, the header of the first RPC message further comprises first information, and the first information indicates a position of the first address information in the header of the first RPC message; and / or in the case that the first address information is contained in the data body of the first RPC message, the header of the first RPC message further comprises second information, and the second information indicates a position of the first address information in the data body of the first RPC message.

[0221] It should be noted that the information interaction, execution process and the like between the modules / units in the data processing apparatus described above are based on the same concept as the various method embodiments corresponding to FIGS. 1 to 8 of the present application, and the specific content can be referred to the description of the method embodiments described above, which will not be repeated here.

[0222] Referring to FIG. 10, FIG. 10 is a structural schematic diagram of a computing device provided by an embodiment of the present application. As shown in FIG. 10, the computing device 1000 (which can be used to present the first device or the second device described above) includes a processor 1001, a memory 1002, a communication interface 1003, and a bus 1004, wherein the processor 1001, the memory 1002, and the communication interface 1003 are coupled through the bus (not shown in the figure). The memory 1002 stores instructions, and when the instructions stored in the memory 1002 are executed, the computing device 1000 performs the method performed by the first device or the second device in the method embodiments described above.

[0223] The computing device 1000 can be one or more integrated circuits configured to implement the above method, for example, one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms. For another example, when the units in the apparatus can be implemented in the form of a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can invoke programs. For another example, these units can be integrated together to implement a system-on-a-chip (SOC).

[0224] The processor 1001 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor can be a microprocessor, or any conventional processor.

[0225] The memory 1002 can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0226] The executable program code stored in the memory 1002, in one implementation, if the execution subject of the above data processing method is the RPC software in the computing device 1000, the processor 1001 executes the executable program code to realize the functions of the aforementioned receiving module, obtaining module, processing module and other modules, thereby realizing the above data processing method. That is, the memory 1002 stores instructions for executing the above data processing method. In another implementation, if the execution subject of the above data processing method is the RPC software and the RPC hardware in the computing device 1000, the computing device 1000 can also include the RPC hardware, and the RPC software and the RPC hardware intercommunicate, the processor 1001 executes the executable program code to realize the functions of the RPC software in the above method embodiments, that is, the memory 1002 stores instructions for executing the steps of the RPC software in the above data processing method.

[0227] The communication interface 1003 can be a network communication module in the above method embodiments, using a transceiving module such as but not limited to a network interface card, a transceiver, etc., to realize the communication between the computing device 1000 and other devices or communication networks.

[0228] Bus 1004 can include a data bus, a power bus, a control bus, and a state signal bus, among others. The bus can be a peripheral component interconnect express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0229] Referring to FIG. 11, FIG. 11 is a structural schematic diagram of a computing device cluster provided by an embodiment of the present application. As shown in FIG. 11, the computing device cluster 1100 includes at least one computing device 1000, and the at least one computing device 1000 includes a first device and a second device, wherein the first device is configured to perform the steps performed by the first device in the method embodiments, and the second device is configured to perform the steps performed by the second device in the method embodiments.

[0230] As shown in FIG. 11, the computing device cluster 1100 includes at least one computing device 1000. The memory 1002 in one or more computing devices 1000 in the computing device cluster 1100 can store the same instructions for implementing the steps performed by the first device described above; or the memory 1002 in one or more computing devices 1000 in the computing device cluster 1100 can store the same instructions for implementing the steps performed by the second device described above.

[0231] In some possible implementations, the memory 1002 in one or more computing devices 1000 in the computing device cluster 1100 can also respectively store part of the instructions for implementing the steps performed by the first device (or the second device) described above. In other words, the combination of one or more computing devices 1000 can collectively perform the instructions for implementing the steps performed by the first device (or the second device) described above.

[0232] It should be noted that the memory 1002 in different computing devices 1000 in the computing device cluster 1100 can store different instructions, respectively, for performing part of the functions of the first device (or the second device) described above. That is, the instructions stored in the memory 1002 in different computing devices 1000 can implement the functions of the first device (or the second device).

[0233] In some possible implementation manners, one or more of the computing devices 1000 in the computing device cluster 1100 can be connected through a network. The network can be a wide area network or a local area network, etc.

[0234] Referring to FIG. 12, FIG. 12 is a schematic diagram of connection of the computing devices in the computer cluster through the network according to an embodiment of the present application. As shown in FIG. 12, the two computing devices 1000A and 1000B are connected through the network. Specifically, the computing devices are connected to the network through the communication interfaces in the computing devices.

[0235] In a possible implementation manner, the execution subjects of the data processing method are all RPC software, and the memory in the computing device 1000A stores instructions for implementing the functions of the receiving module, the obtaining module, and the like. Meanwhile, the memory in the computing device 1000B stores instructions for implementing the functions of the processing module and the like.

[0236] It should be understood that the functions of the computing device 1000A shown in FIG. 12 can also be completed by multiple computing devices. Similarly, the functions of the computing device 1000B can also be completed by multiple computing devices.

[0237] In another possible implementation manner, the execution subjects of the data processing method are all RPC software and RPC hardware, and the computing device 1000A is deployed with the RPC hardware for implementing the instructions of the functions of the receiving module, the obtaining module, and the like. Meanwhile, the memory in the computing device 1000B stores instructions for implementing the functions of the processing module and the like.

[0238] The embodiment of the present application also relates to a computer storage medium, which stores a program for performing signal processing, and when the program is run on a computer, the computer is caused to perform the steps performed by the first device or the second device in the embodiment shown in FIG. 2.

[0239] The embodiment of the present application also relates to a computer program product, which stores instructions, and the instructions cause a computer to perform the steps performed by the first device or the second device in the embodiment shown in FIG. 2 when the instructions are executed by the computer.

[0240] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the system, the device and the unit described above can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein.

[0241] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. The division of the units is merely a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0242] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0243] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of software functional units.

[0244] When the integrated unit is implemented in the form of software functional units and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the present application essentially or partly, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), magnetic disk or optical disk, and various other media that can store program codes.

Claims

1. A data processing method, characterized by, The method comprises: The first device receives a first remote procedure call (RPC) message sent by a second device, wherein the first RPC message comprises first address information of each of at least one to-be-processed data; The first device obtains the to-be-processed data corresponding to the first address information from the second device in a manner of remote direct memory access (RDMA); The first device performs a data processing operation based on the to-be-processed data.

2. The method of claim 1, wherein, The first address information is contained in a header of the first RPC message and / or a data body of the first RPC message.

3. The method of claim 2, wherein, In the case that the first address information is contained in the header of the first RPC message, the header of the first RPC message further comprises first information indicating a position of the first address information in the header of the first RPC message; and / or, In the case that the first address information is contained in the data body of the first RPC message, the header of the first RPC message further comprises second information indicating a position of the first address information in the data body of the first RPC message.

4. The method of claim 3, wherein, The header of the first RPC message further comprises at least one of the following information: a quantity of the first address information contained in the header of the first RPC message, a quantity of the first address information contained in the data body of the first RPC message, a quantity of at least one first address information corresponding to the at least one to-be-processed data, or a data quantity of each of the to-be-processed data.

5. The method according to any one of claims 1 to 4, characterized in that, The first device comprises RPC software and RPC hardware, wherein the RPC software and the RPC hardware both have processing capabilities defined by an RPC protocol, the RPC software is deployed on an operating system of the first device, the RPC hardware is hardware providing an RPC service, and the RPC software and the RPC hardware intercommunicate.

6. The method of claim 5, wherein, The first device further comprises a network communication module, and functions of the network communication module include receiving and sending RPC messages; functions of the RPC hardware include: obtaining a received RPC message from the network communication module, transmitting a first RPC request corresponding to the received RPC message to the RPC software, performing a processing operation corresponding to a second RPC request corresponding to the received RPC message, transmitting an RPC message generated by the RPC software to the network communication module, and transmitting an RPC message generated by the RPC hardware to the network communication module.

7. The method of claim 5, wherein, The data processing operation corresponding to the first RPC message is writing the at least one to-be-processed data into a memory of the first device, and the method is performed by the RPC hardware in the first device.

8. The method of claim 5, wherein, The method further comprises: The first device receives a second RPC message, and a type of an RPC request corresponding to the second RPC message is data query; the second RPC message comprises a logical table and / or an index of each of at least one to-be-queried data; The first device determines second address information of each of the to-be-queried data according to the logical table and / or the index of each of the to-be-queried data; The first device sends a third RPC message corresponding to the second RPC message to a third device, the third RPC message including second address information of each of the to-be-queried data, the second address information being used for the third device to obtain data corresponding to the second address information from the first device in a manner of remote direct memory access.

9. The method of claim 8, wherein, The second RPC message is obtained by the RPC hardware from a network communication module of the first device, the second address information of each of the to-be-queried data is determined by the RPC software, and the third RPC message is sent by the RPC hardware through the network communication module.

10. The method of claim 5, wherein, The data processing operation corresponding to the first RPC message is a calculation operation based on the at least one to-be-processed data, the to-be-processed data is obtained by the RPC hardware, and the calculation operation is performed by the RPC software.

11. The method according to any one of claims 1 to 4, characterized in that, The first device is a storage node, and the second device is a calculation node; or the first device is a calculation node, and the second device is a storage node.

12. A data processing method, characterized by, The method comprises: The second device obtains a first RPC message, wherein the first RPC message includes first address information of each of at least one to-be-processed data; The second device sends the first RPC message to a first device, and the first address information is used for the first device to obtain the to-be-processed data corresponding to the first address information in a manner of remote direct memory access.

13. The method of claim 12, wherein, The first address information is contained in a header of the first RPC message and / or a data body of the first RPC message.

14. The method of claim 13, wherein, In a case where the first address information is contained in the header of the first RPC message, the header of the first RPC message further includes first information, and the first information indicates a position of the first address information in the header of the first RPC message; and / or In a case where the first address information is contained in the data body of the first RPC message, the header of the first RPC message further includes second information, and the second information indicates a position of the first address information in the data body of the first RPC message.

15. An apparatus, comprising: The device comprises a remote procedure call (RPC) message processing system, and the RPC message processing system is used for: receiving a first RPC message sent by a second device, wherein the first RPC message includes first address information of each of at least one to-be-processed data; obtaining the to-be-processed data corresponding to the first address information from the second device in a manner of remote direct memory access; performing a data processing operation based on the to-be-processed data.

16. The apparatus of claim 15, wherein, The first address information is contained in a header of the first RPC message and / or a data body of the first RPC message.

17. The apparatus of claim 15 or 16, wherein, The processing system of the RPC message comprises RPC software and RPC hardware, wherein the RPC software and the RPC hardware both have processing capabilities defined by an RPC protocol, the RPC software is deployed on an operating system of the device, the RPC hardware is hardware providing an RPC service, and the RPC software and the RPC hardware intercommunicate.

18. The apparatus of claim 17, wherein, The device further comprises a network communication module, and functions of the network communication module include receiving and sending RPC messages; functions of the RPC hardware include: obtaining the received RPC messages from the network communication module, transmitting a first RPC request corresponding to the received RPC message to the RPC software, executing a processing operation corresponding to a second RPC request corresponding to the received RPC message, transmitting an RPC message generated by the RPC software to the network communication module, and transmitting an RPC message generated by the RPC hardware to the network communication module.

19. An apparatus, comprising: The device comprises a processing system of a remote procedure call RPC message, and the processing system of the RPC message is used for: obtaining a first RPC message, wherein the first RPC message comprises first address information of each of at least one to-be-processed data; sending the first RPC message to a first device, and the first address information is used for the first device to obtain the to-be-processed data corresponding to the first address information in a manner of remote direct memory access.

20. A cluster of computing devices, characterized in that, The computing device cluster comprises a first device and a second device, the first device executes steps executed by the first device in the method according to any one of claims 1 to 14, and the second device executes steps executed by the second device in the method according to any one of claims 1 to 14.

21. A computer storage medium, comprising, The computer storage medium stores one or more instructions, and the instructions, when executed by one or more computers, cause the one or more computers to execute the method according to any one of claims 1 to 14.

22. A computer program product, characterised in that, The computer program product stores instructions, and the instructions, when executed by a computer, cause the computer to execute the method according to any one of claims 1 to 14.

Images