Remote data processing method, apparatus, and computing device

Abstract

Embodiments of the present application relate to the technical field of computers, and provide a remote data processing method, an apparatus, and a computing device, capable of improving the efficiency of remote data processing. The method is applied to a first device, and the first device remotely communicates with a second device. The method comprises: receiving an operation request for data to be processed sent by the second device, the operation request comprising: a read request or a write request; in response to the operation request, when a preset condition is satisfied, performing, on the data to be processed, an operation indicated by the operation request, the preset condition being used for indicating that the data to be processed is in a cache coherence state; and in response to the operation request, when the preset condition is not satisfied, performing an invalidation operation on the data to be processed, the invalidation operation being used for invalidating the data to be processed in a replica device, the replica device being a device that caches the data to be processed.

Description

A method, apparatus and computing device for remote data processing

[0001] This application claims priority to Chinese Patent Application No. 202411540437.4, filed with the State Intellectual Property Office of China on October 30, 2024, entitled “A method, apparatus and computing device for remote data processing”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This invention relates to the field of computer technology, and in particular to a method, apparatus, and computing device for remote data processing. Background Technology

[0003] With the continuous development of computer technology, application scenarios for reading and writing data based on remote access are becoming increasingly common. Data systems based on remote access typically include client devices, management devices, and storage devices. Storage devices are used to store data, management devices manage the data in the storage devices, and client devices perform operations (read or write) on the data in the storage devices through interaction with the management devices.

[0004] In order to speed up access to remote data in a file system based on remote access, remote data information is usually cached on the client side, such as caching the specific data to be accessed.

[0005] In a file system based on remote access, to ensure cache consistency, after the client sends a first request message to the remote management device for operation (e.g., update operation) on the target data, the remote management device first determines whether the target data is currently cached consistently (i.e., the target data stored on all devices in the system is consistent). If the target data cache is consistent (if inconsistent, an invalidation operation is performed first), the management device sends a notification message to the client device. Upon receiving the notification message, the client device sends a second request message to the management device, which then sends the target data back to the client device, allowing the client device to perform operations on the target data.

[0006] However, because the data operation requires multiple interactions between the client device and the management device to complete the operation, the efficiency of remote data processing is not high. Summary of the Invention

[0007] This application provides a method, apparatus, and computing device for remote data processing, which can improve the efficiency of remote data processing.

[0008] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0009] In a first aspect, embodiments of this application provide a method for remote data processing, applied to a first device, wherein the first device and a second device communicate remotely. The method includes: receiving an operation request for data to be processed sent by the second device, the operation request including a read request or a write request; responding to the operation request, performing the operation indicated by the operation request on the data to be processed if a preset condition is met; the preset condition is used to indicate that the data to be processed is in a cache consistency state; responding to the operation request, performing a invalidation operation on the data to be processed if the preset condition is not met; wherein the invalidation operation is used to invalidate the data to be processed in a replica device, the replica device being a device that caches the data to be processed.

[0010] Because the above method allows the first device to directly execute the operation request instruction on the data to be processed when the preset conditions are met, it omits the related operations of the first device replying to the second device with a condition met message, and the second device sending a second operation request to the first device after receiving the condition met message. This reduces the number of interactions between the first and second devices, thereby improving the efficiency of remote data processing.

[0011] In one possible implementation, the preset conditions include: the last operation on the data to be processed is an invalidation operation, or the operation request is a read request and the last operation on the data to be processed is a read operation.

[0012] In one possible implementation, the operation request is a first message, which uses the Remote Direct Data Access Protocol (RDMA) and carries a Cache Coherence Protocol (CC) message. The message type field in the first message indicates the cache coherence type.

[0013] In one possible implementation, the operation of executing the operation request instruction on the data to be processed includes: sending the data to be processed to the second device; and, if the operation request is a write request, the second device performs the write request instruction operation on the received data to be processed.

[0014] In the above embodiment, when the preset conditions are met, the first DPU directly sends the data to be processed to the second DPU, so that the second DPU can perform operations on the data to be processed. This eliminates the need for the first DPU to reply with a condition met message to the second DPU, the second DPU to receive the condition met message, and the related operations of sending a second operation request to the first DPU. This reduces the number of interactions between the first DPU and the second DPU, thereby improving the efficiency of remote data processing.

[0015] In one possible implementation, sending the data to be processed to the second device includes: sending a second message carrying the data to be processed to the second device, wherein the data field of the second message is used to carry the data to be processed, the second message adopts the RDMA protocol, the second message carries a CC message, and the message type field in the second message indicates the cache consistency type.

[0016] In one possible implementation, when the operation request is a write request, the first message also includes an operation field, which is used to indicate the updated data of the data to be processed; and performs the operation indicated by the operation request on the data to be processed, including: updating the data to be processed according to the updated data; and replacing the replica device with the second device.

[0017] In the above embodiment, when the operation request is a write request, the message sent by the second DPU to the first DPU, which indicates the write request, carries updated data for the data to be processed. When the aforementioned preset condition is met, the first DPU directly performs an update operation on the data to be processed based on the updated data in the message. This eliminates the need for the first DPU to reply with a condition-met message to the second DPU, the second DPU to receive the condition-met message, and the second DPU to send a second operation request to the first DPU. This reduces the number of interactions between the first and second DPUs, thereby improving the efficiency of remote data processing.

[0018] In one possible implementation, the first device and the second device communicate remotely, including: a first data accelerator in the first device communicating remotely with a second data accelerator in the second device, wherein the data accelerator has better data processing performance than a central processing unit (CPU).

[0019] In the above method, the data accelerator determines whether the preset conditions are met, and if the preset conditions are met, the data accelerator sends the data to be processed to the data accelerator in the second device. If the preset conditions are not met, the data to be processed is invalidated. Since the data processing performance of the data accelerator is better than that of the CPU, the efficiency of remote data processing is further improved.

[0020] In one possible implementation, the last operation on the data to be processed is a write operation. Before performing the invalidation operation on the data to be processed, the method further includes: storing the data to be processed cached by the replica device to the storage space managed by the first device for storing the data to be processed.

[0021] In the above embodiment, when the status information of the data to be processed is in a write state, the last operation on the data to be processed is a write operation. That is, the operation on the data to be processed most recently is an update operation. Since the updated data to be processed is not instantiated to the storage address of the data to be processed, that is, the data stored at the storage address of the data to be processed is the data to be processed before the update, the updated data to be processed is instantiated and then the invalidation operation is performed on the data to be processed, thereby ensuring the accuracy and consistency of the data.

[0022] In one possible implementation, the status information of the data to be processed is configured to indicate the last operation on the data to be processed. The method further includes updating the status information after the target operation is performed on the data to be processed, so that the updated status information indicates the target operation, which includes a read operation, a write operation, or a invalidation operation.

[0023] In one possible implementation, the aforementioned state information is cached in the memory of the first device.

[0024] The above embodiments cache the status information of the data in the memory of the first device. Since the cache space of the memory is larger than the cache space in the CPU, the memory of the first device can cache the status information of more data (such as all data). This avoids, to a certain extent, the situation where the status information of the data to be processed does not exist on the first device, and the first device has to obtain the status information of the data to be processed from other devices. Therefore, the efficiency of remote data processing is improved.

[0025] In one possible implementation, the data accelerator includes a data processor (DPU) or a smart network interface card (NIC).

[0026] In one possible implementation, the above operation request is a read request, and the method further includes adding a second device to the replica device.

[0027] Secondly, embodiments of this application provide a data processing apparatus deployed on a first device, the first device communicating remotely with a second device. The data processing apparatus includes a transceiver module and a processing module. The transceiver module is used to receive an operation request for data to be processed sent by the second device, the operation request including a read request or a write request. The processing module is used to respond to the operation request and, under preset conditions, execute the operation indicated by the operation request on the data to be processed. The preset conditions are used to indicate that the data to be processed is in a cache consistency state.

[0028] The processing module is used to respond to an operation request and, if preset conditions are not met, perform a invalidation operation on the data to be processed; wherein, the invalidation operation is used to invalidate the data to be processed in the replica device, and the replica device is a device that caches the data to be processed.

[0029] In one possible implementation, the aforementioned preset conditions include: the last operation on the data to be processed was an invalidation operation, or the operation request was a read request and the last operation on the data to be processed was a read operation.

[0030] In one possible implementation, the aforementioned preset conditions include: the last operation on the data to be processed is an invalidation operation, or the operation request is a read request and the last operation on the data to be processed is a read operation.

[0031] In one possible implementation, the above operation request is a first message, which adopts the Remote Direct Data Access Protocol (RDMA), carries a Cache Coherence Protocol (CC) message, and the message type field in the first message indicates the cache coherence type.

[0032] In one possible implementation, the transceiver module is used to send data to be processed to the second device; when the operation request is a write request, the second device is used to perform the write request instruction operation on the received data to be processed.

[0033] In one possible implementation, the transceiver module is used to send a second message carrying data to be processed to the second device. The data field of the second message is used to carry the data to be processed. The second message adopts the RDMA protocol and carries a CC message. The message type field in the second message indicates the cache consistency type.

[0034] In one possible implementation, the processing module is used to update the data to be processed based on the update data when the operation request is a write request; the first message also includes an operation field, which is used to indicate the updated data of the data to be processed; the processing module is used to replace the replica device with the second device.

[0035] In one possible implementation, a first data accelerator in a first device communicates remotely with a second data accelerator in a second device, and the data accelerator has better data processing performance than a central processing unit (CPU).

[0036] In one possible implementation, the data processing apparatus includes: a storage module; and a storage device for storing the data to be processed cached by the copy device to a storage space managed by the first device for storing the data to be processed.

[0037] In one possible implementation, the processing module is used to update the status information after performing a target operation on the data to be processed, so that the updated status information indicates the target operation, which may include a read operation, a write operation, or a invalidation operation.

[0038] In one possible implementation, the aforementioned state information is cached in the memory of the first device.

[0039] In one possible implementation, the aforementioned data accelerator includes a data processor (DPU) or a smart network interface card (NIC).

[0040] In one possible implementation, the aforementioned processing module is used to add the second device to the copy device.

[0041] Thirdly, embodiments of this application provide a computing device including a memory and at least one processor connected to the memory. The memory is used to store computer program code, which includes computer instructions. When the computer instructions are executed by the at least one processor, the computing device performs the method described in the first aspect and any of its possible implementations.

[0042] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions, which, when executed on a computer, perform any one of the methods of the first aspect and its possible implementations.

[0043] Fifthly, embodiments of this application provide a computer program product comprising computer instructions that, when executed on a computer, perform the method described in the first aspect and any of its possible implementations.

[0044] It should be understood that the beneficial effects of the technical solutions and corresponding possible implementations of the second to fifth aspects of this application can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0045] Figure 1 is a schematic diagram of a remote data processing method provided in an embodiment of this application;

[0046] Figure 2 is a schematic diagram of a remote data operating system provided in an embodiment of this application;

[0047] Figure 3 is a hardware schematic diagram of a computing device provided in an embodiment of this application;

[0048] Figure 4 is a schematic flowchart of a remote data processing method provided in an embodiment of this application;

[0049] Figure 5 is a schematic diagram of the structure of an RDMA message provided in an embodiment of this application;

[0050] Figure 6 is a schematic diagram of the structure of a CC over RDMA message provided in an embodiment of this application;

[0051] Figure 7 is a second schematic flowchart of a remote data processing method provided in an embodiment of this application;

[0052] Figure 8 is a third schematic flowchart of a remote data processing method provided in an embodiment of this application;

[0053] Figure 9 is a fourth schematic flowchart of a remote data processing method provided in an embodiment of this application;

[0054] Figure 10 is a schematic diagram of another remote data operating system provided in an embodiment of this application;

[0055] Figure 11 is a schematic diagram of another remote data operating system provided in an embodiment of this application;

[0056] Figure 12 is a schematic diagram of data processing time provided in an embodiment of this application;

[0057] Figure 13 is a schematic diagram of a data processing device provided in an embodiment of this application. Detailed Implementation

[0058] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0059] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first device" and "second device," etc., are used to distinguish different devices, not to describe a specific order of devices.

[0060] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0061] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processors means two or more processors.

[0062] In remote access-based file systems, to accelerate access to remote data, remote data information is typically cached on the client side, such as the specific data to be accessed. Based on this, to ensure cache consistency, a common remote data processing method is shown in Figure 1. When client 1 needs to perform an update operation on the data to be processed (e.g., ob), client 1 sends a first request message to the management device (e.g., management device or switch) for the operation (e.g., update operation) on the target data. In response to the first request message, the management device determines that the last operation on ob1, as indicated by ob1's status information, was a read operation, and that ob1 exists on a replica device (i.e., client 2), which is the device that caches ob1. The management device sends a invalidation request to client 2, causing client 2 to delete its cached ob1, and then sends a invalidation response message to the management device. In response to the invalidation response message, the management device sends a feedback message to client 1, informing client 1 that ob1 can be updated. In response to the feedback message, client 1 sends a second request message to the management device, causing the management device to update the ob1 stored in the storage space indicated by the storage address of ob1 based on the second request message.

[0063] However, because the data operation requires multiple interactions between the client device and the management device to complete the operation, the efficiency of remote data processing is not high.

[0064] To address the aforementioned issues, this application provides a method for remote data processing. This method is applied to a first device that remotely communicates with a second device. Upon receiving an operation request (i.e., a read request or a write request) from the second device, the first device, in response to the operation request, executes the operation indicated by the operation request on the data to be processed if preset conditions are met; otherwise, it performs a invalidation operation on the data to be processed. The preset conditions include: the last operation on the data to be processed was a invalidation operation, or the operation request was a read request and the last operation on the data to be processed was a read operation. Because the above method directly executes the operation indicated by the operation request on the data to be processed when the preset conditions are met, it omits the operations of the first device replying to the second device with a condition-met message, and the second device sending a second operation request to the first device after receiving the condition-met message. This reduces the number of interactions between the first and second devices, thus improving the efficiency of remote data processing.

[0065] The remote data processing method provided in this application embodiment is applied to a remote data operating system as shown in Figure 2. The system includes a management device, client 1, and client 2. Specifically, any one of the management device, client 1, and client 2 can be a computing device with transmission and reception capabilities, such as a central server, edge server, desktop computer, mobile phone, or satellite equipment.

[0066] Client 1 is the device that needs to perform operations on the data to be processed (e.g., ob1); Client 1 is used to send operation requests for the data to be processed to the management device.

[0067] The management device is used to respond to the above operation request, determine whether the preset conditions are met, and if the preset conditions are met, execute the operation indicated by the above operation request on the data to be processed; if the preset conditions are not met, control the cached data to be processed on the replica device to become invalid; the replica device is a device that caches the data to be processed.

[0068] Client 2 is the aforementioned replica device, used to perform invalidation operations on the cached pending data of Client 2 under the control of the management device.

[0069] In one implementation, when the data to be processed is data stored on a storage device, the remote data operating system further includes: a storage device; and the storage address of the data to be processed (e.g., ob1) is cached in the management device.

[0070] Based on this, the specific implementation of the operation request instruction for the data to be processed by the aforementioned management device is that the management device performs the operation request instruction on the data to be processed on the storage device according to the storage address of the data to be processed.

[0071] The aforementioned storage devices can be computing devices with data storage functions, such as database servers, object storage service (OBS) storage servers, and cloud storage servers.

[0072] For example, Figure 3 is a schematic diagram of the hardware structure of the aforementioned management device as a computing device; Figure 3 is a schematic diagram of the hardware structure of a computing device provided in an embodiment of this application. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone. The various components shown in Figure 3 can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0073] As shown in Figure 3, the computing device may include: a processor 201, a memory 202, a communication interface 203, and a data accelerator 205. The processor 201, memory 202, communication interface 203, and data accelerator 205 can be connected to each other via a bus 204 or other means.

[0074] Processor 201 includes one or more CPUs. The CPU can be a single-core CPU or a multi-core CPU. Optionally, processor 201 may also include a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), etc.

[0075] The memory 202 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical memory, disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer. In this embodiment, the memory 202 may store computer instructions and other information.

[0076] In one possible implementation, the memory 202 may exist independently of the processor 201. The memory 202 can be connected to the processor 201 via a bus 204 and is used to store data, instructions, or program code. When the processor 201 calls and executes the instructions or program code stored in the memory 202, it can implement the relevant steps in the remote data processing method provided in the embodiments of this application.

[0077] In another possible implementation, the memory 202 can also be integrated with the processor 201.

[0078] The communication interface 203 can be a transceiver module used to communicate with other devices or communication networks, such as Ethernet, RAN, wireless local area networks (WLAN), etc. The communication interface 203 can receive commands, messages, or data. The transceiver module can be a transceiver or similar device.

[0079] Optionally, the communication interface 203 can also be a transceiver circuit located within the processor 201, used to implement signal input and signal output of the heterogeneous processor. The communication interface 203 can be a wired interface (port), such as a fiber distributed data interface (FDDI) or a gigabit Ethernet (GE) interface, or it can also be a wireless interface.

[0080] Bus 204 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. This bus can be categorized as an address bus, data bus, control bus, etc. Buses can also be classified as serial buses and parallel buses. For ease of representation, only one thick line is used in Figure 3, but this does not indicate that there is only one bus or one type of bus.

[0081] The data accelerator 205 can be a data processing unit (DPU), a field-programmable gate array (FPGA), or a smart network interface card, used to communicate with the data accelerator at the other end and to perform operations on the data in the memory of the other end.

[0082] It should be noted that the computing device in Figure 3 is merely an example of a computing device. The computing device may have more or fewer components than those shown in Figure 3, may combine two or more components, or may have different component configurations. For example, the computing device may also include a smart network card, such as a data processing unit (DPU).

[0083] The remote data processing method provided in this application embodiment is applied to a management device in a remote data operating system shown in FIG2. The first device communicates remotely with a second device. The first device can be the management device shown in FIG2, and the second device can be the client 1 shown in FIG2. As shown in FIG4, the method includes: S110-S140.

[0084] S110, the first device receives the operation request for the data to be processed sent by the second device.

[0085] The aforementioned operation requests include either read requests or write requests. A read request is used to request the acquisition of the data to be processed; a write request is used to request the updating of the data to be processed. In other words, a read request indicates a read operation to acquire the data to be processed, and a write request indicates a write operation (i.e., an update operation) to update the data to be processed. For example, the operation request received by the first device could be a read request to read data (e.g., ob1) or a write request to update ob1.

[0086] The above operation request can be sent to the first device via a message or via a data stream. This application embodiment does not specifically limit the specific implementation of the request.

[0087] When the aforementioned operation request is sent to the first device via a message (hereinafter referred to as the "first message"), the first message is used to indicate the operation request (i.e., the operation request is the first message). The first message employs the Remote Direct Memory Access (RDMA) protocol and carries a cache coherence (CC) protocol message. In other words, the first message indicating the aforementioned operation request is an RDMA message, which encapsulates a CC message; that is, the message structure of the first message is CC over RDMA.

[0088] The first message mentioned above includes a message type field, which indicates a cache consistency type. This cache consistency type is used to indicate that the current message is the CC over RDMA message mentioned above. When the first device parses the received message type as a CC over RDMA message, it executes the following S120-S140.

[0089] It should be noted that RDMA messages are used to carry message type fields for CC messages and cache coherency types, but the RDMA protocol does not define messages that carry message type fields for CC messages and cache coherency types. Therefore, this application embodiment requires protocol extensions to the existing RDMA protocol-defined messages to implement messages used for transmitting message type fields for CC messages and cache coherency types.

[0090] For example, an existing RDMA message, as shown in Figure 5, includes an RDMA message header and an RDMA message body. The "BTH" field in the RDMA message header represents the basic transport header (BTH), which indicates basic information about the transmitted data and control commands. The "OpCode" field indicates the message type; the "SE" (sequence element) field indicates an element in the message sequence; the "M" field indicates the message type; the "Pad" field is used for padding to ensure the header length meets specific alignment requirements; the "Tver" field indicates the transport version; the "Partition Key" indicates the partition key; "F" is used for flow control; the "B" field indicates the buffer model; the "Reserve" field represents reserved bits; the "dst_qid" field indicates the queue identifier; the "A" field is a flag field indicating whether the acknowledgment number field is valid; "Reserve" is a second reserved bit; and the "PSN" field indicates the packet sequence number. The "payload" field in the RDMA message body carries the valid data to be transmitted.

[0091] Based on the example in Figure 5, this embodiment of the application adds a field value to the “OpCode” field in the header of the RDMA message to indicate the consistency type, or adds a message type field (such as “OpCode2”), which is used to indicate the cache consistency type; and encapsulates the CC message in the “payload” field in the body of the RDMA message, or adds a field (such as the “message” field) to the body of the RDMA message to carry the CC message.

[0092] For example, the message structure of the first message is shown in Figure 6. The first message includes: an RDMA message header, a CC message header, and a CC message body. Among them, the "OpCode" field in the RDMA message header has a new field value, which is used to represent the cache consistency type. For example, the original value range of the "OpCode" field is "00" to "01". A "10" field value is added to the "OpCode" field so that the value range of the "OpCode" field is "00" to "10". The "10" field value represents the cache consistency type.

[0093] The "CCTH" field in the CC header indicates the protocol thread to which the packet belongs; the "OpCode" field indicates the operation type; for example, in the case of a read request, the value of the "OpCode" field indicates a read operation; in the case of a write request, the value of the "OpCode" field indicates a write operation. The "ResCode" field indicates whether the operation was successful; the "Reserve" field indicates a reserved field; the "ClientID" field indicates the identifier of the device that sent the request; the "PoolID" field indicates the identifier of the buffer pool; and the "PageID" field indicates the page identifier. The "payload" field in the CC body carries the valid data to be transmitted.

[0094] S120, The first device determines whether the preset conditions are met.

[0095] The preset condition is used to indicate that the data to be processed is in a cache consistency state.

[0096] For example, the above preset conditions include: the last operation on the data to be processed from the current time is a invalidation operation, or the operation request is a read request and the last operation on the data to be processed is a read operation.

[0097] The above invalidation operation is used to invalidate the pending data cached in the replica device, so that when the replica device needs to obtain the pending data again, it cannot directly obtain the pending data from the local cache, but needs to obtain the pending data again from the storage location where the pending data is stored; the replica device is the device that caches the pending data; its specific implementation is described in S140 below, and will not be repeated here.

[0098] It should be understood that the above-mentioned preset conditions are not met in the following situations: (1) the operation request is a read request and the last operation on the data to be processed is a write operation; (2) the operation request is a write operation and the last operation on the data to be processed is a read operation; (3) the operation request is a write operation and the last operation on the data to be processed is a write operation.

[0099] It should be noted that the last operation on the data to be processed can be obtained from the log or from the consistency information cached by the first device. This application does not specifically limit the specific implementation of the data.

[0100] For example, consistency information can be used to indicate the last operation on the data. When the last operation on the data to be processed is obtained from the consistency information cached in the first device, the consistency information includes: the correspondence between data identifiers and status information, that is, the status information of data (e.g., data to be processed) configured in the first device; wherein, the status information of a data (e.g., ob1) is used to indicate the last operation on ob1.

[0101] Specifically, the status information includes: read status, write status, or invalidation status; where, when the data status is read, it indicates that the last operation on the data was a read operation; when the status is write, it indicates that the last operation on the data was a write operation; and when the status is invalidation, it indicates that the last operation on the data was an invalidation operation.

[0102] It should be noted that when the status information is in an invalid state, there is no device that caches the data (i.e., a copy device). In other words, the invalid state can also be used to indicate that the last operation on the data was an operation on which there is no copy device for the data. This application does not specifically limit this.

[0103] For example, the consistency information, as shown in Table 1 below, includes three data identifiers and three status information. The three data identifiers include ob1 to ob3, and the three status information includes: read status, write status, and invalidation status. Specifically, the status information of ob1 is read status, meaning that the last operation on ob1 was a read operation; the status information of ob2 is write status, meaning that the last operation on ob2 was a write operation; and the status information of ob3 is invalidation status, meaning that the last operation on ob3 was an invalidation operation.

[0104] Table 1

[0105] If the preset conditions are met, in response to the operation request received in S110 above, S130 is executed.

[0106] If the preset conditions are not met, S140 is executed in response to the operation request received in S110 above.

[0107] S130, The first device performs the operation request instruction on the data to be processed.

[0108] In one implementation, S130 is implemented as follows: the first device sends data to be processed to the second device, so that the second device can operate the data to be processed.

[0109] For example, in the case of a read request, in S130 the first device sends the data to be processed to the second device so that the second device can obtain the data. In the case of a write request, in S130 the first device sends the data to be processed to the second device so that the second device can perform an update operation on the data. The specific implementation of S130 is described in S210-S240 and S310-S360 below, and will not be repeated here.

[0110] The method for the first device to send data to be processed to the second device includes: the first device sending a second message carrying the data to be processed to the second device. The second message uses the RDMA protocol and carries a CC message. The message type field in the second message indicates the cache consistency type; that is, the message structure of the second message, as shown in Figure 6, is CC over RDMA. This second message is a reply or response message to the first message.

[0111] The second message mentioned above includes a data field, which is used to carry the data to be processed. For example, the data field can be the "payload" field in the CC over RDMA message shown in Figure 6, or it can be a newly added field (such as the "message" field) in the CC message body of the CC over RDMA message, which is used to carry the data to be processed.

[0112] In another implementation, S130 is implemented as follows: when the operation request is a write request, the first device performs an update operation on the data to be processed based on the write request; when the operation request is a read request, the first device sends the data to be processed to the second device so that the second device can obtain the data to be processed; the specific implementation of S130 is described in S210-S240 and S410-S450 below, and will not be repeated here.

[0113] For example, assuming the above operation request is a write request to update ob1, and the status information of ob1 is used to indicate that the last operation on ob1 was a invalidation operation, then the above preset condition is met; the first device performs an update operation on ob1.

[0114] S140, The first device performs a invalidation operation on the data to be processed.

[0115] The aforementioned invalidation operation is used to invalidate the pending data cached in the replica device, which is the device that caches the pending data, and to update the status information of the pending data to an invalidated state. Specifically, the invalidation operation can either delete the pending data cached in the replica device, or mark the pending data cached in the replica device as invalid, so that when the replica device needs to perform an operation on the pending data, it must re-acquire the pending data.

[0116] It should be noted that the above-mentioned method of deleting the cached pending data in the replica device can be that the first device sends a invalidation command to the replica device, and the replica device responds to the invalidation command by deleting the locally cached pending data; or the DPU in the first device can delete the cached pending data in the replica device based on RDMA one-sided mode. The specific embodiments of this application do not limit it in detail.

[0117] For example, based on Table 2, assuming the above operation request is a write request to update ob1, and the status information of ob1 is used to indicate that the last operation on ob1 was a read operation, then ob1 does not meet the above preset conditions; the first device deletes the cached ob1 in the replica device.

[0118] Optionally, after executing S140 above, the first device performs the operation of the operation request instruction on the data to be processed, as detailed in S130 above, which will not be repeated here.

[0119] It should be noted that, when the status information of the data to be processed indicates the last operation on the data to be processed, after the first device performs the target operation on the data to be processed (such as the operation indicated by the operation request or the invalidation operation), the first device updates the status information of the data to be processed so that the updated status information indicates the target operation.

[0120] The aforementioned target operations include: read operations, write operations, or invalidation operations; wherein, when the first device performs a read operation on the data to be processed, the first device updates the status information of the data to be processed to the read status; when the first device performs a write operation on the data to be processed, the first device updates the status information of the data to be processed to the write status; when the first device performs an invalidation operation on the data to be processed, the first device updates the status information of the data to be processed to the invalidation status.

[0121] Because the above method allows the first device to directly execute the operation request instruction on the data to be processed when the preset conditions are met, it omits the related operations of the first device replying to the second device with a condition met message, and the second device sending a second operation request to the first device after receiving the condition met message. This reduces the number of interactions between the first and second devices, thereby improving the efficiency of remote data processing.

[0122] It should be noted that, as shown in the experimental data in Figure 12, in a single-threaded scenario, the latency for the CPU in the management device to process the queued tasks is 4.15 microseconds; in a high-concurrency scenario, the latency for the CPU in the management device to process the queued tasks is 51.15 microseconds. It can be seen that in a high-concurrency scenario, the latency for the CPU to process the queued tasks increases by 10 times, making the CPU the performance bottleneck of the background technology, thus reducing the efficiency of remote data processing.

[0123] Based on this, in one implementation, the first and second devices employ internal data accelerators to implement the processes S110 to S140. For example, the data accelerator can be a DPU or a smart network interface card (NIC).

[0124] Based on the remote data processing method shown in Figure 4, this application provides Embodiment 1 and Embodiment 2. Embodiment 1 is a specific implementation of this application when the operation request in S110 is a read request; Embodiment 2 is two different specific implementations of this application when the operation request in S110 is a write request.

[0125] Example 1

[0126] Based on the remote data processing method shown in Figure 4, when the operation request in S110 is a read request, this application embodiment provides a specific implementation method. This method is applied to a first data accelerator in a first device, which remotely communicates with a second data accelerator in a second device. The data accelerator (i.e., the first data accelerator and the second data accelerator) has better data processing performance than a CPU. Specifically, the data accelerator can be a DPU or a smart network interface card (NIC). For ease of description, this application embodiment uses a DPU as an example for illustration, and will not elaborate further thereafter.

[0127] As shown in Figure 7, the specific implementation method provided in this application embodiment includes: S210-S250.

[0128] S210, the first DPU in the first device receives a first request message sent by the second DPU in the second device to indicate a read request.

[0129] The above read request is used to request the data to be processed.

[0130] The first message used to indicate the operation request in S110 includes the first request message; that is, when the operation request in S110 is a read request, the first request message is the first message in S110; the first request message carries an identifier of the data to be processed.

[0131] It should be noted that the message structure of the first request message is similar to that of the first message in S110. For a detailed description of the first request message, please refer to the relevant description of the message structure of the first message in S110. It will not be repeated here.

[0132] It should be noted that the aforementioned first request message may be sent proactively by the second DPU to the first DPU, or it may be sent by the CPU in the second device to the second DPU as a command, and the second DPU responds to the command by sending a message to the first DPU. The specific embodiments of this application do not specifically limit it.

[0133] The first DPU responds to the first request message and executes the following S220.

[0134] S220, the first DPU determines whether the preset conditions are met.

[0135] The above-mentioned S220 can be implemented in the following ways: S221-S222.

[0136] S221, The first DPU acquires the status information of the data to be processed.

[0137] It should be noted that when the status information of the data to be processed is in the read state, this status information indicates that the last operation on the data to be processed was a read operation; when the status information of the data to be processed is in the write state, this status information indicates that the last operation on the data to be processed was a write operation; when the status information of the data to be processed is in the invalidated state, this status information indicates that the last operation on the data to be processed was a invalidated read operation.

[0138] It should be noted that the content of the above consistency information is shown in Table 1 above. Specifically, the consistency information can be cached in the memory of the first device or in the cache of the first DPU; that is, the status information of the above data can be cached in the memory of the first device or in the cache of the first DPU; the specific embodiments of this application do not limit it.

[0139] The above embodiments cache the status information of the data in the memory of the first device. Since the cache space of the memory is larger than the cache space in the CPU, the memory of the first device can cache the status information of more data (such as all data). This avoids, to a certain extent, the situation where the status information of the data to be processed does not exist on the first device, and the first device has to obtain the status information of the data to be processed from other devices. Therefore, the efficiency of remote data processing is improved.

[0140] For example, as shown in Figure 10, the first device caches the status information of data A to the status information of data B; the third device caches the status information of data C to the status information of data D. When the second device sends an operation request to the first device to operate on data C, if the first device needs to obtain the status information of data C, since the first device does not store the status information of data C, the first device needs to obtain the status information of data C from the third device, thus reducing the efficiency of remotely processing data C.

[0141] Based on this, as shown in Figure 11, when the first device caches the status information of the data in its memory, the first device caches the status information of data A to the status information of data D in its memory. Therefore, when the second device sends an operation request to the first device to operate on data C, the first device directly obtains the status information of data C from its memory and does not need to obtain the status information of data C from the third device.

[0142] S222. The first DPU determines whether the preset conditions are met based on the status information of the data to be processed.

[0143] For example, suppose the preset conditions include: the last operation on the data to be processed since the current time was a invalidation operation, or the operation request was a read request and the last operation on the data to be processed was a read operation. Further suppose the consistency information is as shown in Table 1 above. If the identifier of the data to be processed is ob1, the status information of ob1 indicates that the last operation on ob1 was a read operation; since the first request message is used to indicate a read request, the preset conditions are met. If the identifier of the data to be processed is ob2, the status information of ob2 indicates that the last operation on ob2 was a write operation; since the first request message is used to indicate a read request, the preset conditions are not met. If the identifier of the data to be processed is ob3, the status information of ob3 indicates that the last operation on ob3 was a invalidation operation, so the preset conditions are met.

[0144] If the preset conditions are met, in response to the first request message received in S210 above, the following S230-S240 are executed.

[0145] If the preset conditions are not met, in response to the first request message received in S210 above, the following S250 is executed.

[0146] S230, the first DPU sends a first response message carrying the data to be processed to the second DPU.

[0147] The second message carrying the data to be processed in S130 above includes the first response message above, that is, the first response message is the second message in S130 above; the first response message is a response message or reply message to the first request message above.

[0148] It should be noted that the message structure of the first response message is similar to that of the first message in S110. For a detailed description of the first response message, please refer to the relevant description of the message structure of the first message in S110. It will not be repeated here.

[0149] For example, the message structure of the first response message mentioned above is shown in Figure 6 as CC over RDMA; at this time, the “OpCode” in the CC message header of the CC over RDMA message is used to indicate the read operation; the “ResCode” field indicates that the operation was successful; the “ClientID” field indicates the identifier of the second DPU; and the “payload” field in the CC message body carries the data to be processed.

[0150] The data to be processed may be data stored on the first device or data stored on other devices (such as storage devices). In specific embodiments of this application, the storage location of the data to be processed is not limited.

[0151] When the data to be processed is stored on the first device, the above-mentioned S230 is implemented as follows: the first DPU obtains the data to be processed from the memory of the first device and sends the data to be processed to the second DPU through the above-mentioned first response message, so that the second device where the second DPU is located caches the data to be processed.

[0152] When the data to be processed is stored on other devices (such as storage devices), the above-mentioned S230 is implemented as follows: the first DPU obtains the data to be processed from the storage space indicated by the storage address of the data to be processed, and sends the data to be processed to the second DPU through the above-mentioned first response message, so that the second device where the second DPU is located caches the data to be processed.

[0153] In one implementation, the consistency information includes the storage address of the data to be processed; specifically, this consistency information includes the storage address corresponding to each piece of data, as shown in Table 2 below, based on Table 1 above. Specifically, the storage address of ob1 is address 1, the storage address of ob2 is address 2, and the storage address of ob3 is address 3.

[0154] For example, if the data to be processed is ob1, the first DPU obtains the storage address of ob1 (i.e., address 1) from the cache consistency information; then, the first DPU obtains the data indicated by ob1 from the storage space indicated by address 1, and sends the data to the second device through the first response message mentioned above.

[0155] Table 2

[0156] S240, the first DPU adds the second device to the copy device.

[0157] The aforementioned replica device is a device that caches data that is yet to be processed.

[0158] It should be understood that since the second DPU will cache the data to be processed after acquiring it, a second device where the second DPU is located is added to the replica device so that the replica device includes the second device.

[0159] For example, assuming the status information of the data to be processed is in the read state, the replica device includes: device A and device B; after the second device is added to the replica device, the replica device includes: device A, device B and the second device.

[0160] It should be noted that when the status information of the data to be processed is in an invalidated state, there is no replica device for the data to be processed; after executing the above S240, the method further includes: updating the status information of the data to be processed to a read state.

[0161] In one implementation, the aforementioned consistency information also includes replica devices of the device to be processed. Specifically, the consistency information is shown in Table 3 below, based on Table 2 above. Among them, the replica devices of ob1 include device A and device B, the replica device of ob2 includes device B, and since the status information of ob3 is in a deactivated state, that is, the last operation on ob3 was a deactivated operation, the replica device of ob3 is empty, that is, ob3 does not have a replica device.

[0162] For example, if the data to be processed is ob3, the second device is added to the replica device of ob3, and the status information of ob3 is updated to read status, as shown in Table 4.

[0163] Table 3

[0164] Table 4

[0165] S250, the first DPU performs a invalidation operation on the data to be processed.

[0166] It should be understood that when the operation request for the data to be processed is a read operation, and the above-mentioned preset conditions are not met, the status information of the data to be processed is a write operation.

[0167] The aforementioned invalidation operation also includes: invalidating the pending data cached in the replica device; then, updating the status information of the pending data to the invalidated state, and deleting the identifier of the replica device of the pending data from the consistency information.

[0168] In the above embodiment, when the operation request is a read request, if the preset condition is met, the first DPU directly sends the data to be processed to the second DPU, omitting the operations of the first DPU replying to the second DPU with a condition met message, the second DPU receiving the condition met message, and sending a second operation request to the first DPU. This reduces the number of interactions between the first DPU and the second DPU, thereby improving the efficiency of remote data processing.

[0169] Furthermore, in the above method, the data accelerator determines whether the preset conditions are met, and if the preset conditions are met, the data accelerator sends the data to be processed to the data accelerator in the second device; if the preset conditions are not met, the data to be processed is invalidated. Since the data processing performance of the data accelerator is better than that of the CPU, the efficiency of remote data processing is further improved.

[0170] Example 2

[0171] Based on the remote data processing method shown in Figure 4, when the operation request in S110 is a write request, this application embodiment provides a specific implementation method, as shown in Figure 8, which includes: S310-S390.

[0172] S310, the first DPU receives a second request message sent by the second DPU to indicate a write request.

[0173] The write request described above is used to request an update of the data to be processed.

[0174] The first message used to indicate the operation request in S110 includes the second request message. That is, when the operation request in S110 is a write request, the second request message is the first message in S110. The second request message carries an identifier of the data to be processed.

[0175] S320, the first DPU determines whether the preset conditions are met.

[0176] It should be understood that when the second request message is used to indicate a write request, the above-mentioned preset condition is met when the status information of the data to be processed is in an invalidated state; the above-mentioned preset condition is not met when the status information of the data to be processed is in a read state or a write state.

[0177] If the preset conditions are met, in response to the second request message received in S310 above, the following S330-S360 are executed.

[0178] If the preset conditions are not met, in response to the second request message received in S310 above, the following S370-S390 are executed.

[0179] S330: The first DPU sends a second response message carrying the data to be processed to the second DPU.

[0180] The second message carrying the data to be processed in S130 above includes the second response message, that is, the second response message is the second message in S130 above; the second response message is a response message or reply message to the second request message above.

[0181] It should be noted that the implementation methods of S310-S330 are similar to those of S210-S230. For a detailed description of S310-S330, please refer to the relevant descriptions of S210-S230 above. They will not be repeated here.

[0182] S340, the second DPU performs update operations on the data to be processed.

[0183] It should be understood that, since the second response message is an RDMA protocol message, it is sent directly from the first DPU to the buffer of the second DPU. After receiving the second response message, the second DPU parses it and obtains the data to be processed from its buffer.

[0184] The above-mentioned S340 is implemented by the second DPU performing an update operation on the data to be processed in its cache to obtain the updated cache data.

[0185] For example, suppose the write request indicated by the second request message above requests update data ob3, and the consistency information of the first device cache is shown in Table 3 above; the status information of ob3 is invalidated, that is, the last operation on ob3 was invalidated; then, the above preset condition is satisfied; then, the first DPU sends the data A indicated by ob3 to the cache of the second DPU through the second response message above; the second DPU responds to the second response message to update the data A in the cache, and obtains the updated data B, that is, the updated data indicated by ob3 is data B.

[0186] It should be noted that the updated data to be processed is cached in the second DPU's cache, and is not instantiated to the storage address of the data to be processed (e.g., address 3). The next time a device performs an operation on the data to be processed, the updated data to be processed will be instantiated (i.e., stored) in the storage space indicated by address 3, as detailed in S380 below, which will not be repeated here.

[0187] S350, the first DPU updates the status information of the data to be processed to write status.

[0188] It should be understood that S340 above performed a write operation (i.e., an update operation) on the data to be processed, so the last operation on the data to be processed was a write operation. Therefore, the first DPU updates the status information in the consistency information of the data to be processed to the write status.

[0189] S360, the first DPU replaces the replica device with the second device.

[0190] It should be understood that S340 above performs a write operation on the data to be processed in the cache of the second device. At this time, the device with the data to be processed after the write operation (i.e., the updated data to be processed) in the cache is the second device, so the first DPU determines the second device as the copy device of the data to be processed.

[0191] For example, based on the example of S340 above, the status information of ob3 is updated from the invalidated state to the write state; and the replica device of ob3 is updated to the second device; as shown in Table 5 below.

[0192] Table 5

[0193] S370. Determine whether the status information of the data to be processed is in write status.

[0194] If the status information of the data to be processed is in the write state, execute S380-S390.

[0195] If the status information of the data to be processed is in the read state, execute S390.

[0196] S380, the first DPU stores the data to be processed cached by the replica device into the storage space managed by the first device for storing the data to be processed.

[0197] It should be noted that when the status information of the data to be processed is in the write state, the last operation on the data to be processed is a write operation. That is to say, the operation on the data to be processed most recently is an update operation. Since the updated data to be processed has not been instantiated to the storage address of the data to be processed, the data to be processed cached by the above-mentioned replica device is the updated data to be processed.

[0198] The implementation of S380 above includes: obtaining the updated pending data cached by the replica device, and updating the updated pending data to the storage space indicated by the storage address of the pending data, so that the data stored in the storage space is the updated pending data.

[0199] For example, suppose the data to be processed is the data indicated by ob3, the status information of ob3 is write status, and the replica device is device B; the data indicated by ob3 cached by device B is data B, and the data indicated by ob3 stored at the storage address of ob3 is data A; then, the first DPU obtains the data B cached by device B, and updates the data A stored at the storage address of ob3 based on the data B, so that the data stored at the storage address of ob3 is data B.

[0200] In the above embodiment, when the status information of the data to be processed is in a write state, the last operation on the data to be processed is a write operation. That is, the operation on the data to be processed most recently is an update operation. Since the updated data to be processed is not instantiated to the storage address of the data to be processed, that is, the data stored at the storage address of the data to be processed is the data to be processed before the update, the updated data to be processed is instantiated and then the invalidation operation is performed on the data to be processed, thereby ensuring the accuracy and consistency of the data.

[0201] S390, the first DPU performs a invalidation operation on the data to be processed.

[0202] It should be understood that when the message received by the first device is used to indicate a write request, if the above-mentioned preset conditions are not met, the status information of the data to be processed is either a read status or a write status.

[0203] The aforementioned invalidation operation also includes: invalidating the pending data cached in the replica device; then, updating the status information of the pending data to the invalidated state, and deleting the identifier of the replica device corresponding to the pending data from the consistency information.

[0204] In the above embodiment, when the operation request is a write request, if the above preset conditions are met, the first DPU directly sends the data to be processed to the second DPU so that the second DPU can perform an update operation on the data to be processed. This eliminates the need for the first DPU to reply with a condition met message to the second DPU, the second DPU to receive the condition met message and send a second operation request to the first DPU, thereby reducing the number of interactions between the first DPU and the second DPU and thus improving the efficiency of remote data processing.

[0205] Based on the remote data processing method shown in Figure 4, when the operation request in S110 is a write request, this application embodiment provides another specific implementation method, as shown in Figure 9, which includes: S410-S460.

[0206] S410, the first DPU receives a third request message sent by the second DPU to indicate a write request.

[0207] The write request described above is used to request an update of the data to be processed.

[0208] The first message used to indicate the operation request in S110 includes the third request message. That is, when the operation request in S110 is a write request, the third request message is the first message in S110. The third request message contains an operation field, which is used to indicate (i.e., carry or transmit) the updated data of the data to be processed; that is, the third request message is a message that includes updated data. The updated data can be the updated data to be processed, or it can be an update operation on the data to be processed (e.g., an operation to increment the data to be processed by 1). Specifically, this application embodiment does not limit the specific content of the updated data.

[0209] It should be noted that since the first message in S110 above includes the third request message above, that is, the third request message is the first message above, the third request message is the CC over RDMA message shown in Figure 6; wherein, the above operation field can be a newly added field in the CC message body of the CC over RDMA message (such as the "mess" field), or it can be the original "payload" field in the CC message body. The specific embodiments of this application do not specifically limit it.

[0210] S420, the first DPU determines whether the preset conditions are met.

[0211] It should be understood that when the aforementioned third request message is used to indicate a write request, the aforementioned preset condition is satisfied when the status information of the data to be processed is in an invalidated state; the aforementioned preset condition is not satisfied when the status information of the data to be processed is in a read state or a write state.

[0212] It should be noted that the implementation of S420 is similar to that of S220. For a detailed description of S420, please refer to the relevant description of S220 above. It will not be repeated here.

[0213] If the preset conditions are met, in response to the third request message received in S410 above, the following S430-S450 are executed.

[0214] If the preset conditions are not met, in response to the third request message received in S410 above, the following S460 is executed.

[0215] S430 and the first DPU update the data to be processed based on the updated data.

[0216] The implementation of S430 above includes: the first DPU obtains the storage address of the data to be processed; then, according to the update data, it performs an update operation on the data to be processed stored in the storage address so that the data stored in the storage address is the updated data to be processed.

[0217] The storage address of the aforementioned data (e.g., data to be processed) can be included in the consistency information cached in memory (i.e., the content of the first device) (as shown in Table 3 above), or it can be cached separately in the memory of the first device, or it can be stored in the hard disk of the first device. The specific embodiments of this application do not specifically limit it.

[0218] S440, the first DPU updates the status information of the data to be processed to write status.

[0219] S450, the first DPU replaces the replica device with the second device.

[0220] It should be noted that the implementation methods of S440-S450 are similar to those of S350-S360. For a detailed description of S440-S450, please refer to the relevant descriptions of S350-S360 above. They will not be repeated here.

[0221] S460, the first DPU performs a invalidation operation on the data to be processed.

[0222] It should be noted that the implementation of S460 is similar to that of S390. For a detailed description of S460, please refer to the relevant description of S390 above. It will not be repeated here.

[0223] In the above embodiment, when the operation request is a write request, the message sent by the second DPU to the first DPU, which indicates the write request, carries updated data for the data to be processed. When the aforementioned preset condition is met, the first DPU directly performs an update operation on the data to be processed based on the updated data in the message. This eliminates the need for the first DPU to reply with a condition-met message to the second DPU, the second DPU to receive the condition-met message, and the second DPU to send a second operation request to the first DPU. This reduces the number of interactions between the first and second DPUs, thereby improving the efficiency of remote data processing.

[0224] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, the data processing apparatus includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0225] This application embodiment can, according to the above method, exemplarily divide a data processing device into functional modules. For example, the data processing device may include functional modules corresponding to each functional division, or two or more functions may be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; in actual implementation, there may be other division methods.

[0226] Figure 13 shows a possible structural diagram of the data processing device involved in the above embodiments, where each functional module is divided according to its corresponding function. This data processing device is deployed in the management device of the remote data operating system shown in Figure 2, and includes a transceiver module 1301 and a processing module 1302.

[0227] The transceiver module 1301 is used to receive operation requests for data to be processed sent by the second device; for example, to execute step S110 in the above method embodiment.

[0228] The processing module 1302 is used to respond to the above operation request and, if the preset conditions are met, perform the operation indicated by the operation request on the data to be processed; for example, to execute step S120 in the above method embodiment.

[0229] The processing module 1302 is used to respond to the above operation request and, if the preset conditions are not met, perform a invalidation operation on the data to be processed; for example, to execute step S130 in the above method embodiment.

[0230] Optionally, the above preset conditions include: the last operation on the data to be processed was an invalidation operation, or the operation request was a read request and the last operation on the data to be processed was a read operation.

[0231] Optionally, the above operation request is a first message. The first message adopts the Remote Direct Data Access Protocol (RDMA). The first message carries a Cache Coherence Protocol (CC) message. The message type field in the first message indicates the cache coherence type.

[0232] Optionally, the transceiver module 1301 is used to send data to be processed to the second device.

[0233] Optionally, the transceiver module 1301 is used to send a second message carrying data to be processed to the second device.

[0234] Optionally, the processing module 1302 is used to update the data to be processed according to the updated data; for example, by performing step S430 in the above method embodiment.

[0235] Processing module 1302 is also used to replace the copy device with a second device; for example, by performing step S450 in the above method embodiment.

[0236] Optionally, the first data accelerator in the first device communicates remotely with the second data accelerator in the second device, and the data accelerator has better data processing performance than the central processing unit (CPU).

[0237] Optionally, the above data processing device further includes a storage module 1303.

[0238] The storage module 1303 is used to store the data to be processed cached by the copy device to the storage space managed by the first device for storing the data to be processed; for example, to execute step S380 in the above method embodiment.

[0239] Optionally, the transceiver module 1301 is used to update the status information after performing the target operation on the data to be processed, so that the updated status information indicates the target operation.

[0240] Optionally, the aforementioned status information is cached in the memory of the first device.

[0241] Optionally, the aforementioned data accelerator may include a data processor (DPU) or a smart network interface card (NIC).

[0242] Optionally, the processing module 1302 is used to add the second device to the copy device; for example, by performing step S240 in the above method embodiment.

[0243] This application provides a computing device including a memory and at least one processor connected to the memory. The memory is used to store computer program code, which includes computer instructions. When the computer instructions are executed by the at least one processor, the computing device performs the method described above.

[0244] This application provides a computer-readable storage medium storing computer instructions, which, when executed on a computer, perform the methods described above.

[0245] This application provides a computer program product containing computer instructions that, when executed on a computer, perform the methods described above.

[0246] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center integrating one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0247] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0248] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0249] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0250] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0251] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as flash memory, portable hard disk, read-only memory, random access memory, magnetic disk, or optical disk.

[0252] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for remote data processing, characterized in that, Applied to a first device, the first device and a second device communicating remotely, the method includes: Receive an operation request for data to be processed sent by the second device, the operation request including: a read request or a write request; In response to the operation request, if a preset condition is met, the operation indicated by the operation request is performed on the data to be processed; the preset condition is used to indicate that the data to be processed is in a cache consistency state. In response to the operation request, if the preset conditions are not met, the invalidation operation is performed on the data to be processed; wherein, the invalidation operation is used to invalidate the data to be processed in the replica device, and the replica device is a device that caches the data to be processed.

2. The method according to claim 1, characterized in that, The preset conditions include: the last operation on the data to be processed was a invalidation operation, or the operation request was a read request and the last operation on the data to be processed was a read operation.

3. The method according to claim 1 or 2, characterized in that, The operation request is a first message, which uses the Remote Direct Data Access Protocol (RDMA). The first message carries a Cache Coherence Protocol (CC) message, and the message type field in the first message indicates the cache coherence type.

4. The method according to claim 3, characterized in that, The operation of executing the operation request instruction on the data to be processed includes: Send the data to be processed to the second device; When the operation request is a write request, the second device is used to perform the operation indicated by the write request on the received data to be processed.

5. The method according to claim 4, characterized in that, Sending the data to be processed to the second device includes: A second message carrying the data to be processed is sent to the second device. The data field of the second message is used to carry the data to be processed. The second message adopts the RDMA protocol and carries a CC message. The message type field in the second message indicates the cache consistency type.

6. The method according to claim 3, characterized in that, If the operation request is a write request, the first message further includes an operation field, which is used to indicate the updated data of the data to be processed; The operation of executing the operation request instruction on the data to be processed includes: Update the data to be processed based on the updated data; Replace the copy device with the second device.

7. The method according to any one of claims 1-6, characterized in that, The first device communicates remotely with the second device, including: The first data accelerator in the first device communicates remotely with the second data accelerator in the second device, and the data accelerator has better data processing performance than the central processing unit (CPU).

8. The method according to any one of claims 1-7, characterized in that, The last operation on the data to be processed is a write operation. Before performing the invalidation operation on the data to be processed, the method further includes: The data to be processed cached by the replica device is stored in the storage space managed by the first device for storing the data to be processed.

9. The method according to any one of claims 1-8, characterized in that, Configure the status information of the data to be processed to indicate the last operation on the data to be processed, and the method further includes: After performing the target operation on the data to be processed, the status information is updated so that the updated status information indicates the target operation, which includes a read operation, a write operation, or the invalidation operation.

10. The method according to claim 9, characterized in that, The status information is cached in the memory of the first device.

11. The method according to claim 7, characterized in that, The data accelerator includes a data processor (DPU) or a smart network interface card (NIC).

12. The method according to claim 4 or 5, characterized in that, The operation request is a read request, and the method further includes: Add the second device to the copy device.

13. A data processing apparatus, characterized in that, The data processing device is deployed on the first device, and the first device communicates remotely with the second device. The data processing device includes: a transceiver module and a processing module. The transceiver module is used to receive operation requests for data to be processed sent by the second device, the operation requests including: read requests or write requests; The processing module is configured to respond to the operation request and, under the condition that a preset condition is met, execute the operation indicated by the operation request on the data to be processed; the preset condition is used to indicate that the data to be processed is in a cache consistency state; The processing module is configured to respond to the operation request and, if the preset conditions are not met, perform the invalidation operation on the data to be processed; wherein the invalidation operation is used to invalidate the data to be processed in the replica device, and the replica device is a device that caches the data to be processed.

14. A computing device, characterized in that, The device includes a memory and at least one processor connected to the memory, the memory being used to store computer program code, the computer program code including computer instructions, which, when executed by the at least one processor, cause the computing device to perform the method as described in any one of claims 1 to 12.

Images