Data access method, device and equipment

Abstract

The invention discloses a data access method, device and equipment. According to the method, high-frequency concurrently read / written data is stored in a DPU (Data Processing Unit) memory of a memory node, and other data is stored in a node memory; and if the computing node determines that the first target data to be read / written is stored in the DPU memory, sending a first request to the memory node to request a DPU processor of the memory node to read / write the first target data in the DPU memory, otherwise, sending a second request to request an RDMA network card of the memory node to read / write the first target data in the node memory. The DPU processor reads and writes the data in the DPU memory, so that the reading and writing efficiency can be guaranteed when data conflicts exist; according to the method, the RDMA network card reads and writes the data in the node memory, the read-write efficiency can be guaranteed when no data conflict exists, and therefore the read-write efficiency of the data can be guaranteed no matter whether the data conflict exists or no data conflict exists, the system efficiency is improved, and the timeliness of service processing is guaranteed.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a data access method, apparatus and device. Background Art

[0002] In an era of low network performance, in order to build large-scale distributed systems, the architecture of coupled computing and storage effectively solves the problem of system scalability.

[0003] In distributed computing and storage systems, the main ways for computing nodes to access data stored in memory nodes include remote procedure call (RPC) and remote direct memory access (RDMA). RPC is a commonly used communication method between servers, that is, a server initiates an RPC request, and the server that receives the RPC request executes a pre-registered function and returns the result to the sender; specifically in a computing and memory separation system, the computing node sends an RPC request to the data processing unit (DPU) of the memory node, and the DPU parses the received RPC request and reads or writes the corresponding data from the node memory of the memory node. RDMA is a high-speed network technology that can run on Ethernet and Infiniband. It supports a server to directly operate the memory of another remote server through one-sided verbs without involving the CPU of the remote server. Specifically, in a system with separated computing and memory, the computing node sends RDMA atomic instructions to the RDMA network card of the memory node, thereby directly reading data from the node memory or writing data to the node memory without the need for data interaction through the DPU in the memory node.

[0004] If the computing node uses RPC to access the data in the memory node, it can effectively avoid the data conflict problem of multi-threading and multi-computing nodes accessing the same data, but the throughput is low, thereby reducing the performance of the computing and memory separation system. The computing node uses RDMA to access the data in the memory node. The throughput is high when there is no data conflict, but when there is a data conflict, the throughput is low and the system efficiency is poor. Summary of the invention

[0005] The present application provides a data access method and device, which are used to overcome the shortcomings of a single access mechanism, that is, low access efficiency when there is a data conflict or low access efficiency when there is no data conflict, and balance the access efficiency with and without data conflicts.

[0006] In a first aspect, an embodiment of the present application provides a data access method, the method comprising: determining a first target data to be read / written; if the key value of the first target data belongs to a first key value set, sending a first request to a memory node, the first request being used to request a first reader / writer of the memory node to read / write the first target data in a first storage medium, the first storage medium storing a first data set; if the key value of the first target data belongs to a second key value set, sending a second request to the memory node, the second request being used to request a second reader / writer of the memory node to read / write the first target data in a second storage medium, the second storage medium storing a second data set; the data in the first data set are read / written concurrently The frequency is higher than the frequency at which data in the second data set is concurrently read / written, the first storage medium supports being accessed by the first reader / writer but does not support being accessed by the second reader / writer, the second storage medium supports being accessed by the second reader / writer, and the reading and writing speed of the first reader / writer to the second storage medium is lower than the reading and writing speed of the second reader / writer to the second storage medium; the first data set and the second data set respectively include at least one key-value pair, each of the key-value pairs includes a key value and a value, the key value is used to index the value, the first key value set is a set of at least one key value contained in the first data set, and the second key value set is a set of at least one key value contained in the second data set.

[0007] Whether there is a data conflict or not, the efficiency of the first reader / writer to read and write the data in the first storage medium is relatively high, but the first storage medium is usually a processor memory, and its storage space is limited and cannot store a large amount of data. Therefore, all data cannot be stored in the first storage medium. When there is no data conflict, the second reader / writer reads and writes the data in the second storage medium at a high speed, but when there is a data conflict, the efficiency of the second reader / writer to read and write the data in the second storage medium may be significantly reduced. Therefore, the data that is prone to data conflict is stored in the first storage medium to ensure that when a data conflict occurs, the read / write efficiency is not too low, thereby reducing the efficiency of the entire system; the second data set that is not prone to data conflict is stored in the second storage medium to ensure the efficiency of reading and writing the data in the second data set when no data conflict occurs. Therefore, the data access method provided in the embodiment of the present application, through the reasonable distribution of data, can ensure the read and write efficiency of data regardless of whether there is a data conflict or no data conflict, thereby improving the efficiency of the system and ensuring the timeliness of business processing.

[0008] In a possible implementation, the method further includes: receiving first indication information sent by the memory node, the first indication information being used to indicate a processing method for the target data to be read / written determined according to a third key value set, the third key value set being a set of at least one key value contained in a third data set, the first indication information being sent by the memory node to the computing node after the memory node stores the third data set in the first storage medium and transfers and stores the data stored in the first storage medium except the third data set in the second storage medium; after receiving the first indication information, determining the second target data to be read / written; if the key value of the second target data belongs to the third key value set, sending a third request to the memory node, the third request being used to request the first reader / writer to read / write the second target data in the first storage medium; if the key value of the second target data does not belong to the third key value set, sending a fourth request to the memory node, the fourth request being used to request the second reader / writer to read / write the second target data in the second storage medium.

[0009] As the business progresses, the key-value pairs with a high frequency of concurrent access may change. Therefore, the first data set stored in the first storage medium and the second data set stored in the second storage medium can be updated rather than being immutable. After the first data set and the second data set are updated, the memory node can send the first indication information to the computing node, so that the computing node determines the processing method for the target data to be read / written according to the updated data set.

[0010] In one possible implementation, before receiving the first indication information sent by the memory node, the method also includes: receiving second indication information sent by the memory node, the second indication information being used to indicate a processing method for the target data to be read / written based on the first key value set, the second indication information being sent to the computing node when the third data set has not been completely stored in the first storage medium of the memory node; after receiving the second indication information, determining the third target data to be read / written; if the key value of the third target data belongs to the third key value set and does not belong to the first key value set, suspending the read / write operation on the third target data.

[0011] When the third data set has not been completely stored in the first storage medium of the memory node, the memory node sends a second indication message to instruct the computing node to determine the processing method of the target data to be read / written based on the first key value, so that the computing node does not have to completely interrupt the business and wait for the memory node to store all the data in the third data set to the first storage medium, thereby reducing the impact of the data update process on the business.

[0012] In one possible implementation, after suspending the read / write operation on the third target data, the method further includes: after receiving the third indication information or the first indication information sent by the memory node, sending a fifth request to the memory node, the fifth request being used to request the first reader / writer to read / write the third target data in the first storage medium; the third indication information being used to indicate a processing method for the target data to be read / written based on the first key value set and the third key value set, the third indication information being sent to the computing node when the memory node has not yet stored the data to be transferred in the second storage medium, and the data to be transferred is data stored in the first storage medium other than the third data set.

[0013] After receiving the second indication information, the computing node suspends access to the data whose key value belongs to the third key value set and does not belong to the first key value set. The computing node can resume access to the data whose key value belongs to the third key value set and does not belong to the first key value set after receiving the third indication information or the first indication information. When the computing node receives the first indication information, it means that the storage node has completed storing the data of the third data set in the first storage medium and storing other data in the second storage medium. Therefore, the computing node can access the data whose key value belongs to the third key value set and does not belong to the first key value set according to the third data set. When the computing node receives the third indication information, although the storage node may not have completed storing the data of the third data set in the first storage medium, the memory node can synchronously complete storing the data belonging to the third data set in the first storage medium in the process of responding to the access request of the computing node.

[0014] In a possible implementation, before receiving the first indication information sent by the memory node, the method also includes: receiving third indication information sent by the memory node, the third indication information being used to indicate a processing method for the target data to be read / written determined based on the first key value set and the third key value set, the third indication information being sent to the computing node when the memory node has not yet stored the data to be transferred in the second storage medium, and the data to be transferred is data stored in the first storage medium other than the third data set; after receiving the third indication information, determining fourth target data to be read / written; if the key value of the fourth target data belongs to the third key value set, sending a sixth request to the memory node, the sixth request being used to request the first reader / writer to transfer the data in the first storage medium Read / write the fourth target data in the medium; if the key value of the fourth target data does not belong to the first key value set and does not belong to the third key value set, send a seventh request to the memory node, and the seventh request is used to request the second reader / writer to read / write the fourth target data in the second storage medium; if the key value of the fourth target data belongs to the first key value set and does not belong to the third key value set, and the fourth target data is the target data to be read, send an eighth request to the memory node, and the eighth request is used to request the first reader / writer to read the fourth target data in the first storage medium; if the key value of the fourth target data belongs to the first key value set and does not belong to the third key value set, and the fourth target data is the target data to be written, suspend the write operation on the fourth target data.

[0015] When the memory node has not yet completely stored the data except the third data set in the second storage medium, it sends a second indication message to the computing node, so that the computing node determines the processing method for the target data to be read / written according to the first key value set and the third key value set, so that the computing node does not have to completely interrupt the business and wait for the memory node to store all the data except the third data set in the second storage medium, thereby reducing the impact of the data update process on the business.

[0016] In one possible implementation, after suspending the write operation on the fourth target data, the method further includes: after receiving the first indication information, sending a ninth request to the memory node, and the ninth request is used to request the second reader / writer to write the fourth target data in the second storage medium.

[0017] After the computing node receives the first indication information, it indicates that the memory node has completed storing the third data set in the first storage medium and storing data other than the third data set in the second storage medium. Then the computing node can determine the processing method for the target data to be read / written based only on the third key value, and thus can perform write operations on data whose key value belongs to the first key value set and does not belong to the third key value set.

[0018] In a possible implementation, the first reader / writer is a data processing unit DPU, and / or the second reader is a remote direct memory access RDMA network card.

[0019] In a second aspect, an embodiment of the present application provides a data access method, the method comprising: receiving a first request sent by a computing node, the first request being used to request a first reader / writer of the memory node to read / write first target data in a first storage medium, the first request being sent by the computing node when a key value of the first target data belongs to a first key value set; according to the first request, controlling the first reader / writer to read and write the first target data in the first storage medium; the memory node receiving a second request sent by the computing node, the second request being used to request a second reader / writer of the memory node to read / write the first target data in a second storage medium; the second request being sent by the computing node when the key value of the first target data belongs to a second key value set; according to the second request, controlling the second reader / writer to read and write the first target data in the second storage medium; the first A first data set is stored in a storage medium, and a second data set is stored in a second storage medium. The first data set and the second data set each include at least one key-value pair, and each key-value pair includes a key value and a numerical value, and the key value is used to index the numerical value. The first key value set is a set of at least one key value contained in the first data set, and the second key value set is a set of at least one key value contained in the second data set. The frequency of concurrent reading / writing of data in the first data set is higher than the frequency of concurrent reading / writing of data in the second data set. The first storage medium supports access by the first reader / writer but does not support access by the second reader / writer. The second storage medium supports access by the second reader / writer. The reading and writing speed of the first reader / writer to the second storage medium is lower than the reading and writing speed of the second reader / writer to the second storage medium.

[0020] In a possible implementation, the method further includes: storing a third data set in the first storage medium, and transferring and storing data stored in the first storage medium except the third data set in the second storage medium; sending first indication information to the computing node, the first indication information being used to indicate a processing method for the target data to be read / written according to a third key value set, the third key value set being a set of at least one key value contained in the third data set; after sending the first indication information, receiving a third request sent by the computing node, the third request being used to request the first reader / writer to read / write second target data in the first storage medium, the key value of the second target data belonging to the third key value set; according to the third request, controlling the first reader / writer to read and write the second target data in the first storage medium; after sending the first indication information, receiving a fourth request sent by the computing node, the fourth request being used to request the second reader / writer to read / write second target data in the second storage medium, the key value of the second target data not belonging to the third key value set; according to the fourth request, controlling the second reader / writer to read and write the second target data in the second storage medium.

[0021] In one possible implementation, before sending the first indication information to the computing node, the method also includes: determining first data to be transferred, reading the first data to be transferred from the second storage medium and storing it in the first storage medium, wherein the first data to be transferred is data belonging to the third data set and not stored in the first storage medium; determining second data to be transferred, reading the second data to be transferred from the first storage medium and storing it in the second storage medium, wherein the second data to be transferred belongs to the first data set and is not data belonging to the third data set.

[0022] In one possible implementation, before sending the first indication information to the computing node, the method also includes: when the third data set has not been completely stored in the first storage medium, sending second indication information to the computing node, the second indication information being used to indicate a processing method for the target data to be read / written based on the first key value set, and suspending read / write operations on the target data whose key value belongs to the third key value set and does not belong to the first key value set.

[0023] In a possible implementation, the method also includes: when the second data to be transferred is not stored in the second storage medium, sending third indication information to the computing node, the second data to be transferred is data belonging to the first data set and not belonging to the third data set, and the third indication information is used to indicate the processing method for the target data to be read / written based on the first key value set and the third key value set; after sending the third indication information, receiving a fifth request sent by the computing node, the fifth request is used to request the first reader / writer to read / write the third target data in the first storage medium, the key value of the third target data belongs to the third key value set and not belongs to the first key value set; according to the fifth request, controlling the first reader / writer to read and write the third target data in the first storage medium.

[0024] In one possible implementation, before sending the first indication information to the computing node, the method also includes: when the second data to be transferred is not stored in the second storage medium, sending third indication information to the computing node, the third indication information is used to indicate a processing method for the target data to be read / written based on the first key value set and the third key value set, the second data to be transferred being data that belongs to the first data set and does not belong to the third data set; suspending write operations on the target data whose key value belongs to the first key value set and does not belong to the third key value set.

[0025] In a possible implementation, after sending the third indication information to the computing node, the method further includes at least one of the following: receiving a sixth request sent by the computing node, the sixth request being used to request the first reader / writer to read / write fourth target data in the first storage medium, the key value of the fourth target data belonging to the third key value set; according to the sixth request, controlling the first reader / writer to read / write the fourth target data in the first storage medium; receiving a seventh request sent by the computing node, the seventh request being used to request the second reader / writer to read / write the fourth target data in the second storage medium, the key value of the fourth target data not belonging to the first key value set and not belonging to the third key value set; according to the seventh request, controlling the second reader / writer to read / write the fourth target data in the second storage medium; receiving an eighth request sent by the computing node, the eighth request being used to request the first reader / writer to read the fourth target data in the first storage medium, the key value of the fourth target data belonging to the first key value set and not belonging to the third key value set, and according to the eighth request, controlling the first reader / writer to read the fourth target data in the first storage medium and feeding it back to the computing node.

[0026] In one possible implementation, receiving a sixth request sent by the computing node, wherein the sixth request is used to request the first reader / writer to read / write fourth target data in the first storage medium, includes: receiving the sixth request sent by the computing node; if the sixth request is used to request the first reader / writer to write the fourth target data in the first storage medium, controlling the first reader / writer to write the fourth target data into the first storage medium; if the sixth request is used to request the first reader / writer to read the fourth target data in the first storage medium and the fourth target data is not stored in the first storage medium, controlling the first reader / writer to read the fourth target data from the second storage medium and write it into the first storage medium, and returning the read fourth target data to the computing node.

[0027] In one possible implementation, after suspending the write operation on the target data whose key value belongs to the first key value set and does not belong to the third key value set, the method also includes: receiving a ninth request sent by the computing node, the ninth request being used to request the second reader / writer to write the fourth target data in the second storage medium, the key value of the fourth target data belonging to the first key value set and not belonging to the third key value set; and according to the ninth request, controlling the second reader / writer to write the fourth target data in the second storage medium.

[0028] In a possible implementation, the first reader / writer is a data processing unit DPU, and / or the second reader is a remote direct memory access RDMA network card.

[0029] In a third aspect, an embodiment of the present application provides a data access device, which includes modules / units for executing the above-mentioned first aspect and any possible design method of the first aspect; these modules / units can be implemented by hardware, or corresponding software can be implemented by hardware.

[0030] Exemplarily, the device may include a communication interface and a processing unit; the communication interface is used to send and receive information; the processing unit is used to execute the method described in the first aspect and any possible implementation method of the first aspect through the communication interface.

[0031] In a fourth aspect, an embodiment of the present application provides a data access device, which includes modules / units for executing the above-mentioned second aspect and any possible design method of the second aspect; these modules / units can be implemented by hardware, or corresponding software can be implemented by hardware.

[0032] Exemplarily, the device may include a communication interface and a processing unit; the communication interface is used to send and receive information; the processing unit is used to execute the method described in the second aspect and any two possible implementation methods of the first aspect through the communication interface.

[0033] In a fifth aspect, an embodiment of the present application provides a data access system, comprising: at least one data access device as described in the third aspect and at least one data access device as described in the fourth aspect.

[0034] In a sixth aspect, an embodiment of the present application provides a computer device, comprising: a memory and a processor; the memory stores a computer program; the processor is used to call the computer program stored in the memory to execute the method described in the first aspect and any implementation of the first aspect, or to execute the method described in the second aspect and any implementation of the second aspect.

[0035] In the seventh aspect, a computer-readable storage medium is provided in an embodiment of the present application, wherein the computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed on a computer, the method described in any possible implementation of the first aspect is executed, or the method described in any possible implementation of the second aspect is executed.

[0036] In an eighth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the method described in the first aspect and any possible implementation method to be executed, or enables the method described in any possible implementation method of the second aspect to be executed.

[0037] For the technical effects that can be achieved by any possible implementation method in any of the second to eighth aspects mentioned above, please refer to the description of the technical effects that can be achieved by the corresponding implementation scheme in the first aspect mentioned above, and the repetitions will not be discussed here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of the system architecture used in the embodiments of the present application;

[0039] Figure 2 A flowchart of a data access method provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of data to be transferred provided in an embodiment of the present application;

[0041] Figure 4 A schematic diagram of transferring data to be transferred provided in an embodiment of the present application;

[0042] Figure 5A schematic diagram of storing high-conflict data sets and low-conflict data sets provided in an embodiment of the present application;

[0043] FIG6(a), FIG6(b), FIG6(c), and FIG6(d) are schematic diagrams of updating a high-conflict data set provided in an embodiment of the present application;

[0044] FIG. 7( a ) and FIG. 7( b ) are schematic diagrams of updating a high-conflict data set and a low-conflict data set provided in an embodiment of the present application;

[0045] Figure 8 A schematic diagram of the structure of a data access device provided in an embodiment of the present application;

[0046] Fig. 9 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to build large-scale distributed systems, the architecture of coupled computing and storage effectively solves the problem of system scalability. However, with the development of high-speed network technologies represented by RDMA, the communication bandwidth and latency between nodes have been greatly improved. For example, in a distributed architecture, the bandwidth of accessing remote server memory through RDMA and the bandwidth of accessing local memory are already very close. The bottleneck of the distributed system has shifted from the network to the CPU. The architecture of coupled computing and storage is no longer suitable for new hardware conditions, and the ratio of computing and storage capabilities is mutually constrained. When one resource is insufficient and needs to be expanded, the other resource can only be expanded accordingly, resulting in a waste of resources.

[0048] The computing and storage separation architecture physically separates different hardware resources, which can better solve the problem of independent expansion of computing and storage resources. Due to the technological advances in hardware, networks, and protocols, the cost of accessing data across nodes is getting lower and lower. The computing and storage separation architecture not only means the separation of computing and storage, but also the pooling and separation of memory. In other words, the system architecture has shifted from the past "server-centric" to "resource-centric", with modular and pooled processors, memory, and storage. In the computing and storage separation architecture, the industry currently studies more about system implementation, but the research on memory separation is still insufficient.

[0049] A key issue under the memory separation architecture is how to design a high-performance data indexing method to support computing nodes to access the index structure in the memory node through the RDMA network, modify or retrieve the corresponding key-value pairs (a key-value pair includes a key value and a value, the key value is used to index the value, and the value in the key-value pair is usually a pointer to the real data). Due to the limited computing resources of the memory node, the computing node needs to use the RDMA unilateral primitive for index access and concurrency control. In order to speed up access to the memory node, the memory node can be equipped with DPU hardware, that is, the memory node can include node memory and DPU, such as Figure 1 A DPU usually includes an RDMA network card chip, a programmable processor (such as ARM CPU, FPGA), and a dedicated acceleration engine (such as an encryption engine, a decompression engine), etc.

[0050] When using RDMA unilateral primitives to access the index structure in the memory node, for write operations, in order to ensure the correctness of concurrency, the client thread on the computing node implements a lock-based or lock-free concurrency control method based on RDMA atomic instructions; regardless of the lock-based or lock-free design, when the client thread fails to execute the RDMA atomic instruction, such as a lock failure or a write failure, the client thread needs to retry. For read operations, existing designs mostly use a lock-free method, that is, the client thread accesses the index in the memory node through the unilateral RDMA read primitive, and uses a checksum or version number to ensure the correctness of read and write concurrency. Therefore, in the scenario of data conflict, that is, in the scenario of concurrent data reading and writing, the data access mechanism based on RDMA will cause the RDMA atomic instructions to fail frequently, requiring the client thread to retry frequently, which will not only cause high latency in read and write operations, but also consume a large amount of the limited throughput of the RDMA network card, thereby reducing the efficiency of the entire system.

[0051] When using RPC to access the index structure in the memory node, the client thread sends an RPC request to the memory node. After receiving the RPC request, the DPU in the memory node parses the request and accesses the index structure. However, since the DPU processor in the memory node is usually a low-performance, cheap processor, its processing power is limited in the scenario without data conflict, and its efficiency is much lower than the efficiency of accessing the index structure in the memory node based on the RDMA unilateral primitive.

[0052] In view of this, an embodiment of the present application provides a data access method for overcoming the shortcomings of a single access mechanism, that is, low access efficiency when there is a data conflict or low access efficiency when there is no data conflict, and balancing the access efficiency with and without data conflicts.

[0053] The data access method provided in the embodiment of the present application can be applied to Figure 1 In the system architecture shown in Figure 1As shown, the distributed system may include at least one memory node and at least one computing node. Figure 1 In the example, a memory node and two computing nodes are taken as examples. The memory node may include a DPU and a node memory. Among them, the DPU may include an RDMA network card, a DPU processor, and a DPU memory. Since both the node memory and the DPU memory can be used to store data, in the embodiment of the present application, the DPU memory may be referred to as the first storage medium, and the memory node may be referred to as the second storage medium. Both the RDMA network card and the DPU processor may access the node memory, and perform read and write operations on the data stored in the node memory, and the read / write speed of the RDMA network card for the data in the node memory is higher than the read / write speed of the DPU processor for the data in the node memory. The DPU processor may access the DPU memory, and perform read and write operations on the data stored in the DPU memory; the DPU processor may also have a faster read / write speed for the data in the DPU memory, but since the storage space of the DPU memory is usually small, the data that the DPU memory can store is limited, and not all data can be stored in the DPU memory. Since both the DPU processor and the RDMA network card can read and write, in the embodiment of the present application, the DPU processor may be referred to as the first reader / writer, and the RDMA network card may be referred to as the second reader / writer.

[0054] It should be understood that Figure 1 The example shown is only a specific application scenario that can be applied to the embodiment of the present application, but does not limit the application scenario of the embodiment of the present application. In actual applications, the DPU processor and DPU memory can also be replaced by other types of processors; the node memory can also be implemented through various types of storage media.

[0055] The flowchart of the data access method provided in the embodiment of the present application can be as follows: Figure 2 As shown, the method may include the following steps:

[0056] Step 201: A computing node determines first target data to be read / written.

[0057] When executing business processes, computing nodes often need to access key-value pair data. A key-value pair includes a key value and a value, and the key value is used to index the value. For example, based on the business process, the computing node determines that it currently needs to access the value v1 corresponding to the key value k1. At this time, the first target data determined by the computing node may include the key value k1 that needs to be accessed. For another example, based on the business process, the computing node determines that it currently needs to update the data v2 corresponding to the key value k2, that is, write a new value v2. At this time, the first target data determined by the computing node may include the key value k2 and the updated value v2.

[0058] One or more threads may exist in a computing node. When there are multiple threads, the multiple threads may be used to execute multiple business processes or multiple tasks in one business process, thereby improving the efficiency of the computing node. In this case, a thread of the computing node may determine the first target data to be read / written; at the same time, other threads may also determine the target data to be read / written, and the target data to be read / written determined by other threads may be the same as or different from the first target data.

[0059] In a system where computing and storage are separated, since the computing node may not contain storage resources or only contain limited storage resources, the data that needs to be read / written is usually stored in the memory node. Therefore, the computing node usually needs to read the required data from the memory node or write the updated data to the memory node, that is, the computing node needs to send a read request or a write request for the first target data to the memory node. Specifically, the computing node can determine whether to execute step 202a or step 202b based on the key value of the first target data. If the computing node executes step 202a, the memory node correspondingly executes step 203a; if the computing node executes step 202b, the memory node correspondingly executes step 203b.

[0060] Step 202a: If the key value of the first target data belongs to the first key value set, the computing node sends a first request to the memory node, wherein the first request is used to request a first reader / writer of the memory node to read / write the first target data in a first storage medium of the memory node.

[0061] The first storage medium of the memory node supports access (i.e., read / write) by the first reader / writer of the memory node, but does not support access by the second reader / writer of the memory node. For example, the first storage medium can be Figure 1 The DPU memory shown, the first reader can be Figure 1 The DPU processor shown, the second reader can be Figure 1 For the RDMA NIC shown, the DPU memory can be accessed by the DPU processor but cannot be directly accessed by the RDMA NIC.

[0062] A first data set is stored in a first storage medium, and the first data set includes at least one key-value pair. The first key-value set is a set of at least one key value contained in the first data set. For example, if the first data set includes key-value pairs (k1, v1), (k2, v2), and (k3, v3), then the first key-value set includes k1, k2, and k3.

[0063] After executing step 201, that is, after determining the first target data, the computing node can determine whether the key value of the first target data belongs to the first key value set before executing step 202a or step 202b, which is equivalent to determining whether the first target data is stored in the first storage medium of the memory node. If the key value of the first target data belongs to the first key value set, that is, the first target data is stored in the first storage medium, then the computing node can send a first request to the memory node to request the first reader / writer of the memory node to read / write the first target data in the first storage medium.

[0064] In a specific example, the computing node can generate a filter based on the first key value set, the input of the filter is the key value, and the output is "true" or "false", which respectively indicates that the first target data is stored in the first storage medium or the second storage medium of the memory node.

[0065] Optionally, the first request may be a read / write request based on an RPC mechanism.

[0066] Step 203a: The memory node controls the first reader / writer to read / write the first target data in the first storage medium according to the first request.

[0067] If the first request is used to read the first target data, the memory node controls the first reader to read the first target data from the first storage medium. For example, if the first request is used to request to read the value corresponding to the key value k1, the first reader reads the value v1 corresponding to the key value k1 from the first storage medium. If the first request is used to write the value v2' to the position corresponding to the key value k2, the first reader writes the value v2' to the position corresponding to the key value k2 in the first storage medium, that is, the original key-value pair (k2, v2) is updated to (k2, v2').

[0068] by Figure 1 Taking the application scenario shown as an example, the above step 203a is: the RDMA network card in the memory node receives the first request, transmits the first request to the DPU processor, and the DPU processor reads / writes the first target data in the DPU memory according to the first request.

[0069] Step 202b: If the key value of the first target data belongs to the second key value set, the computing node sends a second request to the memory node, wherein the second request is used to request the second reader / writer of the memory node to read / write the first target data in the second storage medium of the memory node.

[0070] The second storage medium of the memory node supports being accessed by the second reader / writer (i.e., read / write). In addition, the second storage medium also supports being accessed by the first reader / writer. However, the speed at which the second storage medium is read and written by the first reader / writer is lower than the speed at which the second storage medium is read and written by the second reader / writer. For example, the second storage medium can be Figure 1 The memory node shown, the second storage medium can be Figure 1 The RDMA network card (i.e., the second reader / writer) shown in the figure can also be directly accessed by Figure 1 The DPU processor directly accesses the memory node, but the reading and writing speed of the RDMA network card to the memory node is higher than the reading and writing speed of the DPU processor to the memory node.

[0071] The second storage medium stores a second data set, and the second data set includes at least one key-value pair. The second key-value set is a set of at least one key value included in the second data set. For example, if the second data set includes key-value pairs (k4, v4), (k5, v5), and (k6, v6), then the second key-value set includes k4, k5, and k6.

[0072] After the computing node executes step 201, that is, after determining the first target data, if it is determined that the key value of the first target data belongs to the second key value set, which is equivalent to determining that the first target data is stored in the second storage medium, then the computing node can send a second request to the memory node to request the second reader / writer of the memory node to read / write the first target data in the second storage medium.

[0073] Optionally, the second request may be a read / write request based on an RDMA mechanism.

[0074] Step 203b: The memory node controls the second reader / writer to read / write the first target data in the second storage medium according to the second request.

[0075] If the second request is used to read the first target data, the memory node controls the second reader to read the first target data from the second storage medium. For example, if the second request is used to request to read the value corresponding to the key value k4, the second reader reads the value v4 corresponding to the key value k4 from the second storage medium. If the first request is used to write the value v5' to the position corresponding to the key value k5, the second reader writes the value v5' to the position corresponding to the key value k5 in the second storage medium, that is, the original key-value pair (k5, v5) is updated to (k5, v5').

[0076] by Figure 1 Taking the application scenario shown as an example, the above step 203b is: the RDMA network card in the memory node receives the second request, and the RDMA network card reads / writes the first target data in the memory node according to the second request.

[0077] exist Figure 2In the illustrated method embodiment, the frequency of concurrent reading / writing of data in the first data set is higher than the frequency of concurrent reading / writing of data in the second data set. Since the frequency of concurrent reading / writing of data in the first data set is higher, the frequency of concurrent reading / writing of data in the first data set by multiple threads (which may be threads in different computing nodes or multiple threads in the same computing node) is higher, that is, data conflicts are more likely to occur in the data in the first data set.

[0078] Whether there is a data conflict or not, the efficiency of the first reader / writer in reading and writing data in the first storage medium is relatively high. When there is no data conflict, the efficiency of the first reader / writer in reading and writing data in the first storage medium is relatively high; when there is a data conflict, although the reading and writing efficiency is lower than when there is no data conflict, it is significantly higher than the efficiency of the second reader / writer in reading and writing data in the second storage medium when there is a data conflict. However, the first storage medium is usually a processor memory, which has limited storage space and cannot store a large amount of data. Therefore, all data cannot be stored in the first storage medium, so data that is prone to data conflicts is stored in the first storage medium to ensure that when a data conflict occurs, the read / write efficiency is not too low, thereby reducing the efficiency of the entire system.

[0079] When there is no data conflict, the second reader / writer reads and writes data in the second storage medium at a high speed. However, when there is a data conflict, the efficiency of the second reader / writer reading and writing data in the second storage medium may be significantly reduced. Therefore, the second data set that is not prone to data conflict is stored in the second storage medium to ensure the efficiency of reading and writing data in the second data set when there is no data conflict.

[0080] In summary, the first data set that is prone to data conflicts is stored in the first storage medium, and the first data set is read and written by the first reader / writer, and the second data set that is not prone to data conflicts is stored in the second storage medium, and the second data set is read and written by the second reader / writer. This can ensure the efficiency of reading and writing the data of the second data set when there is no data conflict, and can also ensure the efficiency of reading and writing the data of the first data set when there is or is not a data conflict. Therefore, the data access method provided in the embodiment of the present application, through the reasonable distribution of data, utilizes the reading and writing characteristics of the first reader / writer in the memory node to the first storage medium, and the reading and writing characteristics of the second reader / writer to the second storage medium, so as to ensure the reading and writing efficiency of data regardless of whether there is a data conflict or not, thereby improving the efficiency of the system and further ensuring the timeliness of business processing.

[0081] The key-value pairs contained in the first data set are key-value pairs with a high frequency of concurrent access. However, as the business proceeds, the key-value pairs with a high frequency of concurrent access may change. Therefore, the first data set stored in the first storage medium and the second data set stored in the second storage medium can be updated rather than unchanged. In a specific example, the statistical node can obtain access information for a period of time or a certain number of times in the past, determine the number of times each key-value pair (including the key-value pairs in the first data set and the key-value pairs in the second data set) has been accessed, or determine the number of times each key-value pair has been accessed concurrently, thereby determining the key-value pairs that may be accessed concurrently with a high frequency. These key-value pairs that may be accessed concurrently with a high frequency constitute the third data set; the set of key values ​​contained in the third data set is the third key-value set. The third data set is the data set that will replace the first data set and be stored in the first storage medium. Data other than the third data set (which may constitute the fourth data set) will be stored in the second storage medium.

[0082] The statistical node may be a memory node or a computing node, a node in the computing and storage separation system, or a node outside the system. The embodiment of the present application does not limit this.

[0083] After determining the third key value set, the statistical node can send the third key value set to the memory node and each computing node. When the statistical node is a memory node, the memory node sends the determined third key value set to each computing node; when the statistical node is a computing node, the computing node sends the determined third key value set to other computing nodes and the memory node.

[0084] After receiving the third key value set, the memory node stores the third data set corresponding to the third key value set in the first storage medium, and transfers and stores the data stored in the first storage medium except the third data set in the second storage medium, so that the first storage medium stores the third data set and the second storage medium stores the fourth data set. After completing the above data transfer operation, the memory node can send first indication information to the computing node, which is used to instruct the computing node to determine the processing method for the target data to be read / written according to the third key value set.

[0085] In a possible implementation, when the memory node performs data transfer, the first data to be transferred and the second data to be transferred may be determined first. The first data to be transferred is data belonging to the third data set and stored in the first storage medium, that is, data belonging to the third data set but not belonging to the first data set, such as Figure 3 As shown; the second data to be transferred is data belonging to the first data set and not belonging to the third data set, such as Figure 3The memory node reads the first data to be transferred from the second storage medium and stores it in the first storage medium, and reads the second data to be transferred from the first storage medium and stores it in the second storage medium, as shown. Figure 4 shown.

[0086] After receiving the third key value set, the computing node does not immediately determine the processing method for the target data to be read / written according to the third key value set, but determines the processing method for the target data to be read / written according to the third key value set after receiving the first indication information. After receiving the first indication information, the computing node determines that the second target data needs to be read / written at present. If the key value of the second target data belongs to the third key value set, the computing node sends a third request to the memory node to request the first reader / writer of the memory node to read / write the second target data in the first storage medium; the memory node controls the first reader / writer to read / write the second target data in the first storage medium. If the key value of the second target data does not belong to the third key value set, a fourth request is sent to the memory node to request the second reader / writer of the memory node to read / write the second target data in the second storage medium; the memory node controls the second reader / writer to read / write the second target data in the second storage medium.

[0087] Optionally, the computing node may also generate a new filter based on the third key value set, and after receiving the first indication information, enable the new filter to determine whether the second target data to be read / written is stored in the first storage medium, that is, to determine whether the computing node should generate a third request or a fourth request.

[0088] Since the time when each computing node and memory node receives the third key value set may be different, the memory node still needs a certain amount of time to perform the data transfer operation. If the computing node immediately determines the processing method of the target data to be read / written according to the third key value set after receiving the third key value set, then the memory node may not have completed the data transfer, and the data reading and writing may not be completed or a reading and writing error may occur. In the embodiment of the present application, the memory node sends the first indication information to the computing node after completing the data transfer, and each computing node determines the processing method of the target data to be read / written according to the third key value set after receiving the first indication information, thereby avoiding the occurrence of data reading and writing failures or reading and writing errors.

[0089] In a possible implementation, when the first storage medium has not yet completely stored the third data set, the memory node may send a second indication message to the computing node to instruct the computing node to still determine the processing method for the target data to be read / written based on the first key value set, and to suspend the read / write operation for the target data whose key value belongs to the third key value set and does not belong to the first key value set. After receiving the second indication message, if the computing node determines that the key value of the third target data currently to be read / written belongs to the third key value set and does not belong to the first key value set, it will not generate an access request to the third target data for the time being.

[0090] When the memory node has not completely transferred the first data to be transferred to the first storage medium, the memory node sends a second indication message to instruct the computing node to determine the processing method for the target data to be read / written based on the first key value, so that the computing node does not have to completely interrupt the business and wait for the memory node to complete the transfer of the first data to be transferred and / or the second data to be transferred, thereby reducing the impact of the data update process on the business.

[0091] Since there may be a delay between the time when the memory node sends the second indication information and the time when each node receives and responds to the second indication information, it is possible that the memory node has sent the second indication information but still receives a read / write request from the computing node for target data that belongs to the third key value set and does not belong to the first key value set. The memory node will suspend the read / write operation of the target data that belongs to the third key value set and does not belong to the first key value set. Furthermore, the memory node can also return an "error" response message to the computing node.

[0092] Optionally, after receiving the first indication information or the third indication information, the computing node may generate a read / write request for target data whose key value belongs to the third key value set and does not belong to the first key value set. After the memory node sends the first indication information or sends the third indication information, the memory node may also respond to the read / write request for target data whose key value belongs to the third key value set and does not belong to the first key value set. Among them, the third indication information is used to indicate the processing method for the target data to be read / written based on the first key value set and the third key value set. The role of the third indication information will be explained in detail later.

[0093] Optionally, after receiving the third key value set, the computing node may send a message to the memory node to indicate that it has received the third key value set. The memory node may send the first indication information to the computing node after each computing node has received the third key value set, so as to avoid the situation where the computing node has not yet received the third key value set and is unable to suspend read / write operations on target data whose key values ​​belong to the third key value set and do not belong to the first key value set.

[0094] In one possible implementation, when the memory node does not store the second data to be transferred in the second storage medium, the memory node may send third indication information to the computing node to instruct the computing node to determine a processing method for the target data to be read / written based on the first key value set and the third key value set.

[0095] Determining a processing method for the target data to be read / written according to the first key value set and the third key value set may include:

[0096] 1. If the key value of the fourth target data that needs to be read / written currently belongs to the third key value set, the computing node sends a sixth request to the memory node, requesting the first reader / writer of the memory node to read / write the fourth target data in the first storage medium. Correspondingly, after receiving the sixth request, the memory node controls the first reader / writer to read / write the fourth target data in the first storage medium.

[0097] Since the memory node has not completed the data transfer at this time, the fourth target data may be stored in the first storage medium, or the fourth target data may not be stored in the first storage medium.

[0098] Optionally, if the sixth request is used to request writing of the fourth target data, the memory node may control the first reader to write the fourth target data into the first storage medium after receiving the sixth request. Regardless of whether the fourth target data is already stored in the first storage medium, the memory node may write the fourth target data into the first storage medium.

[0099] If the sixth request is for reading the fourth target data, the first reader / writer may first determine whether the fourth target data is stored in the first storage medium. If it is stored, the stored fourth target data is read and the fourth target data is returned to the computing node; if the fourth target data is not stored in the first storage medium, the first reader / writer may read the fourth target data from the second storage medium, then write the read fourth target data into the first storage medium, and return the fourth target data to the computing node.

[0100] 2. If the key value of the fourth target data that needs to be read / written does not belong to the first key value set or the third key value set, the computing node sends a seventh request to the memory node to request the second reader / writer of the memory node to read / write the fourth target data in the second storage medium. If the key value of the fourth target data does not belong to the first key value set or the third key value set, it means that the fourth target data was stored in the second storage medium before being updated, and will still be stored in the second storage medium after being updated, so the computing node requests the second reader / writer to read / write the fourth target data in the second storage medium. Accordingly, after receiving the seventh request, the memory node controls the second reader / writer to read / write the fourth target data in the second storage medium.

[0101] 3. If the key value of the fourth target data that needs to be read / written currently belongs to the first key value set and does not belong to the third key value set, and the fourth target data is the target data to be read, the computing node sends an eighth request to the memory node to request the first reader / writer of the memory node to read the fourth target data in the first storage medium. If the key value of the fourth target data belongs to the first key value set and does not belong to the third key value set, it means that the fourth target data was stored in the first storage medium before the update, and should be stored in the second storage medium after the update. However, since the memory node has not completely stored the second data to be transferred in the second storage medium at this time, the fourth target data may not be stored in the second storage medium, and the fourth target data is still stored in the first storage medium, so the computing node requests the first reader / writer to read the fourth target data in the first storage medium. Accordingly, after receiving the eighth request, the memory node controls the first reader / writer to read the fourth target data in the first storage medium.

[0102] 4. If the key value of the fourth target data currently to be read / written belongs to the first key value set and does not belong to the third key value set, and the fourth target data is the target data to be written, suspend the write operation on the fourth target data.

[0103] Since the memory node has not yet completely stored the second data to be transferred in the second storage medium at this time, the fourth target data may or may not be stored in the second storage medium. If the fourth target data is allowed to be written in the first storage medium, the fourth target data that has been transferred to the second storage medium becomes erroneous data because no new data has been written; if the fourth target data is allowed to be written in the second storage medium, the fourth target data read from the first storage medium becomes erroneous data because no new data has been written. Therefore, when the memory node has not yet completely stored the second data to be transferred in the second storage medium, the write operation on the target data whose key value belongs to the first key value set and does not belong to the third key value set is suspended.

[0104] After the memory node sends the third indication information and before sending the first indication information, if the memory node receives a write request from the computing node for target data that belongs to the first key value set and does not belong to the third key value set, the write operation on the target data that belongs to the first key value set and does not belong to the third key value set will be suspended. Since there may be a delay between the time when the memory node sends the third indication information and the time when each node receives and responds to the third indication information, it is possible that the memory node has sent the third indication information but still receives a write request from the computing node for target data that belongs to the first key value set and does not belong to the third key value set, and the memory node will suspend the write operation. Furthermore, the memory node can also return an "error" response message to the computing node.

[0105] When the memory node sends the first indication information to the computing node, it indicates that the memory node has completed the data transfer and realized the storage of the third data set in the first storage medium. The data other than the third data set is stored in the second storage medium. At this time, if the computing node determines that the key value of the target data currently to be written belongs to the first key value set and does not belong to the third key value set, the computing node can send a ninth request to the memory node after determining that the key value of the target data does not belong to the third key value set, requesting the second reader / writer of the memory node to write the target data in the second storage medium. After sending the first indication information, the memory node can also respond to the ninth request and write the target data whose key value does not belong to the third key value set in the second storage medium.

[0106] It should be understood that if the memory node sends the first indication information, the second indication information, and the third indication information to the computing node during the process of executing data transfer, the memory node will first send the second indication information after receiving the third key value set, and then send the third indication information to the computing node before the second data to be transferred is stored in the second storage medium. After the first data to be transferred and the second data to be transferred are all transferred, the first indication information will be sent to the computing node.

[0107] Alternatively, the memory node may also send the second indication information first after receiving the third key value set, and send the first indication information to the computing node after the first data to be transferred and the second data to be transferred are all transferred.

[0108] Alternatively, the memory node may also send a third indication message to the computing node after receiving the third key value set and before storing the second data to be transferred in the second storage medium, and send the first indication message to the computing node after the first data to be transferred and the second data to be transferred are all transferred.

[0109] After the computing node and the memory node receive the third key value set, they do not immediately determine the processing method for the target data according to the third key value set, but wait for the memory node to send a corresponding instruction according to the data storage state, so that the computing node can determine whether to still determine the processing method according to the first key value set, or to determine the processing method according to the first key value set and the third key value set, or to determine the processing method according to the third key value set. In addition, during the process of the memory node executing data transfer, the business impact on the computing node is relatively low, and the computing node's read and write operations on the data are not completely suspended, so that some of the computing node's read and write operations can still be carried out normally, thereby ensuring the system efficiency during the data transfer process.

[0110] In order to understand the above embodiments more clearly, Figure 5 , Figure 6(a) to Figure 6(d) Give an example.

[0111] The key-value pairs stored in the memory node include: (k1, v1), (k2, v2), (k3, v3), (k4, v4), (k5, v5), (k6, v6), (k7, v7), (k8, v8), (k9, v9). Among them, (k1, v1), (k2, v2), (k3, v3) are key-value pairs that are often read / written concurrently by multiple threads. Therefore, the high-conflict data set (i.e., the first data set) containing (k1, v1), (k2, v2), (k3, v3) is stored in the DPU memory of the memory node (i.e., the first storage medium of the memory node); the low-conflict data set (i.e., the second data set) consisting of (k4, v4), (k5, v5), (k6, v6), (k7, v7), (k8, v8), (k9, v9) is stored in the memory node of the memory node (i.e., the second storage medium of the memory node), as shown in FIG. Figure 5 shown.

[0112] Computing nodes 1 and 2 are both provided with conflict filters, which are used to determine whether the target data to be read / written is stored in the DPU memory or in the node memory, that is, to determine whether the target data belongs to a high-conflict data set or a low-conflict data set. In a specific embodiment, the input of the conflict filter is the key value of the target data, and the output is "true" or "false", which respectively indicates that the target data is stored in the DPU memory (the target data belongs to the high-conflict data set) or in the node memory (the target data belongs to the low-conflict data set). For example, when thread 1 of computing node 1 wants to read the value v1, thread 1 inputs the key value k1 corresponding to the value v1 into the conflict filter, and the conflict filter outputs "true" to thread 1; if thread 2 of computing node 1 wants to read the value v4, thread 2 inputs the key value k4 corresponding to the value v4 into the conflict filter, and the conflict filter outputs "false" to thread 2.

[0113] When thread 1 of computing node 1 wants to update the value v2 in the key-value pair (k2, v2) to v2', that is, to write the value v2, thread 1 of computing node 1 inputs the key value k2 corresponding to the value v2 into the conflict filter of computing node 1, and the conflict filter of computing node 1 outputs "true" to thread 1 of computing node 1, indicating that the key-value pair (k2, v2) belongs to the high-conflict data set and is stored in the DPU memory of the memory node. Therefore, thread 1 of computing node 1 generates an RPC write request for the key-value pair (k2, v2), which includes the key value k2 and the value v2', and sends the RPC write request to the memory node to request that the value v2' be written to the position corresponding to the key value k2. Thread 1 of the DPU processor of the memory node updates the key-value pair (k2, v2) in the high-conflict set stored in the DPU memory to the key-value pair (k2, v2') according to the received RPC write request. When updating v2 to v2', thread 1 of the DPU processor can first obtain the write lock of v2, that is, obtain the right to write to v2. When thread 1 of the DPU processor obtains the write lock of v2, other threads will not be able to read or write to v2.

[0114] When thread 1 of compute node 2 wants to update the value v4 in the key-value pair (k4, v4) to v4', that is, to write the value v4, thread 1 of compute node 2 inputs the key value k4 corresponding to the value v4 into the conflict filter of compute node 2, and the conflict filter of compute node 2 outputs "false" to thread 1 of compute node 2, indicating that the key-value pair (k4, v4) belongs to the low-conflict data set and is stored in the node memory. Therefore, thread 1 of compute node 2 generates an RDMA write request for the key-value pair (k4, v4), which includes the key value k4 and the value v4', and sends the RDMA write request to the memory node to request that the value v4' be written to the position corresponding to the key value k4. The RDMA network card of the memory node can directly respond to the RDMA write request and update the key-value pair (k4, v4) in the low-conflict set stored in the node memory to the key-value pair (k4, v4') without being executed by the DPU processor. When thread 1 of the RDMA network card of the memory node updates v4 to v4', it can first obtain the write lock of v4, that is, obtain the right to write to v4; after obtaining the write lock of v4, it updates it. At this time, other threads will not be able to read or write to v4.

[0115] When thread 2 of compute node 1 wants to read the value v5 in the key-value pair (k5, v5), that is, to perform a read operation on the value v5, thread 2 of compute node 1 inputs the key value k5 corresponding to the value v5 into the conflict filter of compute node 1, and the conflict filter of compute node 1 outputs "false" to thread 2 of compute node 1, indicating that the key-value pair (k5, v5) belongs to the low-conflict data set and is stored in the node memory. Therefore, thread 2 of compute node 1 generates an RDMA read request for the key-value pair (k5, v5) and sends the RDMA read request to the memory node to request to read the value v5. The RDMA network card of the memory node can directly respond to the RDMA read request and read the value v5 from the low-conflict data set in the node memory without being executed by the DPU processor. When reading the value v5, thread 2 of the RDMA network card of the memory node can first obtain the read lock of v5, that is, obtain the right to read v5. At this time, other threads will not be able to write to v5.

[0116] When thread 2 of compute node 2 wants to read the value v3 in the key-value pair (k3, v3), that is, to perform a read operation on the value v3, thread 2 of compute node 2 inputs the key value k3 corresponding to the value v3 into the conflict filter of compute node 2, and the conflict filter of compute node 2 outputs "true" to thread 2 of compute node 2, indicating that the key-value pair (k3, v3) belongs to the high-conflict data set and is stored in the DPU memory of the memory node. Therefore, thread 2 of compute node 2 generates an RPC read request for the key-value pair (k3, v3) and sends the RPC read request to the memory node to request to read the value v3. Thread 2 of the DPU processor of the memory node reads the value v3 from the high-conflict data set in the DPU memory according to the received RPC read request. When reading the value v3, thread 2 of the DPU processor can first obtain the read lock for v3, that is, obtain the right to read v3. At this time, other threads will not be able to write to v3.

[0117] After a period of time, since each computing node often makes concurrent write requests for the values ​​corresponding to key values ​​k2, k3, and k4, the statistical node determines that the key value set included in the new high-conflict data set is {k2, k3, k4}, so the new high-conflict data set (i.e., the third data set mentioned above) should include (k2, v2), (k3, v3), and (k4, v4). Among them, (k4, v4) that originally belonged to the low-conflict data set belongs to the high-conflict data set after the update, and (k1, v1) that originally belonged to the high-conflict data set belongs to the low-conflict data set after the update.

[0118] The above-mentioned statistical node is a node used to collect statistics on high-conflict data, and can be any computing node, a memory node, or other nodes. The following takes a memory node as a statistical node as an example for explanation.

[0119] After determining the new high-conflict data set, the memory node can allocate storage space for the new high-conflict data set in the DPU memory, and write the key values ​​k2, k3, and k4 contained in the new high-conflict set into the allocated storage space. However, the value corresponding to the key value is "empty" at this time, waiting for subsequent writing, as shown in Figure 6(a).

[0120] The memory node sends the key value set {k2, k3, k4} contained in the new high-conflict set to each computing node. Computing node 1 and computing node 2 generate new conflict filters based on the key value set contained in the received new high-conflict set, but each thread does not use the new conflict filter immediately. Instead, it determines whether to enable the new conflict filter based on the instruction information of the memory node. Each computing node can maintain the synchronization variable "Stage" and determine whether to use the new conflict filter or the old conflict filter based on the value of the synchronization variable "Stage". Each thread of the computing node can read the value of the synchronization variable "Stage" before performing each read / write operation on the target data to determine whether to use the new conflict filter.

[0121] After sending the key value set {k2, k3, k4} contained in the new high-conflict set to each computing node, the memory node can send the synchronization information LazySync ("Stage", 2) (i.e., the second indication information mentioned above) to each computing node, indicating that the current value of the synchronization variable "Stage" is 2. When the thread of the computing node reads the value of the synchronization variable "Stage" as 2, it is determined that the old conflict filter is still used, and according to the key value set contained in the old high-conflict data set and the key value set contained in the new high-conflict data set, the key value that belongs to the new high-conflict set and does not belong to the old high-conflict set is determined, i.e., k4. If the key value of the target data that thread 1 of computing node 1 is currently to perform a read / write operation on is k4, a read / write request (whether an RPC request or an RDMA request) is not generated temporarily; if the key value of the target data that thread 2 of computing node 1 is currently to perform a read / write operation on is not k4, the key value of the target data is input into the old conflict filter of computing node 1, and a corresponding request is generated according to the output of the old conflict filter and sent to the memory node. When all threads of a computing node read that the value of the synchronization variable "Stage" is 2, the computing node can send a second response to the memory node, indicating that the value of the synchronization variable "Stage" of the computing node has been updated.

[0122] For a memory node, if an RPC request with a key value of k4 is received from a compute node when the value of the synchronization variable "Stage" is 2, an "error" is returned to the compute node.

[0123] After receiving the second response sent by each computing node, the memory node can send the synchronization information LazySync ("Stage", 3) (i.e., the third indication information mentioned above) to each node, indicating that the current value of the synchronization variable "Stage" is 3. When the thread of the computing node reads the value of the synchronization variable "Stage" as 3, it is determined to use a new conflict filter, and according to the key value set contained in the old high-conflict data set and the key value set contained in the new high-conflict data set, the key value that belongs to the old high-conflict set and does not belong to the new high-conflict set is determined, i.e., k1, and the new conflict filter is not applicable to the target data with the key value of k1. If the key value of the target data that thread 1 of computing node 1 is currently to be written is k1, a write request (whether it is an RPC write request or an RDMA write request) is not generated for the time being; if the key value of the target data that thread 2 of computing node 1 is currently to be read / written is not k1, the key value of the target data is input into the new conflict filter of computing node 1, and a corresponding request is generated according to the output of the new conflict filter and sent to the memory node. Optionally, if the key value of the target data to be read by thread 1 of computing node 2 is k1, a corresponding RPC read request is generated and sent to the memory node. Alternatively, a read request may not be generated temporarily, and the memory node may wait for the memory node to send the synchronization information LazySync("Stage", 4) before performing a read operation on the target data with the key value k1. When all threads of a computing node read the value of the synchronization variable "Stage" as 3, the computing node may send a third response to the memory node, indicating that the value of the synchronization variable "Stage" of the computing node has been updated.

[0124] After the memory node sends the synchronization message LazySync("Stage", 3), before receiving the third response sent by all computing nodes:

[0125] If an RPC write request with a key value of k1 is received, the memory node returns an "error" to the computing node.

[0126] If an RDMA read request with a key value of k1 is received, the memory node returns an "error" to the compute node.

[0127] If an RPC write request with a key value of k2 (or k3, or k4) is received, the memory node DPU processor writes the value into the storage space allocated for the new high-conflict set according to the write request, as shown in Figure 6(b).

[0128] If an RPC read request with a key value of k4 is received, and there is no value with a key value of k4 in the storage space allocated for the new high-conflict set in the DPU memory, the DPU processor reads the value v4 corresponding to the key value k4 from the node memory, writes the value v4 to the storage space allocated for the new high-conflict set in the DPU memory, as shown in Figure 6(c), and returns the value v4 to the computing node.

[0129] If an RPC read request with a key value of k2 (or k3) is received, and there is no value with a key value of k2 (or k3) in the storage space allocated for the new high-conflict set in the DPU memory, the memory node DPU processor reads the value v2 (or v3) corresponding to the key value k2 (or k3) from the old high-conflict set in the DPU memory, writes the value v2 (or v3) to the storage space allocated for the new high-conflict set in the DPU memory, as shown in Figure 6(d), and returns the value v2 (or v3) to the computing node.

[0130] After receiving the third response sent by each computing node, the DPU processor can read the data in the DPU memory or the node memory to complete the still empty values ​​in the storage space allocated for the new high-conflict set in the DPU memory; write (k1, v1) that belongs to the old high-conflict set but not the new high-conflict set into the node memory, as shown in Figure 7(a). After the memory node DPU processor performs the above operations, it can release the storage space storing the old high-conflict set in the DPU memory, as shown in Figure 7(b), and send the synchronization information LazySync("Stage", 4) (i.e. the above first indication information) to each node, indicating that the current value of the synchronization variable "Stage" is 4.

[0131] When the thread of the computing node reads the value of the synchronization variable "Stage" as 4, it is determined to use the new conflict filter, and it is not necessary to suspend the read / write request for the target data according to the old high conflict set. When all threads in a computing node read the value of the synchronization variable "Stage" as 4, the computing node can release the storage space occupied by the old conflict filter.

[0132] In the above embodiment, after generating a new conflict filter, each computing node does not immediately adopt the new conflict filter, but determines whether to adopt the new conflict filter and whether it is necessary to suspend the read / write operation of part of the data based on the instruction information sent by the memory node. In this way, in the process of updating the high-conflict set, it is not necessary to suspend access to all data, and errors that are prone to occur in the process of writing and reading data due to the different response speeds of each computing node and each thread to the synchronization information will not occur.

[0133] Based on the same technical concept, the embodiment of the present application also provides a data access device, including a module / unit that performs the computing node or memory node function in the above method embodiment. The device can be a computing node / memory node, or a component of a computing node / memory node (such as a processor, chip, or chip system, etc.), or a logical node, logical module or software that can implement all or part of the network device function. These modules / units can be implemented by hardware, or they can be implemented by hardware executing corresponding software.

[0134] Exemplarily, the data access device may be as follows: Figure 8 As shown, it includes an interface module 801 and a processing module 802. The interface module 801 is used for sending and receiving messages, and the processing module 802 is used to implement the processing of messages by the device. It should be understood that the processing module 802 in the embodiment of the present application can be implemented by a processor or a processor-related circuit component (or, referred to as a processing circuit), and the interface module 801 can be implemented by a transceiver or a transceiver-related circuit component.

[0135] When the data access device is a computing node, the processing module 802 is used to determine the first target data to be read / written; if the key value of the first target data belongs to the first key value set, the interface module 801 sends a first request to the memory node, and the first request is used to request the first reader / writer of the memory node to read / write the first target data in the first storage medium, and the first storage medium stores a first data set; if the key value of the first target data belongs to the second key value set, the interface module 801 sends a second request to the memory node, and the second request is used to request the second reader / writer of the memory node to read / write the first target data in the second storage medium, and the second storage medium stores a second data set; the data in the first data set is The frequency of concurrent reading / writing is higher than the frequency of concurrent reading / writing of data in the second data set, the first storage medium supports access by the first reader / writer but does not support access by the second reader / writer, the second storage medium supports access by the second reader / writer, and the reading and writing speed of the first reader / writer to the second storage medium is lower than the reading and writing speed of the second reader / writer to the second storage medium; the first data set and the second data set respectively include at least one key-value pair, each of the key-value pairs includes a key value and a value, the key value is used to index the value, the first key value set is a set of at least one key value contained in the first data set, and the second key value set is a set of at least one key value contained in the second data set.

[0136] In addition, the above modules can also be used to support other processes executed by the computing nodes in any of the above embodiments. The beneficial effects can be referred to the above description, which will not be repeated here.

[0137] When the data access device is a memory node, the interface module 801 is used to receive a first request sent by a computing node, wherein the first request is used to request a first reader / writer of the memory node to read / write a first target data in a first storage medium, and the first request is sent by the computing node when the key value of the first target data belongs to a first key value set; the processing module 802 is used to control the first reader / writer to read and write the first target data in the first storage medium according to the first request; the interface module 801 is also used to receive a second request sent by the computing node, wherein the second request is used to request a second reader / writer of the memory node to read / write the first target data in a second storage medium; the second request is sent by the computing node when the key value of the first target data belongs to a second key value set; the processing module 802 is also used to control the second reader / writer to read and write in the second storage medium according to the second request. The first target data; a first data set is stored in the first storage medium, and a second data set is stored in the second storage medium, the first data set and the second data set respectively include at least one key-value pair, one key-value pair includes a key value and a numerical value, and the key value is used to index the numerical value; the first key value set is a set of at least one key value contained in the first data set, and the second key value set is a set of at least one key value contained in the second data set; the frequency of concurrent reading / writing of data in the first data set is higher than the frequency of concurrent reading / writing of data in the second data set, the first storage medium supports access by the first reader / writer but does not support access by the second reader / writer, the second storage medium supports access by the second reader / writer, and the reading and writing speed of the first reader / writer to the second storage medium is lower than the reading and writing speed of the second reader / writer to the second storage medium.

[0138] In addition, the above modules can also be used to support Figure 2 to Figure 7(b) The other processes and beneficial effects performed by the memory node in the illustrated embodiment can be referred to the previous description and will not be described in detail here.

[0139] The present application also provides a computer device. The computer device can be Fig. 9 As shown, it includes a processor 901 and a memory 902. It may further include a communication interface 903 and a communication bus 904.

[0140] The processor 901 may be a general processor 901, a microprocessor 901, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or one or more integrated circuits for controlling the execution of the program of the present application. The general processor 901 may be a microprocessor 901 or any conventional processor 901. The steps of the method disclosed in the embodiment of the present application may be directly embodied as being executed by the hardware processor 901, or may be executed by a combination of hardware and software modules in the processor 901.

[0141] The memory 902 is used to store program instructions and / or data so that the processor 901 can call the instructions and / or data stored in the memory 902 to implement the above functions of the processor 901. The memory 902 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM) or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 902 can be independent, such as an off-chip memory, connected to the processor 901 via a communication bus 904. The memory 902 can also be integrated with the processor 901.

[0142] The communication interface 903 is used to communicate with other devices, such as a PCI bus interface, Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0143] In an embodiment of the present application, the processor 901 is used to call the communication interface 903 to perform the receiving and / or sending functions, and execute the method described in any of the previous possible implementation methods.

[0144] Communication bus 904 may include a pathway for transmitting information between the above-mentioned components.

[0145] The computer device may be a computing node in the aforementioned method embodiment, or may be a memory node.

[0146] When the computer device is a computing node, the processor 901 is used to: determine the first target data to be read / written; if the key value of the first target data belongs to the first key value set, send a first request to the memory node through the communication interface 903, the first request is used to request the first reader / writer of the memory node to read / write the first target data in the first storage medium, and the first storage medium stores a first data set; if the key value of the first target data belongs to the second key value set, send a second request to the memory node through the communication interface 903, the second request is used to request the second reader / writer of the memory node to read / write the first target data in the second storage medium, and the second storage medium stores a second data set; the data in the first data set The frequency of concurrent reading / writing is higher than the frequency of concurrent reading / writing of data in the second data set, the first storage medium supports access by the first reader / writer but does not support access by the second reader / writer, the second storage medium supports access by the second reader / writer, and the reading and writing speed of the first reader / writer to the second storage medium is lower than the reading and writing speed of the second reader / writer to the second storage medium; the first data set and the second data set respectively include at least one key-value pair, a key-value pair includes a key value and a value, the key value is used to index the value, the first key value set is a set of at least one key value contained in the first data set, and the second key value set is a set of at least one key value contained in the second data set.

[0147] In addition, the above modules can also be used to support other processes executed by the computing nodes in any of the above embodiments. The beneficial effects can be referred to the above description, which will not be repeated here.

[0148] When the computer device is a memory node, the processor 901 is used to: receive a first request sent by the computing node through the communication interface 903, the first request is used to request the first reader / writer of the memory node to read / write the first target data in the first storage medium, and the first request is sent by the computing node when the key value of the first target data belongs to the first key value set; according to the first request, control the first reader / writer to read and write the first target data in the first storage medium; receive a second request sent by the computing node through the communication interface 903, the second request is used to request the second reader / writer of the memory node to read / write the first target data in the second storage medium; the second request is sent by the computing node when the key value of the first target data belongs to the second key value set; according to the second request, control the second reader / writer to read and write the first target data in the second storage medium ; A first data set is stored in the first storage medium, and a second data set is stored in the second storage medium. The first data set and the second data set each include at least one key-value pair, and one key-value pair includes a key value and a numerical value, and the key value is used to index the numerical value; the first key value set is a set of at least one key value contained in the first data set, and the second key value set is a set of at least one key value contained in the second data set; the frequency of concurrent reading / writing of data in the first data set is higher than the frequency of concurrent reading / writing of data in the second data set, the first storage medium supports access by the first reader / writer but does not support access by the second reader / writer, the second storage medium supports access by the second reader / writer, and the reading and writing speed of the first reader / writer to the second storage medium is lower than the reading and writing speed of the second reader / writer to the second storage medium.

[0149] In addition, the above modules can also be used to support Figure 2 to Figure 7(b) The other processes and beneficial effects performed by the memory node in the illustrated embodiment can be referred to the previous description and will not be described in detail here.

[0150] An embodiment of the present application provides a data access system, including: at least one computing node (or a data access device for executing a computing node function) and at least one memory node (or a data access device for executing a memory node function).

[0151] In the description of the embodiments of the present application, "A / B" describes the association relationship of the associated objects, indicating that there may be three relationships, such as A alone, A and B together, and B alone. The term "multiple" in the present application refers to two or more.

[0152] In addition, it should be understood that in the description of the present application, the words "first", "second", "third", etc. are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order. The reference to "one embodiment" or "some embodiments" described in this specification means that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0153] An embodiment of the present application provides a computer-readable storage medium storing a computer program, wherein the computer program includes instructions for executing the above method embodiment.

[0154] An embodiment of the present application provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the above method embodiment.

[0155] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0156] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0157] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0158] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0159] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0160] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data access method, characterized in that: Applied to a computing node, the method comprises: determining first target data to be read / written; If the key value of the first target data belongs to the first key value set, sending a first request to the memory node, the first request is used to request a first reader / writer of the memory node to read / write the first target data in a first storage medium, and the first storage medium stores a first data set; If the key value of the first target data belongs to the second key value set, sending a second request to the memory node, the second request is used to request a second reader / writer of the memory node to read / write the first target data in a second storage medium, and the second storage medium stores a second data set; The frequency of concurrent reading / writing of data in the first data set is higher than the frequency of concurrent reading / writing of data in the second data set, the first storage medium supports access by the first reader / writer but does not support access by the second reader / writer, the second storage medium supports access by the second reader / writer, and the reading / writing speed of the first reader / writer to the second storage medium is lower than the reading / writing speed of the second reader / writer to the second storage medium; The first data set and the second data set respectively include at least one key-value pair, each of which includes a key value and a numerical value, and the key value is used to index the numerical value. The first key value set is a set of at least one key value contained in the first data set, and the second key value set is a set of at least one key value contained in the second data set.

2. The method according to claim 1, characterized in that The method further comprises: receiving first indication information sent by the memory node, the first indication information being used to indicate a processing method for target data to be read / written determined according to a third key value set, the third key value set being a set of at least one key value included in a third data set, the first indication information being sent by the memory node to the computing node after the memory node stores the third data set in the first storage medium and transfers and stores data stored in the first storage medium except the third data set in the second storage medium; After receiving the first indication information, determining second target data to be read / written; If the key value of the second target data belongs to the third key value set, sending a third request to the memory node, wherein the third request is used to request the first reader / writer to read / write the second target data in the first storage medium; If the key value of the second target data does not belong to the third key value set, a fourth request is sent to the memory node, and the fourth request is used to request the second reader / writer to read / write the second target data in the second storage medium.

3. The method according to claim 2, characterized in that Before receiving the first indication information sent by the memory node, the method further includes: receiving second indication information sent by the memory node, the second indication information being used to indicate a processing method for target data to be read / written determined according to the first key-value set, the second indication information being sent to the computing node when the third data set has not yet been completely stored in the first storage medium of the memory node; After receiving the second indication information, determining third target data to be read / written; If the key value of the third target data belongs to the third key value set and does not belong to the first key value set, the read / write operation on the third target data is suspended.

4. The method according to claim 3, characterized in that After suspending the read / write operation on the third target data, the method further includes: After receiving the third indication information or the first indication information sent by the memory node, sending a fifth request to the memory node, wherein the fifth request is used to request the first reader / writer to read / write the third target data in the first storage medium; The third indication information is used to indicate a processing method for the target data to be read / written based on the first key value set and the third key value set. The third indication information is sent to the computing node when the memory node has not yet stored the data to be transferred in the second storage medium. The data to be transferred is the data stored in the first storage medium except the third data set.

5. The method according to any one of claims 2 to 4, characterized in that: Before receiving the first indication information sent by the memory node, the method further includes: Receive third indication information sent by the memory node, the third indication information is used to indicate a processing method for target data to be read / written determined according to the first key value set and the third key value set, the third indication information is sent to the computing node when the memory node has not yet stored the data to be transferred in the second storage medium, and the data to be transferred is the data stored in the first storage medium except the third data set; After receiving the third indication information, determining fourth target data to be read / written; If the key value of the fourth target data belongs to the third key value set, sending a sixth request to the memory node, wherein the sixth request is used to request the first reader / writer to read / write the fourth target data in the first storage medium; If the key value of the fourth target data does not belong to the first key value set and does not belong to the third key value set, sending a seventh request to the memory node, wherein the seventh request is used to request the second reader / writer to read / write the fourth target data in the second storage medium; If the key value of the fourth target data belongs to the first key value set and does not belong to the third key value set, and the fourth target data is the target data to be read, sending an eighth request to the memory node, wherein the eighth request is used to request the first reader / writer to read the fourth target data in the first storage medium; If the key value of the fourth target data belongs to the first key value set and does not belong to the third key value set, and the fourth target data is the target data to be written, the write operation on the fourth target data is suspended.

6. The method according to claim 5, characterized in that After suspending the writing operation on the fourth target data, the method further includes: After receiving the first indication information, a ninth request is sent to the memory node, where the ninth request is used to request the second reader / writer to write the fourth target data in the second storage medium.

7. The method according to any one of claims 1 to 6, characterized in that: The first reader / writer is a data processing unit DPU, and / or the second reader is a remote direct memory access RDMA network card.

8. A data access method, characterized in that: Applied to a memory node, the method includes: Receive a first request sent by a computing node, where the first request is used to request a first reader / writer of the memory node to read / write first target data in a first storage medium, and the first request is sent by the computing node when a key value of the first target data belongs to a first key value set; According to the first request, controlling the first reader / writer to read and write the first target data in the first storage medium; The memory node receives a second request sent by the computing node, where the second request is used to request a second reader / writer of the memory node to read / write the first target data in a second storage medium; the second request is sent by the computing node when a key value of the first target data belongs to a second key value set; According to the second request, controlling the second reader / writer to read and write the first target data in the second storage medium; The first storage medium stores a first data set, and the second storage medium stores a second data set, the first data set and the second data set each include at least one key-value pair, one key-value pair includes a key value and a value, and the key value is used to index the value; the first key value set is a set of at least one key value included in the first data set, and the second key value set is a set of at least one key value included in the second data set; The frequency of concurrent reading / writing of data in the first data set is higher than the frequency of concurrent reading / writing of data in the second data set, the first storage medium supports access by the first reader / writer but does not support access by the second reader / writer, the second storage medium supports access by the second reader / writer, and the reading and writing speed of the first reader / writer to the second storage medium is lower than the reading and writing speed of the second reader / writer to the second storage medium.

9. The method according to claim 8, characterized in that The method further comprises: storing a third data set in the first storage medium, and transferring data stored in the first storage medium except the third data set to be stored in the second storage medium; Sending first indication information to the computing node, where the first indication information is used to indicate determining a processing method for target data to be read / written according to a third key value set, where the third key value set is a set of at least one key value included in the third data set; After sending the first indication information, receiving a third request sent by the computing node, the third request being used to request the first reader / writer to read / write second target data in the first storage medium, the key value of the second target data belonging to the third key value set; According to the third request, controlling the first reader / writer to read and write the second target data in the first storage medium; After sending the first indication information, receiving a fourth request sent by the computing node, the fourth request being used to request the second reader / writer to read / write second target data in the second storage medium, the key value of the second target data not belonging to the third key value set; According to the fourth request, the second reader / writer is controlled to read and write the second target data in the second storage medium.

10. The method according to claim 9, characterized in that Before sending the first indication information to the computing node, the method further includes: Determine first data to be transferred, read the first data to be transferred from the second storage medium and store it in the first storage medium, the first data to be transferred being data that belongs to the third data set and is not stored in the first storage medium; Determine second data to be transferred, read the second data to be transferred from the first storage medium and store it in the second storage medium, the second data to be transferred belongs to the first data set and does not belong to the third data set.

11. The method according to claim 9 or 10, characterized in that: Before sending the first indication information to the computing node, the method further includes: When the third data set has not been completely stored in the first storage medium, a second indication message is sent to the computing node, wherein the second indication message is used to indicate a processing method for the target data to be read / written based on the first key value set, and to suspend reading / writing operations on the target data whose key value belongs to the third key value set and does not belong to the first key value set.

12. The method according to claim 11, characterized in that The method further comprises: When the second data to be transferred is not stored in the second storage medium, third indication information is sent to the computing node, where the second data to be transferred is data belonging to the first data set and not belonging to the third data set, and the third indication information is used to indicate a processing method for the target data to be read / written determined according to the first key value set and the third key value set; After sending the third indication information, receiving a fifth request sent by the computing node, the fifth request being used to request the first reader / writer to read / write third target data in the first storage medium, the key value of the third target data belonging to the third key value set and not belonging to the first key value set; According to the fifth request, the first reader / writer is controlled to read and write the third target data in the first storage medium.

13. The method according to any one of claims 9 to 12, characterized in that: Before sending the first indication information to the computing node, the method further includes: When the second data to be transferred is not stored in the second storage medium, third indication information is sent to the computing node, the third indication information is used to indicate a processing method for the target data to be read / written determined according to the first key value set and the third key value set, the second data to be transferred being data belonging to the first data set and not belonging to the third data set; The write operation on the target data whose key value belongs to the first key value set and does not belong to the third key value set is suspended.

14. The method according to claim 13, characterized in that After sending the third indication information to the computing node, the method further includes at least one of the following: receiving a sixth request sent by the computing node, the sixth request being used to request the first reader / writer to read / write fourth target data in the first storage medium, the key value of the fourth target data belonging to the third key value set; and controlling the first reader / writer to read / write the fourth target data in the first storage medium according to the sixth request; receiving a seventh request sent by the computing node, the seventh request being used to request the second reader / writer to read / write the fourth target data in the second storage medium, wherein the key value of the fourth target data does not belong to the first key value set and does not belong to the third key value set; According to the seventh request, controlling the second reader / writer to read and write the fourth target data in the second storage medium; Receive the eighth request sent by the computing node, the eighth request is used to request the first reader / writer to read the fourth target data in the first storage medium, the key value of the fourth target data belongs to the first key value set and does not belong to the third key value set, and according to the eighth request, control the first reader / writer to read the fourth target data in the first storage medium and feed back to the computing node.

15. The method according to claim 14, characterized in that Receiving a sixth request sent by the computing node, where the sixth request is used to request the first reader / writer to read / write fourth target data in the first storage medium, including: receiving a sixth request sent by the computing node; If the sixth request is used to request the first reader / writer to write the fourth target data into the first storage medium, control the first reader / writer to write the fourth target data into the first storage medium; If the sixth request is used to request the first reader / writer to read the fourth target data in the first storage medium, and the fourth target data is not stored in the first storage medium, control the first reader / writer to read the fourth target data from the second storage medium and write it into the first storage medium, and return the read fourth target data to the computing node.

16. The method according to any one of claims 13 to 15, characterized in that: After suspending the write operation on the target data whose key value belongs to the first key value set and does not belong to the third key value set, the method further includes: receiving a ninth request sent by the computing node, wherein the ninth request is used to request the second reader / writer to write the fourth target data in the second storage medium, wherein the key value of the fourth target data belongs to the first key value set and does not belong to the third key value set; According to the ninth request, the second reader / writer is controlled to write the fourth target data into the second storage medium.

17. The method according to any one of claims 8 to 16, characterized in that: The first reader / writer is a data processing unit DPU, and / or the second reader is a remote direct memory access RDMA network card.

18. A data access device, characterized in that: including a communication interface and a processing unit; The communication interface is used to send and receive information; The processing unit is configured to execute the method according to any one of claims 1 to 7 through the communication interface.

19. A data access device, characterized in that: including a communication interface and a processing unit; The communication interface is used to send and receive information; The processing unit is used to execute the method according to any one of claims 8 to 17 through the communication interface.

20. A data access system, characterized in that: include: At least one data access device according to claim 18 and at least one data access device according to claim 19.

21. A computer device, characterized in that: including memory and processor; The memory stores a computer program; The processor is used to call the computer program stored in the memory to execute the method according to any one of claims 1 to 17.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 17.

Images