A data processing method, communication device and system

Abstract

The application discloses a data processing method, a communication device and a system, and relates to the technical field of data processing. A first computing device sends a first directory to a second computing device, so that the second computing device accesses the memory of the first computing device based on the first directory through RDMA technology; capacity expansion processing is performed on a first hash sub-table, so that a second hash sub-table, a third hash sub-table and a second directory are obtained; the first key-value pair in the first hash sub-table is stored in the third hash sub-table, and other key-value pairs in the first hash sub-table except the first key-value pair are stored in the second hash sub-table; the second directory is used for searching the hash sub-table stored in the memory after the capacity expansion processing; and the second directory is sent to the second computing device, so that the second computing device accesses the memory based on the second directory through RDMA technology. The method helps to perform capacity expansion processing on the hash sub-table in the memory of the first computing device, and meanwhile guarantees efficient RDMA access of the second computing device to the memory.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method, communication device and system. Background Technology

[0002] Remote direct memory access (RDMA) is a technology that directly and quickly transfers data from one computer to the memory of another remote computer without the intervention of the operating systems of both computers. This reduces the overhead of the central processing unit (CPU) in the data transfer process, freeing up memory bandwidth and processor cycles, thereby improving system performance and offering high bandwidth, low latency, and low CPU utilization. In the remote computer's memory, data is organized as key-value pairs. A hash table is used to index all stored key-value pairs, providing services such as insertion, querying, deletion, and updating of key-value pairs to several computers connected to the remote computer via RDMA.

[0003] With the development of distributed technologies, the demand for direct RDMA access is increasing. This inevitably requires expanding the capacity of hash tables in remote computer memory (referred to as resizing), for example, doubling the hash table's capacity and then transferring some elements from the old hash table to the new hash table. Because resizing operations are lengthy, they can affect the efficient processing of normal RDMA access requests to the hash tables stored in memory by other computers. Furthermore, during resizing, RDMA access requests need to query both the old and new hash tables, which can lead to access errors.

[0004] Therefore, how to expand the hash table while ensuring efficient RDMA access to the remote computer's memory remains one of the important problems that urgently need to be solved. Summary of the Invention

[0005] This application provides a data processing method, computing device, and system that helps to expand the capacity of a hash sub-table in the memory of a first computing device, while ensuring efficient RDMA access to that memory by a second computing device.

[0006] In a first aspect, embodiments of this application provide a data processing method that can be applied to a first computing device in a communication system. The communication system may also include a second computing device. The second computing device can access the memory of the first computing device through Remote Direct Memory Access (RDMA) technology. The data in the memory is stored in the form of key-value pairs. The memory stores a first directory and at least one hash sub-table. The first directory is used to look up the hash sub-tables stored in the memory, and each hash sub-table is used to store key-value pairs.

[0007] In this method, a first computing device may send the first directory to a second computing device, enabling the second computing device to access the memory of the first computing device via RDMA technology based on the first directory; the capacity of the first hash sub-table in the at least one hash sub-table is expanded to obtain a second hash sub-table, a third hash sub-table, and a second directory, wherein the third hash sub-table is used to store the first key-value pairs migrated from the first hash sub-table, the second hash sub-table is used to store other key-value pairs in the first hash sub-table besides the first key-value pairs, and the second directory is used to look up the hash sub-table stored in the memory after the capacity expansion process; the second directory is sent to the second computing device, enabling the second computing device to access the memory of the first computing device via RDMA technology based on the second directory.

[0008] With this scheme, the first computing device can send the corresponding directory to the second computing device before and after performing capacity expansion processing on the first hash sub-table stored in its memory. This allows the second computing device to access the memory of the first computing device through RDMA technology based on the received directory. This reduces system resource overhead by reducing the number of operations performed by the second computing device to read the directory through RDMA. At the same time, by ensuring the consistency of the directories between the first and second computing devices, the correct RDMA access of the memory of the first computing device by the second computing device is guaranteed.

[0009] In one possible design, each hash sub-table has a bit marker and a directory entry. The bit marker of any hash sub-table represents the number of bits used to maintain the directory entry of the hash sub-table. The directory entry of any hash sub-table is obtained based on the bit marker of the hash sub-table and the hash value of the key-value pairs stored in the hash sub-table. The capacity expansion process is performed on the first hash sub-table among the at least one hash sub-table to obtain a second hash sub-table, a third hash sub-table, and a second directory. This includes: expanding the capacity of the first hash sub-table based on the bit marker of the first hash sub-table, and obtaining the second hash sub-table, the third hash sub-table, and the second directory; wherein the bit marker of the second hash sub-table is the same as the bit marker of the third hash sub-table, and the values ​​of the bit markers of the second hash sub-table and the third hash sub-table are increased by a set value compared to the value of the bit marker of the first hash sub-table.

[0010] This scheme maintains hash sub-tables based on bit tags and directory entries to identify different hash sub-tables stored in the memory of the first computing device. This facilitates capacity expansion of the hash sub-tables and maintenance of updated directories, helps reduce the impact of expansion operations, and ensures correct RDMA access of the memory of the first computing device by the second computing device.

[0011] In one possible design, the first directory and the second directory have bit markers, where the bit marker of the first directory represents the number of bits used to maintain directory entries in the first directory, and the bit marker of the second directory represents the number of bits used to maintain directory entries in the second directory. The process of expanding the capacity of the first hash sub-table based on the bit marker of the first hash sub-table to obtain the second hash sub-table, the third hash sub-table, and the second directory includes: when the bit marker of the first hash sub-table is less than the bit marker of the first directory, expanding the capacity of the first hash sub-table to obtain the second hash sub-table, the third hash sub-table, and the second directory, wherein the value of the bit marker of the second directory is the same as the value of the bit marker of the first directory; or, when the bit marker of the first hash sub-table is the same as the bit marker of the first directory, expanding the capacity of both the first directory and the first hash sub-table to obtain the second hash sub-table, the third hash sub-table, and the second directory, wherein the value of the bit marker of the second directory is increased by the set value compared to the value of the bit marker of the first directory.

[0012] This scheme maintains the directory based on bit tags and directory entries. As the number of hash sub-tables stored in the memory of the first computing device increases, a corresponding directory is maintained so that the hash sub-tables stored in memory can be found according to the directory. This helps ensure that the second computing device can correctly access the memory of the first computing device via RDMA.

[0013] In one possible design, before performing capacity expansion processing on the first hash sub-table in the at least one hash sub-table, the method further includes: receiving first information from the second computing device, the first information being used to instruct capacity expansion of the first hash sub-table; or, determining, based on the capacity of the at least one hash sub-table, the first hash sub-table that satisfies a set capacity expansion condition in the at least one hash sub-table.

[0014] Through this scheme, the first computing device and / or the second computing device can agree on a communication protocol and hash algorithm, and maintain the hash sub-table and directory stored in the memory of the first computing device.

[0015] In one possible design, any hash sub-table includes multiple hash buckets, and any hash bucket is used to store key-value pairs and record the bit flags and directory entries of the corresponding hash sub-table. The method further includes: sending a target hash bucket to the second computing device according to the target hash bucket address received from the second computing device, wherein the target hash bucket address is obtained by hashing the target key to be accessed and the first directory cached locally by the second computing device; before sending the second directory to the second computing device, the method further includes: receiving an RDMA read message from the second computing device, wherein the RDMA read message is sent when the verification of the target hash bucket based on the first directory, the bit flags and directory entries recorded in the target hash bucket fails, and the RDMA read message is used to notify the first computing device to provide feedback on the second directory.

[0016] This scheme enables the second computing device to verify the target hash bucket read from the first computing device based on bit tags and directory entries, and to obtain the second directory from the first computing device if the verification fails. This ensures the consistency of the directories between the first and second computing devices, thereby guaranteeing the correct RDMA access of the second computing device to the memory of the first computing device.

[0017] Secondly, embodiments of this application provide a data processing method applied to a second computing device; wherein the second computing device accesses the memory of a first computing device via Remote Direct Memory Access (RDMA) technology, the data in the memory is stored in the form of key-value pairs, the memory stores a first directory and at least one hash sub-table, the first directory is used to look up the hash sub-table stored in the memory, each hash sub-table is used to store key-value pairs, the method includes: receiving a first directory from the first computing device and then caching the first directory; based on the first directory, accessing the memory of the first computing device via RDMA technology; receiving a second directory from the first computing device, then caching the second directory and deleting the first directory; the second directory is used to look up the hash sub-table stored in the memory after capacity expansion processing; based on the second directory, accessing the memory of the first computing device via RDMA technology.

[0018] In one possible design, each hash sub-table has a bit flag and a directory entry, where the bit flag of any hash sub-table is the number of bits used to maintain the directory entry of the hash sub-table, and the directory entry of any hash sub-table is obtained based on the bit flag of the hash sub-table and the hash value of the key-value pairs stored in the hash sub-table.

[0019] In one possible design, the first directory and the second directory have bit markers, wherein the bit marker of the first directory is the number of bits used to maintain the directory entries in the first directory, and the bit marker of the second directory is the number of bits used to maintain the directory entries in the second directory; the value of the bit marker of the second directory is the same as the value of the bit marker of the first directory; or, the value of the bit marker of the second directory is increased by a set value compared to the value of the bit marker of the first directory.

[0020] In one possible design, any hash sub-table includes multiple hash buckets, and any hash bucket is used to store key-value pairs and record the bit flags and directory entries of the corresponding hash sub-table. Based on the first directory, accessing the memory of the first computing device via RDMA technology includes: performing a hash operation based on the target key to be accessed and the locally cached first directory to obtain the address of the target hash bucket; retrieving the target hash bucket from the memory of the first computing device based on the target hash bucket address; before receiving the second directory from the first computing device, the method further includes: if the verification of the target hash bucket fails based on the first directory, the bit flags and directory entries recorded in the target hash bucket, sending an RDMA read message to the first computing device, the RDMA read message being used to notify the first computing device to provide feedback on the second directory.

[0021] In one possible design, the method further includes: when the first hash sub-table meets the set capacity expansion condition, sending first information to the first computing device, the first information being used to instruct the capacity expansion of the first hash sub-table.

[0022] Thirdly, embodiments of this application provide a communication device applied to a first computing device. The communication device includes: a memory for storing data, wherein a second computing device accesses the memory via Remote Direct Memory Access (RDMA) technology, the data in the memory is stored in key-value pairs, the memory stores a first directory and at least one hash sub-table, the first directory is used to look up the hash sub-table stored in the memory, and each hash sub-table is used to store key-value pairs; a communication unit for sending the first directory to the second computing device, so that the second computing device accesses the memory via RDMA technology based on the first directory; and a processing unit for performing capacity expansion processing on the first hash sub-table in the at least one hash sub-table to obtain a second hash sub-table, a third hash sub-table, and a second directory, wherein the third hash sub-table is used to store first key-value pairs moved out of the first hash sub-table, and the second hash sub-table is used to store other key-value pairs in the first hash sub-table besides the first key-value pairs; the second directory is used to look up the hash sub-table stored in the memory after the capacity expansion processing; the communication unit is also used to send the second directory to the second computing device, so that the second computing device accesses the memory via RDMA technology based on the second directory.

[0023] In one possible design, each hash sub-table has a bit marker and a directory entry. The bit marker of any hash sub-table is the number of bits used to maintain the directory entry of the hash sub-table. The directory entry of any hash sub-table is obtained based on the bit marker of the hash sub-table and the hash value of the key-value pairs stored in the hash sub-table. The processing unit is used to: perform capacity expansion processing on the first hash sub-table based on the bit marker of the first hash sub-table, and obtain the second hash sub-table, the third hash sub-table, and the second directory. Wherein, the bit marker of the second hash sub-table is the same as the bit marker of the third hash sub-table, and the values ​​of the bit marker of the second hash sub-table and the bit marker of the third hash sub-table are increased by a set value compared to the value of the bit marker of the first hash sub-table.

[0024] In one possible design, the first directory and the second directory have bit markers, where the bit marker of the first directory represents the number of bits used to maintain directory entries in the first directory, and the bit marker of the second directory represents the number of bits used to maintain directory entries in the second directory. The processing unit is configured to: when the bit marker of the first hash sub-table is less than the bit marker of the first directory, perform capacity expansion processing on the first hash sub-table to obtain the second hash sub-table, the third hash sub-table, and the second directory, wherein the value of the bit marker of the second directory is the same as the value of the bit marker of the first directory; or, when the bit marker of the first hash sub-table is the same as the bit marker of the first directory, perform capacity expansion processing on the first directory and the first hash sub-table respectively to obtain the second hash sub-table, the third hash sub-table, and the second directory, wherein the value of the bit marker of the second directory is increased by the set value compared to the value of the bit marker of the first directory.

[0025] In one possible design, the communication unit is further configured to: receive first information from the second computing device before the processing unit performs capacity expansion processing on the first hash sub-table in the at least one hash sub-table, the first information being used to instruct capacity expansion of the first hash sub-table; or, the processing unit is further configured to: determine, based on the capacity of the at least one hash sub-table, the first hash sub-table that satisfies a set capacity expansion condition in the at least one hash sub-table before performing capacity expansion processing on the first hash sub-table in the at least one hash sub-table.

[0026] In one possible design, any hash sub-table includes multiple hash buckets, and any hash bucket is used to store key-value pairs and record the bit flags and directory entries of the corresponding hash sub-table. The communication unit is further configured to: send a target hash bucket to the second computing device according to the target hash bucket address received from the second computing device, wherein the target hash bucket address is obtained by hashing the target key to be accessed and the first directory cached locally by the second computing device; before sending the second directory to the second computing device, receive an RDMA read message from the second computing device, wherein the RDMA read message is sent when the verification of the target hash bucket based on the first directory, the bit flags and directory entries recorded in the target hash bucket fails, and the RDMA read message is used to notify the first computing device to provide feedback on the second directory.

[0027] Fourthly, embodiments of this application provide a communication device applied to a second computing device. The communication device includes: a communication unit for receiving a first directory from a first computing device; a caching unit for caching the first directory; and a processing unit for accessing the memory of the first computing device based on the first directory, through the communication unit and Remote Direct Memory Access (RDMA) technology. Data in the memory is stored in key-value pairs. The memory stores the first directory and at least one hash sub-table. The first directory is used to look up the hash sub-table stored in the memory, and each hash sub-table stores key-value pairs. The communication unit is also used to receive a second directory from the first computing device, the second directory being used to look up the hash sub-table stored in the memory after capacity expansion processing. The caching unit is also used to cache the second directory and delete the first directory. The processing unit is also used to access the memory of the first computing device based on the second directory, through the communication unit and RDMA technology.

[0028] In one possible design, each hash sub-table has a bit flag and a directory entry, where the bit flag of any hash sub-table is the number of bits used to maintain the directory entry of the hash sub-table, and the directory entry of any hash sub-table is obtained based on the bit flag of the hash sub-table and the hash value of the key-value pairs stored in the hash sub-table.

[0029] In one possible design, the first directory and the second directory have bit markers, wherein the bit marker of the first directory is the number of bits used to maintain the directory entries in the first directory, and the bit marker of the second directory is the number of bits used to maintain the directory entries in the second directory; the value of the bit marker of the second directory is the same as the value of the bit marker of the first directory; or, the value of the bit marker of the second directory is increased by a set value compared to the value of the bit marker of the first directory.

[0030] In one possible design, any hash sub-table includes multiple hash buckets, and any hash bucket is used to store key-value pairs and record the bit flags and directory entries of the corresponding hash sub-table. The processing unit is used to: perform a hash operation based on the target key to be accessed and the first directory cached locally to obtain the target hash bucket address; and retrieve the target hash bucket from the memory of the first computing device through the communication unit based on the target hash bucket address. The communication unit is also used to: before receiving the second directory from the first computing device, if the verification of the target hash bucket fails based on the first directory, the bit flags recorded in the target hash bucket, and the directory entries, send an RDMA read message to the first computing device. The RDMA read message is used to notify the first computing device to provide feedback on the second directory.

[0031] In one possible design, the processing unit is further configured to: when the first hash sub-table meets the set capacity expansion conditions, send first information to the first computing device through the processing unit, the first information being used to instruct the capacity expansion of the first hash sub-table.

[0032] Fifthly, embodiments of this application provide a communication system, including: a communication device as described in any one of the third aspects above; and a communication device as described in any one of the fourth aspects above.

[0033] Sixthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform any optional implementation of the first or second aspect described above.

[0034] In a seventh aspect, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to perform any optional implementation of the first or second aspect described above.

[0035] Based on the implementations provided in the above aspects, this application can be further combined to provide more implementations. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of dynamic hash resizing in existing technology;

[0037] Figure 2 This is a schematic diagram of the communication system provided in an embodiment of this application;

[0038] Figure 3 A schematic diagram of the directory and hash sub-table provided in the embodiments of this application;

[0039] Figure 4 A flowchart illustrating the data processing method provided in an embodiment of this application;

[0040] Figure 5 A flowchart illustrating the data processing method provided in an embodiment of this application;

[0041] Figures 6a-6d A schematic diagram illustrating the expansion of the hash sub-table provided in an embodiment of this application;

[0042] Figure 7 A flowchart illustrating the data processing method provided in an embodiment of this application;

[0043] Figure 8 A schematic diagram illustrating the verification of the target hash bucket provided in an embodiment of this application;

[0044] Figure 9 A schematic diagram illustrating the verification of the target hash bucket provided in an embodiment of this application;

[0045] Figure 10 A schematic diagram of a communication device provided in an embodiment of this application;

[0046] Figure 11 This is a schematic diagram of a communication device provided in an embodiment of this application. Detailed Implementation

[0047] Currently, to improve the efficiency of hash table resizing, the industry has proposed dynamic resizing technology. The following example uses a remote computer as the server, with a client accessing the server's memory, and combines... Figure 1 This section provides a brief overview of the implementation principle of this dynamic expansion technology.

[0048] See Figure 1 The server divides its maintained hash table into at least one hash sub-table (e.g., sub-table 1, 2, 3, etc.) and maintains a corresponding directory. This directory records the correspondence between directory entries and each hash sub-table. Through this directory, all hash sub-tables in the server's memory can be indexed or searched. When a client accesses the server's memory, it searches for the corresponding hash sub-table in the directory based on the hash value of the target key, and then searches for the corresponding hash bucket within that hash sub-table, thus completing services such as key-value pair insertion, querying, deletion, and updating. In this structure, each hash sub-table can be expanded independently without affecting RDMA access to other hash sub-tables, effectively minimizing the impact of expansion operations.

[0049] However, in some operations based on RDMA technology, sending / receiving is a two-sided operation, meaning the remote client must be aware of and participate in the process to complete the transmission. Reading and writing, on the other hand, are one-sided operations; only the server needs to specify the source and destination addresses of the information, and the client does not need to be aware of the communication. When the above-mentioned dynamic hash table expansion scheme is applied to a one-sided RDMA access scenario, when the client queries a key, it first needs to read the directory of the hash sub-table into the client via a one-sided READ operation and search for the hash sub-table corresponding to the key in that directory. Then, it needs to read the hash bucket corresponding to the key in the corresponding hash sub-table into the client again via a one-sided READ operation and search for the corresponding key-value pair in that hash bucket. Therefore, this dynamic expansion technology introduces an additional RDMA READ directory access operation in a one-sided RDMA access scenario.

[0050] In practical applications, SEND / RECEIVE is mostly used for connection control messages, while data messages are mostly handled by READ / WRITE. This results in significant additional system resource overhead when the scheme is applied to scenarios with frequent one-way RDMA access.

[0051] Therefore, in RDMA-based scenarios, how to expand the hash table (or hash sub-table) while ensuring efficient RDMA access to remote computer memory remains one of the important problems that urgently need to be solved.

[0052] In view of this, embodiments of this application provide a data processing method, communication device, and system that facilitate the expansion of hash tables while ensuring efficient RDMA access to remote computer memory. In this solution, the method and apparatus are based on the same technical concept. Since the methods and apparatus solve problems in similar principles, their implementations can be mutually referenced, and repeated details will not be elaborated further.

[0053] To facilitate understanding, some terms used in this application will be explained below, so that those skilled in the art can comprehend them.

[0054] 1) A computing device is a device that provides business services and has data connectivity functions.

[0055] In practical applications, this computing device can be a terminal device. A terminal device can also be called a terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. A terminal device can also be a chip or an application (APP) installed on it. Terminal equipment can be a handheld device with wireless connectivity, in-vehicle equipment, etc. Currently, some examples of terminal devices include: mobile phones, tablets, laptops, PDAs, mobile internet devices (MIDs), point-of-sale (POS) terminals, wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and various smart meters (smart water meters, smart electricity meters, smart gas meters), etc.

[0056] The computing device can also be a server. A server is a device that provides data connectivity services. Since a server can respond to and process service requests from terminal devices, it should generally have the ability to undertake and guarantee services. In this application, the server can be a server located in a data network (DN), such as a regular server or a server in a cloud platform; or a multi-access edge computing (MEC) server located in the core network, etc. This application does not limit the specific implementation.

[0057] For ease of distinction, in the following embodiments of this application, the computing device accessed by RDMA is referred to as the first computing device, which, exemplarily, can be a server. The computing device performing RDMA access on the first computing device is referred to as the second computing device, which, exemplarily, can be implemented as a terminal device or an application installed on a terminal device. Furthermore, in a communication system, there can be one or more second computing devices accessing the same first computing device. It is understood that the embodiments of this application are merely for ease of distinction and not for any limitation on the functionality of the communication devices. In other embodiments, the second computing device may also be the computing device being accessed by RDMA, and the first computing device may also be the communication device performing RDMA access; this application does not impose any limitations on this.

[0058] 2) A hash table, also called a hash array, is a data structure that allows direct access based on a key value. It uses a mapping function (i.e., a hash function) to map the key value to a location in the table to access records, thus speeding up the search process. This mapping function is called a hash function, and the array storing the records is called a hash table. A hash table can include multiple hash buckets, each of which can be used to store records.

[0059] In this embodiment, the hash table corresponding to the mapping function includes at least one sub-table. Since each sub-table uses the same hash algorithm as the hash table, the sub-table can also be called a hash sub-table. That is, any hash sub-table in this embodiment is a hash table. The first computing device and the second computing device can pre-agree on a hash table expansion strategy. The first computing device can establish a hash sub-table to index the key-value pairs stored in its memory. When the second computing device accesses the memory of the first computing device through an RDMA connection, the hash sub-table can be maintained by the first computing device and / or the second computing device according to the changes in the data in the memory, so as to realize the capacity expansion of the relevant hash sub-table (referred to as expansion), and gradually obtain multiple hash sub-tables.

[0060] 3) Directory, which records the mapping relationship between directory entries and hash sub-tables, and is used to look up multiple hash sub-tables in the memory of the first computing device.

[0061] In this embodiment, the mapping relationship between directory entries and hash sub-tables can be many-to-one, meaning that at least one directory entry can index the same hash sub-table, i.e., they share the same hash sub-table. Since when searching multiple hash sub-tables, the address of the record containing that key in the hash sub-table is determined by substituting the given key into the aforementioned hash function and then using the corresponding hash value, the directory can, for example, be maintained using the number of bits in the hash value, and the directory entries can be obtained according to the corresponding bits of the hash value used in the hash table expansion strategy.

[0062] 4) Bit markers, a parameter used to mark hash sub-tables, such as the number of bits used to maintain the directory entries of hash sub-tables. When the first computing device and / or the second computing device maintain hash sub-tables and directories, they can mark the corresponding hash sub-tables based on the corresponding bit markers. Then, based on the bit markers of the hash sub-tables and the hash values ​​of the key-value pairs stored in the hash sub-table, the directory entries of the hash sub-tables can be obtained. The bit markers of the hash sub-tables and the directory entries can be used as markers for the corresponding hash sub-tables and each hash bucket in the hash sub-tables. Based on the bit markers and directory entries of each hash sub-table, the directory can be obtained so that correct data access and data management can be performed subsequently based on the maintained directory and each hash sub-table. In the embodiments of this application, the bit markers can be the number of bits used to maintain the directory entries of hash sub-tables and directories, and the bit markers of directories and hash sub-tables can be different.

[0063] 5) Multiple refers to two or more.

[0064] 6) At least one refers to one or more.

[0065] 7) "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship.

[0066] In addition, it should be understood that in the description of this application, the words "first" and "second" are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance or order.

[0067] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0068] Figure 2 This is a schematic diagram of the communication system to which the embodiments of this application apply.

[0069] See Figure 2The communication system 200 may include a first computing device 210 and a second computing device 220. For example, the first computing device may be the aforementioned server, and the second computing device may be the aforementioned terminal device.

[0070] The first computing device 210 may include a memory 211 and an Ethernet network interface card (RDMA card) 212 with RDMA functionality. The second computing device 220 may include a memory 221 and an RDMA card 222. The RDMA card 212 and RDMA card 222 are connected via a network to enable communication between the first computing device 210 and the second computing device 220. When the second computing device 220 initiates an RDMA read or write request to the first computing device 210 via the network, the data to be written is directly written from the memory 221 to the memory 211, or the data to be read is directly written from the memory 211 to the memory 221, through the RDMA card 212 and RDMA card 222. It is understood that in practical applications, this communication system may include multiple second computing devices. For ease of description, this embodiment only illustrates the RDMA communication between any one of the multiple second computing devices and the first computing device, and is not intended to limit the technical solution of this application in any way.

[0071] In the solution provided in this application embodiment, the communication connection between the RDMA network card 212 of the first computing device and the RDMA network card 222 of the second computing device can also be referred to as an RDMA connection. The second computing device can directly access the memory of the first computing device through this RDMA connection. The memory of the first computing device can implement data storage and management based on a hash algorithm. The data in the memory can be organized into key-value pairs. The first computing device can provide the second computing device with services such as key-value pair insertion (i.e., writing), querying (i.e., reading), deletion, and updating.

[0072] In one implementation, to facilitate data management and RDMA access to the memory of the first computing device, the memory may include a directory and at least one hash sub-table. The directory can be used to look up the hash sub-table stored in the memory of the first computing device. Each hash sub-table may include multiple hash buckets, and each hash bucket can be used to store key-value pairs. The second computing device can obtain the directory from the first computing device and cache it locally. Then, it can access the memory of the first computing device using RDMA technology based on the locally cached directory. That is, the second computing device looks up the hash sub-table stored in the memory of the first computing device through the locally cached directory.

[0073] Since the second computing device has a local cache containing a directory for looking up hash sub-tables stored in the memory of the first computing device, when the second computing device needs to access the memory of the first computing device via RDMA, it can perform a hash operation based on the target key and the locally cached directory to determine the target hash sub-table where the target key-value pair corresponds to the target key. Once the target hash sub-table is determined, the hash bucket address of the target hash bucket containing the target key-value pair can be determined based on the hash value corresponding to the target key and the table address of the corresponding hash sub-table. Then, based on the hash bucket address of the target hash bucket, operations such as inserting, querying, deleting, and updating target key-value pairs can be performed on the target hash bucket without having to additionally access the directory in the memory of the first computing device each time it accesses the memory, thus minimizing additional system resource overhead.

[0074] Since expanding the capacity of any hash sub-table stored in the memory of the first computing device will cause changes in the number of hash sub-tables stored in the memory and the directory used to look up hash sub-tables, in order to correctly look up hash sub-tables and ensure correct RDMA access to memory, bit flags can be used as parameters to mark hash sub-tables. Bit flags are used to maintain hash sub-tables and hash sub-table directory entries. The directory is maintained according to the directory entries of each hash sub-table. When expanding the capacity of any hash sub-table, the bit flags, directory entries, and directory of the relevant hash sub-tables are updated synchronously so that the memory of the first computing device can be correctly accessed based on the updated directory.

[0075] For ease of distinction, in the following embodiments, a first hash sub-table refers to any one of the at least one hash sub-tables stored in the memory of the first computing device that needs to be expanded in capacity. The directory before each capacity expansion process of the first hash sub-table in the at least one hash sub-table is called the first directory, and the updated directory after each capacity expansion process of the first hash sub-table is called the second directory. The capacity expansion process of the first hash sub-table may include multiple processes, which is not limited in this application.

[0076] The second computing device can obtain and cache the first directory from the first computing device. Then, based on the locally cached first directory, it accesses the memory of the first computing device via RDMA technology. That is, the second computing device searches for the hash sub-table stored in the memory of the first computing device using the locally cached first directory. When the first hash sub-table in the memory of the first computing device undergoes capacity expansion processing, causing the first directory stored in memory to change and be updated to the second directory, the second computing device can use the same method to obtain the second directory from the first computing device and cache it (while simultaneously deleting the locally cached first directory). Then, based on this second directory, it accesses the memory of the first computing device via RDMA technology. That is, the second computing device searches for the hash sub-table stored in the memory of the first computing device after capacity expansion processing based on the locally cached second directory.

[0077] It should be noted that, in the embodiments of this application, the directory entries in the first / second directory and the hash sub-table stored in memory can be in one-to-one correspondence, or two or more directory entries can share a hash sub-table; this application does not limit this. Furthermore, the establishment and maintenance of the corresponding first / second directory and at least one hash sub-table can be implemented based on a hash algorithm. If the first and second computing devices support multiple hash algorithms, the first / second directory and at least one hash sub-table corresponding to different hash algorithms can be maintained in the same way; this application does not limit this.

[0078] It is understood that in the embodiments of this application, the second computing device may periodically request the first / second directory from the first computing device and cache it, or the first computing device may periodically send the first / second directory, or the second computing device may request the first / second directory from the first computing device and cache it when relevant conditions are met, or the first computing device may send the first / second directory when relevant conditions are met. This application does not limit this.

[0079] For example, since the capacity expansion processing of the first hash sub-table on the first computing device side will cause the first directory to be modified to obtain the second directory, and the first directory cached locally on the second computing device is inconsistent with the second directory stored on the first computing device side, it will cause an RDMA access error. Therefore, the second computing device can read the target hash bucket from the first computing device based on the first directory cached locally, verify the target hash bucket, and update the local directory if the verification fails, so as to obtain the second directory from the first computing device and replace the first directory cached locally with the second directory. Then, based on the second directory, the memory of the first computing device is accessed through RDMA technology, so as to ensure the consistency between the directory cached locally on the second computing device and the directory on the first computing device side as much as possible, thereby ensuring correct RDMA access to the memory of the first computing device.

[0080] In one implementation, to facilitate data management and RDMA access to the memory of the first computing device, see [reference needed]. Figure 3 Any hash sub-table stored in the memory of the first computing device can have a bit tag U and a directory entry PB. For example... Figure 3 As shown, each hash bucket of any hash sub-table can be used to store key-value pairs, as well as the bit flag U and directory entry PB of that hash sub-table. For example, the bit flag of any hash sub-table can be the number of bits used to maintain the directory entry of that hash sub-table, which can be obtained, for example, based on the bit flag of the hash sub-table and the hash value of the key-value pairs stored in the hash sub-table. A directory stored in the memory of the first computing device is used to look up the hash sub-table stored in memory, and the directory entry in the directory can be obtained based on the directory entry of each hash sub-table. Figure 3 As shown, the directory can also have bit tags (represented by U' for easy distinction). The value of the bit tag U' of the directory can change with the value of the bit tag of the hash sub-table. The value of the bit tag U' of the directory can be the maximum value of the bit tag U of at least one hash sub-table stored in memory.

[0081] When the first computing device performs capacity expansion processing on the first hash sub-table, it can synchronously update the bit flags and / or directory entries of the relevant hash sub-tables involved in the capacity expansion processing operation, and replace the first directory with the second directory. This allows for subsequent RDMA access and data management of the corresponding hash sub-tables based on the bit flags and directory entries corresponding to the target hash value of the target key, as well as the second directory. After the second computing device performs an RDMA read operation on the target hash bucket in the memory of the first computing device based on the locally cached first directory, it can verify the target hash bucket based on the bit flags and directory entries recorded in the first directory and the target hash bucket. If the verification is successful, the read data is considered valid; otherwise, the read data is considered invalid, and the locally cached first directory needs to be updated so that valid data can be retrieved again based on the latest acquired second directory, thereby ensuring correct RDMA access to the memory of the first computing device.

[0082] As the capacity of the first hash sub-table is expanded, the values ​​of the bit markers in the hash sub-tables involved in the capacity expansion operation gradually increase by a set value. The value of a bit marker in any hash sub-table can be, for example, 0, 1, 2, 3…n, where n is an integer greater than or equal to 0. The first and second computing devices can pre-agree on a communication protocol and hash algorithm, and maintain hash sub-tables and directories based on the bit markers used. Then, they perform RDMA access and data management on the corresponding hash sub-tables based on the directory. Since each bit of the hash value is 0 / 1, when the maximum value of the bit markers in at least one hash sub-table stored in the memory of the first computing device is n, the bit marker U' of the directory maintained based on that at least one hash sub-table is n, and the directory entries in the directory include 2… n For example, when the bit flag is 1, directory entries can include "0" or "1". When the bit flag is 2, directory entries can include "00", "01", "10", or "11". When the bit flag is 3, directory entries can include "000", "001", "010", "100", "100", "101", "110", "111", and so on.

[0083] It should be noted that, in this embodiment, the capacity expansion process for the first hash sub-table includes creating a new hash sub-table for the first hash sub-table and migrating some key-value pairs that should have been stored in the first hash sub-table to the newly created hash sub-table for storage, thereby expanding the capacity of the old first hash sub-table based on the old first hash sub-table and the new hash sub-table. When the capacity expansion process for the first hash sub-table reaches the relevant upper limit, the capacity expansion process for the first directory is performed first, and then the corresponding first hash sub-table is expanded.

[0084] For ease of distinction, in this embodiment, the capacity expansion process for a directory can also be referred to as directory expansion, and the capacity expansion process for a hash sub-table can also be referred to as sub-table expansion. When a directory needs to be updated (including directory updates caused by directory expansion and / or sub-table expansion, including but not limited to updating pointers, increasing the number of directory entries, etc.), the directory before the update is referred to as the first directory, and the directory after the update is referred to as the second directory. When the capacity of the first hash sub-table needs to be expanded, the hash sub-table newly created for the first hash sub-table and used to store some key-value pairs migrated from the first hash sub-table (referred to as the first key-value pairs in the following embodiments) is referred to as the third hash sub-table (i.e., the new hash sub-table), and the hash sub-table used to store other key-value pairs from the first hash sub-table besides the first key-value pairs is referred to as the second hash sub-table, which is the first hash sub-table after the first key-value pairs have been migrated out.

[0085] See Figure 4 The data processing method may include the following steps:

[0086] S410: The first computing device sends a first directory to the second computing device. After receiving the first directory from the first computing device, the second computing device caches the first directory.

[0087] The first directory is used to locate the hash sub-table stored in memory.

[0088] In this embodiment, the first computing device may actively send the first directory to the second computing device, or the first computing device may send the first directory to the second computing device in response to an RDMA read message from the second computing device. This application does not limit the implementation method. After receiving the first directory, the second computing device can establish a cache locally to cache the first directory. If the second computing device already has an old directory with the same function as the first directory cached locally, it can replace the old directory with the first directory after receiving the first directory, so that the second computing device can access the memory of the first computing device based on the newly obtained directory.

[0089] S420: The second computing device accesses the memory of the first computing device via RDMA technology based on the first directory.

[0090] For example, the second computing device can perform a hash operation based on the target key to be accessed and the first directory of the local cache to determine the target hash sub-table where the target key-value pair corresponding to the target key resides. Once the target hash sub-table is determined, the second computing device can determine the hash bucket address of the target hash bucket where the target key-value pair resides in the hash sub-table based on the target hash value corresponding to the target key and the table address of the corresponding hash sub-table. Then, the second computing device can send an RDMA message to the first computing device based on the hash bucket address of the target hash bucket. This RDMA message can include the hash bucket address of the target hash bucket, and it is used to request operations such as insertion, query, deletion, and update of target key-value pairs stored in the memory of the first computing device, thereby realizing RDMA access to the memory of the first computing device.

[0091] S430: The first computing device performs capacity expansion processing on a first hash sub-table in at least one hash sub-table to obtain a second hash sub-table, a third hash sub-table, and a second directory. (This will be discussed in conjunction with...) Figure 5 The specific implementation process of S430 will be described in detail here, but will not be repeated here.

[0092] S440: The first computing device sends the second directory to the second computing device. After receiving the second directory from the first computing device, the second computing device caches the second directory and deletes the first directory.

[0093] This second directory is used to locate the hash sub-table stored in memory after the capacity expansion process.

[0094] Similar to the first directory, in this embodiment, the first computing device may actively send the second directory to the second computing device, or the first computing device may send the second directory to the second computing device in response to an RDMA read message from the second computing device. This application does not limit the implementation method. After receiving the second directory, the second computing device replaces the locally cached first directory with the second directory, so that the second computing device can access the memory of the first computing device based on the newly obtained directory.

[0095] S450: The second computing device accesses the memory of the first computing device via RDMA technology based on the second directory.

[0096] Since S430 in the data processing scheme provided in this application embodiment may include different implementations under different circumstances, for ease of understanding, the following will be combined with... Figure 5 as well as Figures 6a-6dThis document provides an explanation. Here, U represents the bit marker, i.e., the number of bits used to maintain the directory and hash sub-table. PB represents the directory entry of the hash sub-table, which is obtained based on the bit marker of the hash sub-table and the bits of the hash value of the key-value pairs stored in the hash sub-table. Each bit of the hash value takes the value 0 / 1. PB can be obtained based on the bit marker and a combination of prefix bits, suffix bits, or any bits in the hash value. The configuration strategy of PB can be configured according to a pre-agreed hash table expansion strategy, which is not limited in this application. For ease of distinction, in the following embodiments, i represents the identifier of the hash sub-table, i = 1, 2, 3...m, where m is an integer greater than or equal to 1; Ui represents the bit marker corresponding to hash sub-table i, Ui = 0, 1, 2, 3...n, where n is an integer greater than or equal to 0; U' represents the bit marker of the first directory; U” represents the bit marker of the second directory; the first directory contains 2 directory entries. U The second directory contains 2 directory entries. U Each bit of each directory entry takes the value 0 or 1. Different directory entries can point to the same hash sub-table, meaning different directory entries share the same hash sub-table.

[0097] See Figure 5 S430 may include the following steps:

[0098] S501: The first computing device obtains the first bit tag Ui = Y and the first directory entry PBi = XX of the first hash sub-table that needs to undergo capacity expansion processing. Here, i is used to identify any one of the first hash sub-tables stored in the memory of the first computing device that needs capacity expansion processing.

[0099] In this embodiment of the application, the first computing device can set a capacity expansion condition so that when the relevant hash sub-table meets the capacity expansion condition, the capacity expansion processing flow for the corresponding hash sub-table in S430 is triggered.

[0100] For example, the capacity expansion condition may include: the capacity of the hash sub-table is greater than or equal to a set sub-table capacity threshold. In one case, the set sub-table capacity threshold may be the total capacity of the hash sub-table. If an RDMA write to a certain hash sub-table fails, it is considered that the number of key-value pairs stored in the hash sub-table has reached the total capacity, and the capacity of the hash sub-table needs to be expanded. In another case, the set sub-table capacity threshold may also be a predetermined proportion of the total capacity of the hash sub-table. If the capacity of key-value pairs already stored in a certain hash sub-table exceeds the set sub-table capacity threshold, it is considered that the capacity of the hash sub-table needs to be expanded. This application does not limit the specific implementation of this.

[0101] Since the second computing device can access the memory of the first computing device via RDMA technology, in one implementation, the second computing device can maintain the directory and various hash sub-tables in the memory of the first computing device. In this case, when the second computing device determines that the first hash sub-table meets the set capacity expansion conditions, it can send first information to the first computing device, which instructs the capacity of the first hash sub-table to be expanded. Correspondingly, the first computing device can receive the first information from the second computing device and obtain the first bit flag and the first directory entry of the first hash sub-table based on the first information.

[0102] In another implementation, the first computing device itself can maintain the directories and various hash sub-tables stored in its memory. Here, the first computing device can monitor at least one hash sub-table stored in its memory, determine a first hash sub-table that meets the set capacity expansion condition based on the capacity of the at least one hash sub-table, and then obtain the first bit tag and the first directory entry of the first hash sub-table.

[0103] It is understood that the above merely illustrates two implementation methods of the embodiments of this application, and is not intended to limit the method steps in any way.

[0104] S502: Compare the first bit marker Ui = Y of the first hash sub-table with the bit marker U' of the first directory to determine the subsequent process branch to be executed. Specifically, if the first bit marker Ui = Y is different from the U' corresponding to the first directory (e.g., Ui < U'), proceed to S503 to execute the sub-table expansion process in scenario one; if the first bit marker Ui = Y is the same as the U' corresponding to the first directory, proceed to S507 to execute the directory expansion and sub-table expansion processes in scenario two.

[0105] Scenario 1:

[0106] S503: Create a new third hash sub-table, and point the pointer of the third directory entry in the first directory to this third hash sub-table to obtain the second directory.

[0107] S504: Traverse each hash bucket of the first hash sub-table, update the first bit of the record in each hash bucket to the second bit Ui = Y + 1, and keep the first directory entry PB in each hash bucket unchanged.

[0108] S505: Based on the second directory, migrate the key-value pairs (i.e., the first key-value pairs) corresponding to the third directory entries in the first hash sub-table to the third hash table. The first hash sub-table after the first key-value pairs are migrated out becomes the second hash sub-table.

[0109] S506: Traverse each hash bucket of the third hash sub-table, and record the third bit tag and the third directory entry in each hash bucket. The third bit tag is the same as the second bit tag, and the third directory entry and the second directory entry are obtained from the first directory entry.

[0110] Scenario 2:

[0111] S507: Perform directory expansion processing on the first directory according to the bit flag U' of the first directory to obtain the second directory. The bit flag U" of the second directory is increased by a set value compared to U' of the first directory (for example, the set value is 1), and the number of directory entries is doubled.

[0112] S508: In the second directory, maintain the hash sub-tables pointed to by each directory entry pointer, including: pointing the directory entry pointers corresponding to the hash sub-tables that do not need to be expanded in capacity to the corresponding original hash sub-tables, and simultaneously entering S503, in the sub-table expansion process, pointing the second directory entry pointer associated with the first directory entry of the first hash sub-table that needs to be expanded in capacity to the second hash sub-table, and pointing the third directory entry pointer to the third hash sub-table newly created for the first hash sub-table.

[0113] See Figure 6a Initially, the bit flag U' of the first directory is 1, and it stores a hash sub-table 1. The first directory includes directory entries "0" and "1". These two directory entries share the hash sub-table 1. Each hash bucket in the hash sub-table 1 records the bit flag U1 = 0 and the directory entry PB1 = 0 of the hash sub-table.

[0114] When it is necessary to expand the capacity of hash sub-table 1, and hash sub-table 1 satisfies the first condition (i.e., the bit flag U1 of hash sub-table 1 is less than the bit flag U' of the first directory), corresponding to case one, the sub-table expansion process is executed. A new hash sub-table 2 (i.e., the third hash sub-table) is created for hash sub-table 1. The pointer of directory entry "0" (i.e., the second directory entry) in the first directory is pointed to hash sub-table 1' (i.e., the second hash sub-table), and the pointer of directory entry "1" (i.e., the third directory entry) in the directory is pointed to the newly created hash sub-table 2, resulting in the second directory. The bit flag U" of the second directory is the same as the bit flag U' of the first directory. Then, all key-value pairs corresponding to directory entry "1" in hash sub-table 1 (i.e., the first key-value pairs) are transferred to the newly created hash sub-table 2 for storage. At the same time, the bit flag U1' = 1 and the directory entry PB1' = 0 (remaining unchanged) recorded in each hash bucket of hash sub-table 1' are updated respectively, and the bit flag U2 = 1 and the directory entry PB2 = 1 recorded in each hash bucket of hash sub-table 2 are updated respectively.

[0115] See Figure 6bWhen the bit flag U' = 1 in the first directory and hash sub-table 1 and hash sub-table 2 exist, when it is necessary to expand the capacity of hash sub-table 1 (i.e. the first hash sub-table), since the bit flag U1 = 1 in hash sub-table 1 is the same as the bit flag U' = 1 in the first directory, the second condition is satisfied. Corresponding to case two, it is necessary to first expand the capacity of the first directory and then expand the capacity of hash sub-table 1.

[0116] When expanding the capacity of the first directory, the bit flag U' = 1 in the first directory is linearly increased to obtain the bit flag U” = 2 in the second directory. The number of directory entries in the resulting second directory doubles, and the second directory includes new directory entries “00”, “01”, “10”, and “11”. Among them, the new directory entries “00” and “10” correspond to the old directory entry “0” and point to hash sub-table 1 that needs to be expanded, and the new directory entries “01” and “11” correspond to the old directory entry “1” and point to hash sub-table 2 that does not need to be expanded. The underscores in the second directory are used to illustrate the changes in the maintained directory entries caused by expanding the capacity of the first directory. If hash sub-table 2 does not need to be expanded, the bit flags and directory entries recorded in each hash bucket in hash sub-table 2 remain unchanged.

[0117] When expanding the capacity of hash table 1, a new hash table 3 is created for hash table 1. The pointer of the directory entry "00" (i.e., the second directory entry) in the second directory is pointed to hash table 1' (i.e., the second hash table), and the pointer of the directory entry "10" (i.e., the third directory entry) in the second directory is pointed to the newly created hash table 3 (i.e., the third hash table). The directory entries "01" and "11" are pointed to the original hash table 2, resulting in the second directory (i.e., the third directory entry in the second directory is pointed to the third hash table, while the pointers of other directory entries remain unchanged). Then, all first key-value pairs corresponding to the directory entry "10" in hash table 1 are transferred to the newly created hash table 3. At the same time, the bit flag U1'=2 and the directory entry PB1'=00 recorded in each hash bucket of hash table 1', and the bit flag U3=2 and the directory entry PB3=10 recorded in each hash bucket of hash table 3 are updated respectively.

[0118] See Figure 6c When the bit flag U' = 2 in the first directory and there are hash sub-tables 1, 2, and 3, when it is necessary to expand the capacity of hash sub-table 2 (i.e. the first hash sub-table), since the bit flag U2 = 1 of hash sub-table 2 is less than the bit flag U' = 2 in the first directory, the first condition is met. At this time, corresponding to case one, it is only necessary to expand the capacity of hash sub-table 2.

[0119] Here, with Figure 6aThe sub-table expansion process is the same as shown. A new hash sub-table 4 (i.e., the third hash sub-table) can be created for hash sub-table 2. The pointer of the third directory entry "11" in the first directory is set to the newly created hash sub-table 4 (i.e., the third hash sub-table), while the pointers of other directory entries remain unchanged, resulting in the second directory. The bit flag U" of the second directory is the same as the bit flag U' of the first directory. Then, all the first key-value pairs corresponding to the third directory entry "11" in the original hash sub-table 2 are transferred to the newly created hash sub-table 4. At the same time, the bit flag U2' = 2 and the directory entry PB2' = 01 recorded in each hash bucket of hash sub-table 2' are updated respectively, and the bit flag U4 = 2 and the directory entry PB4 = 11 recorded in each hash bucket of hash sub-table 4 are updated respectively. The bit flags and directory entries recorded in each hash bucket of hash sub-tables 1 and 3 remain unchanged.

[0120] See Figure 6d When the bit flag U' = 2 in the first directory and hash sub-table 1, hash sub-table 2, hash sub-table 3 and hash sub-table 4 exist, when it is necessary to expand the capacity of hash sub-table 3, since the bit flag U3 = 2 in hash sub-table 3 is the same as the bit flag U' = 2 in the first directory, the second condition is satisfied. At this time, corresponding to the second case, it is necessary to first expand the capacity of the first directory and then expand the capacity of hash sub-table 3.

[0121] Here, with Figure 6b The directory expansion and sub-table expansion processes are the same. When expanding the first directory, the bit flag U'=2 in the first directory is increased by a set value to U”=3. The number of directory entries in the resulting second directory is doubled. The second directory includes new directory entries "000", "001", "010", "011", "100", "101", "110", and "111". Among them, the new directory entries "000" and "100" correspond to the old directory entry "00" and point to the hash sub-table 1 that does not need to be expanded. "001" and "101" correspond to the old directory entry "01" and point to hash sub-table 2, which does not need to be expanded. The new directory entries "010" and "110" correspond to the old directory entry "10" and point to hash sub-table 3, which needs to be expanded. The new directory entries "011" and "111" correspond to the old directory entry "11" and point to hash sub-table 4, which does not need to be expanded. The underscores in the second directory are used to indicate the changes in the directory entries caused by the capacity expansion of the first directory. The original directory entries pointing to hash sub-tables 1, 2, and 4 remain unchanged.

[0122] When expanding the capacity of hash table 3, a new hash table 5 is created for hash table 3. The pointer of the third directory entry "110" in the second directory is pointed to the newly created hash table 5 (i.e., the third hash table), while the pointers of other directory entries remain unchanged, thus obtaining the second directory. Then, all first key-value pairs corresponding to the directory entry "110" in hash table 3 are transferred to the newly created hash table 5. At the same time, the bit flag U3'=3 and the directory entry PB1'=010 recorded in each hash bucket of hash table 3' are updated respectively, as are the bit flag U5=3 and the directory entry PB5=110 recorded in each hash bucket of hash table 5, and the directory (i.e., the directory entry "110" in the directory points to hash table 5, while the bit flags and directory entries recorded in each hash bucket of hash tables 1, 2, and 4 remain unchanged.

[0123] Therefore, through the above Figures 6a-6d The diagram illustrates that, in both cases, hash sub-tables and directories can be maintained based on bit flags and directory entries to dynamically expand the corresponding hash sub-tables in memory, thereby avoiding the expansion operation from affecting access to other hash sub-tables and reducing the scope of the expansion operation.

[0124] As described above, in this embodiment of the application, in order to reduce the number of times the second computing device accesses the directory of the first computing device, the second computing device can locally cache the first directory obtained from the first computing device. When the second computing device needs to perform RDMA access to the first computing device, it can determine the hash bucket address of the target hash bucket corresponding to the target key to be accessed by RDMA based on the locally cached first directory, and then directly access the target hash bucket stored in the memory of the first computing device through RDMA technology based on the determined hash bucket address, thereby realizing remote direct data access.

[0125] Because the first directory on the first computing device side is modified to obtain the second directory during expansion operations (including directory expansion operations and / or sub-table expansion operations), if the second computing device fails to obtain the second directory from the first computing device in a timely manner, it will still use the locally cached old first directory to perform hash sub-table queries and RDMA access, i.e., "dirty reads". At this time, due to the inconsistency between the directories of the two communicating parties, the second computing device will make RDMA access errors, thereby accessing the wrong hash bucket, i.e., "dirty read" data. To avoid this situation, in this embodiment of the application, when the second computing device has already cached the first directory obtained from the first computing device locally and accesses the memory of the first computing device based on the first directory through RDMA technology, it can also verify the target hash bucket read from the first computing device and decide whether to update the locally cached directory based on the verification result, so as to ensure the consistency between the directory locally cached by the second computing device and the directory on the first computing device side, thereby ensuring correct RDMA access to the memory of the first computing device.

[0126] It is understood that, in the embodiments of this application, since the desired effect of verifying the obtained target hash bucket is to ensure the consistency between the local cache directory and the directory of the first computing device, the verification of the target hash bucket can also be described as verifying the local cached first directory based on the target hash bucket read from the first computing device. This application does not limit this description.

[0127] To facilitate understanding, the following will be combined with... Figure 7 Taking the read access of the second computing device to the memory of the first computing device as an example in the flowchart shown, Figure 4 The specific implementations of S420 and S440 in the code will be explained.

[0128] See Figure 7 S420 may include the following steps:

[0129] S701: The second computing device performs a hash operation based on the target key to be accessed and the first directory of the local cache to determine the hash bucket address of the target hash bucket where the target key-value pair corresponding to the target key is located. See S420 above for details, and will not be repeated here.

[0130] S702: Read the target hash bucket from the first computing device according to the hash bucket address of the target hash bucket. The second computing device may send an RDMA read message to the first computing device, the RDMA read message including the hash bucket address of the target hash bucket, to request reading the target hash bucket from the memory of the first computing device. The first computing device may, in response to the received RDMA read message, send the target hash bucket to the second computing device.

[0131] S703: The second computing device determines the directory item to be verified based on the bit flags recorded in the target hash bucket and the target hash value.

[0132] S704: The second computing device determines whether the determined directory entry to be verified is the same as the directory entry recorded in the target hash bucket. If they are the same, it indicates that the first directory cached locally by the second computing device is consistent with the directory on the first computing device side, and the data in the obtained target hash bucket is valid data. At this time, proceed to S705; if they are not the same, it indicates that the first directory cached locally by the second computing device is inconsistent with the directory on the first computing device side, and the data in the obtained target hash bucket is invalid data. At this time, proceed to S706 to obtain the second directory from the first computing device.

[0133] Here, in one example, see Figure 8 Taking the bit flag U'=2 of the first directory in the local cache of the second computing device as an example, when reading the target hash bucket from the first computing device based on the first directory, and verifying the target hash bucket based on the bit flag and directory entries recorded in the first directory and the target hash bucket, the first computing device may experience issues such as... Figure 6c The sub-table expansion process shown may cause inconsistencies between the directory stored in the memory of the first computing device and the first directory in the local cache of the second computing device. The RDMA read operation of the second computing device may encounter the following situations:

[0134] 1) The target directory entry is “00”. The bit flag U = 2 and the directory entry PB = 00 are recorded in the target hash bucket read from the first computing device. The determined directory entry to be verified is “00”.

[0135] This means that the directory entry to be verified is the same as the directory entry recorded in the target hash bucket, and the bit flag U' of the first directory cached locally by the second computing device is greater than or equal to the bit flag U recorded in the target hash bucket. This situation indicates that the target hash sub-table containing the target hash bucket has not undergone resizing, and the data in the target hash bucket is valid.

[0136] 2) The target directory entry is “01”. The bit flag U = 2 and the directory entry PB = 01 are recorded in the target hash bucket read from the first computing device. The determined directory entry to be verified is “01”.

[0137] This means that the directory entry to be verified is the same as the directory entry recorded in the target hash bucket, and the bit flag U' of the directory cached locally by the second computing device is smaller than the bit flag U recorded in the target hash bucket. This indicates that although the target hash sub-table has undergone expansion on the first computing device side, the target hash bucket is still in the old hash sub-table and has not undergone data migration, and the data in the target hash bucket is valid.

[0138] 3) The target directory entry is “11”. The bit flag U = 2 and the directory entry PB = 01 are recorded in the target hash bucket read from the first computing device. The determined directory entry to be verified is “11”.

[0139] If the directory entry to be verified is different from the directory entry recorded in the target hash bucket, this indicates that the target hash sub-table has undergone expansion on the first computing device side, and the target key of the query has been migrated to the new hash sub-table, making the data in the obtained target hash bucket invalid.

[0140] See another example. Figure 9 Taking U'=2 of the first directory in the local cache of the second computing device as an example, when reading the target hash bucket from the first computing device based on the first directory, and verifying the target hash bucket based on the bit flags and directory entries recorded in the first directory and the target hash bucket, the first computing device may experience issues such as... Figure 6d The directory expansion process and sub-table expansion process shown may cause inconsistencies between the directory stored in the memory of the first computing device and the first directory cached locally by the second computing device. The RDMA read operation of the second computing device may encounter the following situations:

[0141] 1) The target directory entry is “00”. The bit flag U = 2 and the directory entry PB = 00 are recorded in the target hash bucket read from the first computing device. The determined directory entry to be verified is “00”.

[0142] This means that the directory entry to be verified is the same as the directory entry recorded in the target hash bucket, and the bit flag U' of the directory cached locally by the second computing device is greater than or equal to the bit flag U recorded in the target hash bucket. This indicates that the target hash sub-table containing the target hash bucket has not undergone resizing, and the data in the target hash bucket is valid.

[0143] 2) The target directory entry is “01”. The bit flag U = 3 and the directory entry PB = 010 are recorded in the target hash bucket read from the first computing device. The determined directory entry to be verified is “010”.

[0144] This means that the directory entry to be verified is the same as the directory entry recorded in the target hash bucket, and the bit flag U' of the directory cached locally by the second computing device is smaller than the bit flag U recorded in the target hash bucket. This indicates that although the target hash sub-table has undergone expansion on the first computing device side, the target hash bucket is still in the old hash sub-table and has not undergone data migration, and the data in the target hash bucket is valid.

[0145] 3) The target directory entry is “11”. The bit flag U = 3 and PB = 010 recorded in the target hash bucket read from the first computing device, and the determined directory entry to be verified is “110”.

[0146] If the directory entry to be verified is different from the directory entry recorded in the target hash bucket, this indicates that the target hash sub-table has undergone expansion on the first computing device side, and the target key of the query has been migrated to the new hash sub-table, making the data in the obtained target hash bucket invalid.

[0147] S705: If the second computing device successfully verifies the target hash bucket based on the bit flags and directory entries recorded in the first directory and the target hash bucket, it searches for the target key-value pair in the target hash bucket according to the target hash value, and then ends the current RDMA read process.

[0148] S440 may include the following steps:

[0149] S706: If the second computing device fails to verify the target hash bucket based on the first directory, the bit flags recorded in the target hash bucket, and the directory entries, it sends an RDMA read message to the first computing device. This RDMA read message is used to notify the first computing device to provide the second directory. In response to the RDMA read message, the first computing device sends the second directory to the second computing device. The second computing device receives the second directory from the first computing device, caches the second directory, and deletes the first directory. Then, the second computing device can access the memory of the first computing device using RDMA technology based on the locally cached second directory.

[0150] At this point, it has been combined Figures 7-9 This paper introduces a data processing method according to an embodiment of this application. A second computing device can locally cache a first directory obtained from a first computing device and access the memory of the first computing device via RDMA technology based on the first directory, thereby realizing remote data access to the memory of the first computing device. The directory and hash sub-table stored in the memory of the first computing device can be maintained and expanded in capacity according to the bit flags and directory entries of the hash sub-table. The second computing device can verify the target hash bucket based on the bit flags and directory entries recorded in the target hash bucket obtained from the first computing device. If the verification fails, the locally cached first directory is updated to locally cache the second directory obtained from the first computing device. By ensuring the consistency between the locally cached directory and the directory stored in the memory of the first computing device, correct RDMA access to the memory of the first computing device is guaranteed.

[0151] Based on the same technical concept, this application also provides a communication device, see below. Figure 10 The communication device 1000 may include: a memory 1010, a communication unit 1020, and a processing unit 1030.

[0152] This computing device can be applied to the first computing device in the above embodiments. Memory 1010 is used to store data. The second computing device accesses memory 1010 via Remote Direct Memory Access (RDMA) technology. The data in the memory is stored in key-value pairs. The memory stores a first directory and at least one hash sub-table. The first directory is used to look up hash sub-tables stored in the memory. Each hash sub-table stores key-value pairs. Communication unit 1020 is used to send the first directory to the second computing device, enabling the second computing device to access the memory via RDMA technology based on the first directory. Processing unit 1030 is used to perform capacity expansion processing on the first hash sub-table in the at least one hash sub-table to obtain a second hash sub-table, a third hash sub-table, and a second directory. The third hash sub-table stores first key-value pairs migrated from the first hash sub-table, and the second hash sub-table stores other key-value pairs from the first hash sub-table besides the first key-value pairs. The second directory is used to look up hash sub-tables stored in the memory after capacity expansion processing. The communication unit 1020 is also configured to send the second directory to the second computing device, so that the second computing device can access the memory based on the second directory via RDMA technology.

[0153] In one possible design, each hash sub-table has a bit marker and a directory entry. The bit marker of any hash sub-table is the number of bits used to maintain the directory entry of the hash sub-table. The directory entry of any hash sub-table is obtained based on the bit marker of the hash sub-table and the hash value of the key-value pairs stored in the hash sub-table. The processing unit 130 is configured to: perform capacity expansion processing on the first hash sub-table based on the bit marker of the first hash sub-table, and obtain the second hash sub-table, the third hash sub-table, and the second directory. Wherein, the bit marker of the second hash sub-table is the same as the bit marker of the third hash sub-table, and the values ​​of the bit marker of the second hash sub-table and the bit marker of the third hash sub-table are increased by a set value compared to the value of the bit marker of the first hash sub-table.

[0154] In one possible design, the first directory and the second directory have bit markers, where the bit marker of the first directory represents the number of bits used to maintain directory entries in the first directory, and the bit marker of the second directory represents the number of bits used to maintain directory entries in the second directory. The processing unit 1030 is configured to: when the bit marker of the first hash sub-table is less than the bit marker of the first directory, perform capacity expansion processing on the first hash sub-table to obtain the second hash sub-table, the third hash sub-table, and the second directory, wherein the value of the bit marker of the second directory is the same as the value of the bit marker of the first directory; or, when the bit marker of the first hash sub-table is the same as the bit marker of the first directory, perform capacity expansion processing on the first directory and the first hash sub-table respectively to obtain the second hash sub-table, the third hash sub-table, and the second directory, wherein the value of the bit marker of the second directory is increased by the set value compared to the value of the bit marker of the first directory.

[0155] In one possible design, the communication unit is further configured to: receive first information from the second computing device before the processing unit performs capacity expansion processing on the first hash sub-table in the at least one hash sub-table, the first information being used to instruct capacity expansion of the first hash sub-table; or, the processing unit is further configured to: determine, based on the capacity of the at least one hash sub-table, the first hash sub-table that satisfies a set capacity expansion condition in the at least one hash sub-table before performing capacity expansion processing on the first hash sub-table in the at least one hash sub-table.

[0156] In one possible design, any hash sub-table includes multiple hash buckets, and any hash bucket is used to store key-value pairs and record the bit flags and directory entries of the corresponding hash sub-table. The communication unit 1020 is further configured to: send a target hash bucket to the second computing device according to the target hash bucket address received from the second computing device, wherein the target hash bucket address is obtained by hashing the target key to be accessed and the first directory cached locally by the second computing device; before sending the second directory to the second computing device, receive an RDMA read message from the second computing device, wherein the RDMA read message is sent when the verification of the target hash bucket based on the first directory, the bit flags and directory entries recorded in the target hash bucket fails, and the RDMA read message is used to notify the first computing device to provide feedback on the second directory.

[0157] See Figure 11 This application also provides a communication device, which 1100 may include a communication unit 1110, a buffer unit 1120 and a processing unit 1130.

[0158] The communication device 1100 can be applied to the second computing device in the above embodiments. Specifically, a communication unit 1110 is used to receive a first directory from the first computing device; a caching unit 1120 is used to cache the first directory; and a processing unit 1130 is used to access the memory of the first computing device based on the first directory, through the communication unit and Remote Direct Memory Access (RDMA) technology. Data in the memory is stored in key-value pairs. The memory stores the first directory and at least one hash sub-table. The first directory is used to look up the hash sub-table stored in the memory, and each hash sub-table stores key-value pairs. The communication unit 1110 is also used to receive a second directory from the first computing device, the second directory being used to look up the hash sub-table stored in the memory after capacity expansion processing; the caching unit 1120 is also used to cache the second directory and delete the first directory; and the processing unit 1130 is also used to access the memory of the first computing device based on the second directory, through the communication unit and RDMA technology.

[0159] In one possible design, each hash sub-table has a bit flag and a directory entry, where the bit flag of any hash sub-table is the number of bits used to maintain the directory entry of the hash sub-table, and the directory entry of any hash sub-table is obtained based on the bit flag of the hash sub-table and the hash value of the key-value pairs stored in the hash sub-table.

[0160] In one possible design, the first directory and the second directory have bit markers, wherein the bit marker of the first directory is the number of bits used to maintain the directory entries in the first directory, and the bit marker of the second directory is the number of bits used to maintain the directory entries in the second directory; the value of the bit marker of the second directory is the same as the value of the bit marker of the first directory; or, the value of the bit marker of the second directory is increased by a set value compared to the value of the bit marker of the first directory.

[0161] In one possible design, any hash sub-table includes multiple hash buckets, and any hash bucket is used to store key-value pairs and record the bit flags and directory entries of the corresponding hash sub-table. The processing unit 1130 is used to: perform a hash operation based on the target key to be accessed and the first directory cached locally to obtain the target hash bucket address; and retrieve the target hash bucket from the memory of the first computing device through the communication unit based on the target hash bucket address. The communication unit is also used to: before receiving the second directory from the first computing device, if the verification of the target hash bucket fails based on the first directory, the bit flags recorded in the target hash bucket, and the directory entries, send an RDMA read message to the first computing device. The RDMA read message is used to notify the first computing device to provide feedback on the second directory.

[0162] In one possible design, the processing unit 1130 is further configured to: when the first hash sub-table meets the set capacity expansion conditions, send first information to the first computing device through the processing unit, the first information being used to instruct the capacity expansion of the first hash sub-table.

[0163] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the computing device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0164] One embodiment of this application provides a communication system, which includes the aforementioned... Figure 10 The aforementioned communication device and Figure 11 The communication device shown can be used to implement the data processing method illustrated in the above embodiments. Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the communication system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0165] One embodiment of this application provides a computer-readable medium for storing a computer program, the computer program including functions for executing the above. Figure 4 , Figure 5 , Figure 7 The instructions for the method steps in the corresponding method embodiment.

[0166] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, etc.) containing computer-usable program code.

[0167] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0168] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A data processing method, characterized in that, The method is applied to a first computing device; wherein a second computing device accesses the memory of the first computing device via Remote Direct Memory Access (RDMA) technology, the data in the memory is stored in the form of key-value pairs, the memory stores a first directory and at least one hash sub-table, the first directory is used to look up the hash sub-tables stored in the memory, and each hash sub-table is used to store key-value pairs, the method comprising: Send the first directory to the second computing device so that the second computing device can access the memory of the first computing device via RDMA technology based on the first directory; The capacity of the first hash sub-table in the at least one hash sub-table is expanded to obtain a second hash sub-table, a third hash sub-table, and a second directory. The third hash sub-table is used to store the first key-value pair moved out of the first hash sub-table, and the second hash sub-table is used to store other key-value pairs in the first hash sub-table besides the first key-value pair. The second directory is used to look up the hash sub-table stored in memory after the capacity expansion process. The second directory is sent to the second computing device so that the second computing device can access the memory of the first computing device via RDMA technology based on the second directory.

2. The method according to claim 1, characterized in that, Each hash sub-table has a bit flag and a directory entry. The bit flag of any hash sub-table is the number of bits used to maintain the directory entry of the hash sub-table. The directory entry of any hash sub-table is obtained based on the bit flag of the hash sub-table and the hash value of the key-value pairs stored in the hash sub-table. The capacity of the first hash sub-table in the at least one hash sub-table is expanded to obtain a second hash sub-table, a third hash sub-table, and a second directory, including: The capacity of the first hash sub-table is expanded according to the bit flag of the first hash sub-table, and the second hash sub-table, the third hash sub-table, and the second directory are obtained; wherein the bit flags of the second hash sub-table and the third hash sub-table are the same, and the values ​​of the bit flags of the second hash sub-table and the third hash sub-table are increased by a set value compared to the value of the bit flag of the first hash sub-table.

3. The method according to claim 2, characterized in that, The first directory and the second directory have bit markers, wherein the bit marker of the first directory is the number of bits used to maintain the directory entries in the first directory, and the bit marker of the second directory is the number of bits used to maintain the directory entries in the second directory; The capacity of the first hash sub-table is expanded according to the bit flags of the first hash sub-table, and the second hash sub-table, the third hash sub-table, and the second directory are obtained, including: When the bit flag of the first hash sub-table is less than the bit flag of the first directory, the first hash sub-table is expanded to obtain the second hash sub-table, the third hash sub-table, and the second directory, wherein the bit flag of the second directory has the same value as the bit flag of the first directory; or, When the bit tag of the first hash sub-table is the same as the bit tag of the first directory, the capacity expansion process is performed on the first directory and the first hash sub-table respectively to obtain the second hash sub-table, the third hash sub-table and the second directory, wherein the value of the bit tag of the second directory is increased by the set value compared with the value of the bit tag of the first directory.

4. The method according to claim 2 or 3, characterized in that, Before performing capacity expansion processing on the first hash sub-table in the at least one hash sub-table, the method further includes: Receive first information from the second computing device, the first information being used to instruct the capacity expansion of the first hash sub-table; or Based on the capacity of the at least one hash sub-table, determine the first hash sub-table that satisfies the set capacity expansion condition among the at least one hash sub-table.

5. The method according to any one of claims 1-3, characterized in that, Each hash sub-table includes multiple hash buckets, and each hash bucket is used to store key-value pairs and record the bit flags and directory entries of the corresponding hash sub-table. The method further includes: Based on the target hash bucket address received from the second computing device, a target hash bucket is sent to the second computing device, wherein the target hash bucket address is obtained by hashing the target key to be accessed and the first directory cached locally by the second computing device; Before sending the second directory to the second computing device, the method further includes: The first computing device receives an RDMA read message, which is sent when the verification of the target hash bucket based on the first directory, the bit flags recorded in the target hash bucket, and the directory entries fails, and the RDMA read message is used to notify the first computing device to provide feedback on the second directory.

6. A data processing method, characterized in that, The method is applied to a second computing device; wherein the second computing device accesses the memory of a first computing device via Remote Direct Memory Access (RDMA) technology, the data in the memory is stored in the form of key-value pairs, the memory stores a first directory and at least one hash sub-table, the first directory is used to look up the hash sub-tables stored in the memory, and each hash sub-table is used to store key-value pairs, the method comprising: After receiving the first directory from the first computing device, the first directory is cached; Based on the first directory, the memory of the first computing device is accessed via RDMA technology; After receiving the second directory from the first computing device, the second directory is cached and the first directory is deleted; the second directory is used to look up the hash sub-table stored in memory after the capacity expansion processing. Based on the second directory, the memory of the first computing device is accessed via RDMA technology.

7. The method according to claim 6, characterized in that, Each hash subtable has a bit flag and a directory entry. The bit flag of any hash subtable is the number of bits used to maintain the directory entry of the hash subtable. The directory entry of any hash subtable is obtained based on the bit flag of the hash subtable and the hash value of the key-value pairs stored in the hash subtable.

8. The method according to claim 7, characterized in that, The first directory and the second directory have bit markers. The bit marker of the first directory is the number of bits used to maintain the directory entries in the first directory, and the bit marker of the second directory is the number of bits used to maintain the directory entries in the second directory. The value of the bit marker of the second directory is the same as the value of the bit marker of the first directory; or, the value of the bit marker of the second directory is increased by a set value compared to the value of the bit marker of the first directory.

9. The method according to any one of claims 6-8, characterized in that, Each hash sub-table includes multiple hash buckets, and each hash bucket is used to store key-value pairs and record the bit flags and directory entries of the corresponding hash sub-table. Based on the first directory, accessing the memory of the first computing device via RDMA technology includes: The target hash bucket address is obtained by performing a hash operation based on the target key to be accessed and the first directory cached locally; Based on the target hash bucket address, obtain the target hash bucket from the memory of the first computing device; Before receiving the second directory from the first computing device, the method further includes: Based on the first directory, the bit flags and directory entries recorded in the target hash bucket, when the verification of the target hash bucket fails, an RDMA read message is sent to the first computing device. The RDMA read message is used to notify the first computing device to provide feedback on the second directory.

10. The method according to any one of claims 6-8, characterized in that, The method further includes: When the first hash sub-table in the at least one hash sub-table meets the set capacity expansion condition, a first message is sent to the first computing device, the first message being used to instruct the capacity expansion of the first hash sub-table.

11. A communication device, characterized in that, Applied to a first computing device, the communication device includes: Memory is used to store data, wherein a second computing device accesses the memory via Remote Direct Memory Access (RDMA) technology, the data in the memory is stored in the form of key-value pairs, the memory stores a first directory and at least one hash sub-table, the first directory is used to look up the hash sub-table stored in the memory, and each hash sub-table is used to store key-value pairs; A communication unit is configured to send the first directory to the second computing device, so that the second computing device can access the memory based on the first directory via RDMA technology; The processing unit is configured to perform capacity expansion processing on the first hash sub-table of the at least one hash sub-table to obtain a second hash sub-table, a third hash sub-table, and a second directory, wherein the third hash sub-table is used to store the first key-value pair migrated from the first hash sub-table, the second hash sub-table is used to store other key-value pairs in the first hash sub-table besides the first key-value pair; and the second directory is used to search for the hash sub-table stored in the memory after the capacity expansion processing. The communication unit is also configured to send the second directory to the second computing device, so that the second computing device can access the memory based on the second directory via RDMA technology.

12. The communication device according to claim 11, characterized in that, Each hash sub-table has a bit flag and a directory entry. The bit flag of any hash sub-table is the number of bits used to maintain the directory entry of the hash sub-table. The directory entry of any hash sub-table is obtained based on the bit flag of the hash sub-table and the hash value of the key-value pairs stored in the hash sub-table. The processing unit is used for: The capacity of the first hash sub-table is expanded according to the bit flag of the first hash sub-table, and the second hash sub-table, the third hash sub-table, and the second directory are obtained; wherein the bit flags of the second hash sub-table and the third hash sub-table are the same, and the values ​​of the bit flags of the second hash sub-table and the third hash sub-table are increased by a set value compared to the value of the bit flag of the first hash sub-table.

13. The communication device according to claim 12, characterized in that, The first directory and the second directory have bit markers, wherein the bit marker of the first directory is the number of bits used to maintain the directory entries in the first directory, and the bit marker of the second directory is the number of bits used to maintain the directory entries in the second directory; The processing unit is used for: When the bit flag of the first hash sub-table is less than the bit flag of the first directory, the first hash sub-table is expanded to obtain the second hash sub-table, the third hash sub-table, and the second directory, wherein the bit flag of the second directory has the same value as the bit flag of the first directory; or, When the bit tag of the first hash sub-table is the same as the bit tag of the first directory, the capacity expansion process is performed on the first directory and the first hash sub-table respectively to obtain the second hash sub-table, the third hash sub-table and the second directory, wherein the value of the bit tag of the second directory is increased by the set value compared with the value of the bit tag of the first directory.

14. The communication device according to claim 12 or 13, characterized in that, The communication unit is further configured to: receive first information from the second computing device before the processing unit performs capacity expansion processing on the first hash sub-table in the at least one hash sub-table, the first information being used to instruct the capacity expansion of the first hash sub-table; or... The processing unit is further configured to: before performing capacity expansion processing on the first hash table in the at least one hash table, determine, based on the capacity of the at least one hash table, the first hash table that satisfies the set capacity expansion conditions in the at least one hash table.

15. The communication device according to any one of claims 11-13, characterized in that, Each hash sub-table includes multiple hash buckets, and each hash bucket is used to store key-value pairs and record the bit flags and directory entries of the corresponding hash sub-table. The communication unit is also used for: Based on the target hash bucket address received from the second computing device, a target hash bucket is sent to the second computing device, wherein the target hash bucket address is obtained by hashing the target key to be accessed and the first directory cached locally by the second computing device; Before sending the second directory to the second computing device, an RDMA read message is received from the second computing device. The RDMA read message is sent when the verification of the target hash bucket based on the first directory, the bit flags recorded in the target hash bucket, and the directory entries fails, and the RDMA read message is used to notify the first computing device to provide feedback on the second directory.

16. A communication device, characterized in that, The communication device is applied to a second computing device and includes: A communication unit is configured to receive a first directory from a first computing device; A cache unit is used to cache the first directory; The processing unit is configured to access the memory of the first computing device based on the first directory, via the communication unit and Remote Direct Memory Access (RDMA) technology; the data in the memory is stored in key-value pairs, and the memory stores the first directory and at least one hash sub-table, wherein the first directory is used to look up the hash sub-tables stored in the memory, and each hash sub-table is used to store key-value pairs. The communication unit is also configured to receive a second directory from the first computing device, the second directory being used to look up a hash sub-table stored in the memory after capacity expansion processing; The caching unit is also used to cache the second directory and delete the first directory; The processing unit is also configured to access the memory of the first computing device based on the second directory, through the communication unit and RDMA technology.

17. The communication device according to claim 16, characterized in that, Each hash subtable has a bit flag and a directory entry. The bit flag of any hash subtable is the number of bits used to maintain the directory entry of the hash subtable. The directory entry of any hash subtable is obtained based on the bit flag of the hash subtable and the hash value of the key-value pairs stored in the hash subtable.

18. The communication device according to claim 17, characterized in that, The first directory and the second directory have bit markers. The bit marker of the first directory is the number of bits used to maintain the directory entries in the first directory, and the bit marker of the second directory is the number of bits used to maintain the directory entries in the second directory. The value of the bit marker of the second directory is the same as the value of the bit marker of the first directory; or, the value of the bit marker of the second directory is increased by a set value compared to the value of the bit marker of the first directory.

19. The communication device according to any one of claims 16-18, characterized in that, Each hash sub-table includes multiple hash buckets, and each hash bucket is used to store key-value pairs and record the bit flags and directory entries of the corresponding hash sub-table. The processing unit is used for: The target hash bucket address is obtained by performing a hash operation based on the target key to be accessed and the first directory cached locally; Based on the target hash bucket address, the target hash bucket is obtained from the memory of the first computing device through the communication unit; The communication unit is further configured to: before receiving the second directory from the first computing device, if the verification of the target hash bucket fails based on the first directory, the bit flags recorded in the target hash bucket, and the directory entries, send an RDMA read message to the first computing device, the RDMA read message being used to notify the first computing device to provide feedback on the second directory.

20. The communication device according to any one of claims 16-18, characterized in that, The processing unit is also used for: When the first hash sub-table in the at least one hash sub-table meets the set capacity expansion condition, the processing unit sends first information to the first computing device, the first information being used to instruct the capacity expansion of the first hash sub-table.

21. A communication system, characterized in that, include: The communication device as described in any one of claims 11-15; as well as The communication device as described in any one of claims 16-20.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the method of any one of claims 1-10.

23. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method according to any one of claims 1-10.

Images