Network function status sharing method and system based on RDMA

Abstract

The present invention discloses a network function state sharing method and system based on RDMA. The method comprises: obtaining a state name of state data; wherein the state data comprises: state data to be stored or state data to be obtained; based on the state name, accessing the corresponding hash slot in the hash ring to obtain the corresponding node address; when the state data is the state data to be obtained, sending the state name to the second network function node corresponding to the node address, so that the second network function node obtains and returns the state data based on the state name; when the state data is the state data to be stored, sending the state data to be stored to the second network function node corresponding to the node address, so that the second network function node stores the state data to be stored. The present invention effectively reduces the communication delay between the nodes of the state sharing system and improves the throughput.

Description

Technical Field

[0001] The present invention relates to the field of computer communication technology, and in particular to a method and system for sharing network function status based on RDMA. Background Art

[0002] The network functions are deployed on servers in the network in the form of software or containers to process the network traffic passing through, implement firewall, intrusion detection, deep packet inspection and other functions, and improve parallel processing capabilities through clustered deployment. When processing traffic, the above network functions need state information such as flow context and flow rules to assist in processing. For this purpose, the state sharing system is used to store its state persistently and share the state with other network function nodes in the cluster.

[0003] The existing network function status sharing system mainly uses TCP protocol stack or UDP protocol stack for communication between nodes. Its communication delay is limited by the operating system kernel and hardware devices, which makes it difficult to meet the increasing network function processing task requirements as network traffic grows rapidly, thus becoming a bottleneck in network function performance.

[0004] There is currently no effective solution to the problem of bottleneck in network function performance caused by communication delay during access. Summary of the invention

[0005] In response to the problems in the prior art, the purpose of the present invention is to propose a network function state sharing method and system based on RDMA. The system will be embedded in each network function node and deployed together with the network function cluster. It will use RDMA to effectively reduce the communication delay between each node of the state sharing system and improve the throughput.

[0006] The technical solution of the present invention includes:

[0007] A network function state sharing method based on RDMA is applicable to a first network function node in a network function node group based on RDMA communication, the method comprising:

[0008] Acquire the status name of the status data; wherein the status data includes: status data to be stored or status data to be acquired;

[0009] Based on the state name, access the corresponding hash slot in the hash ring to obtain the corresponding node address, wherein the hash ring is used to maintain a mapping relationship between the hash value of the state name and the node address;

[0010] In a case where the state data is the state data to be obtained, sending the state name to the second network function node corresponding to the node address, so that the second network function node obtains and returns the state data based on the state name;

[0011] In a case where the status data is status data to be stored, the status data to be stored is sent to a second network function node corresponding to the node address, so that the second network function node stores the status data to be stored.

[0012] Further, the accessing of the hash slot in the hash ring based on the state name to obtain the corresponding node address includes:

[0013] Calculate a hash value of the state name;

[0014] Using the hash value as a hash slot ID, accessing the hash ring;

[0015] Based on the mapping relationship of <hash slot ID, node address> maintained by the hash ring, the node address storing the status data is obtained.

[0016] Further, when the state data is the state data to be obtained, sending the state name to the second network function node corresponding to the node address so that the second network function node obtains and returns the state data based on the state name includes:

[0017] Determining whether the second network function node is the first network function node;

[0018] In the case where the second network function node is the first network function node, the first network function node uses the state name as a key value to search in a local hash table to obtain the state data; wherein the hash table maintains a mapping relationship between the state name, the state type and the state data;

[0019] In the case where the second network function node is not the first network function node, the second network function node obtains the state data, and uses the state name in the read instruction as a key value to search in a local hash table to obtain and return the state data.

[0020] Further, the first network function node uses the state name as a key value to search in a local hash table to obtain the state data, including:

[0021] Calculate a hash value of the state name;

[0022] Using the hash value as the hash ID of the hash table, obtaining the storage location of the state data in the hash table; wherein the hash table includes: a hash ID field, a state type field and a state data field;

[0023] Determine whether the current data in the storage location is in the target state;

[0024] In the case where the current data is the target state, using the current data as the state data;

[0025] In the case where the current data is not in the target state, the linked list corresponding to the storage location is traversed to find the target state, and the state type of the target state is obtained:

[0026] When the status type is a string type of fixed size, the status data stored in the status data field is returned;

[0027] When the state type is a string type of dynamic size, the state data stored in the memory pool is obtained and returned through the pointer stored in the state data field.

[0028] Further, in the case where the second network function node is not the first network function node, the second network function node obtains the status data, including:

[0029] Checking, by state name, whether there is cached state data in the memory of the first network function node;

[0030] In case that the state data corresponding to the state name is cached in the memory of the first network function node, using the state data in the memory;

[0031] In a case where there is no status data corresponding to the status name cached in the memory of the first network function node, a read instruction is issued to the second network function node through RDMA communication, so that the second network function node queries and returns the status data based on the read instruction.

[0032] Further, when the state data is state data to be stored, sending the state data to be stored to a second network function node corresponding to the node address so that the second network function node stores the state data to be stored includes:

[0033] Determining whether the second network function node is the first network function node;

[0034] In the case where the second network function node is the first network function node, the first network function node stores the state data using the state name as a key value;

[0035] In the case that the second network function node is not the first network function node, the second network function node obtains the state name, and uses the state name in the read instruction as a key value to store the state data.

[0036] Furthermore, the first network function node stores the state data using the state name as a key value, including:

[0037] Calculate a hash value of the state name;

[0038] Using the hash value as the hash ID of the hash table, obtaining the storage location of the state data in the hash table; wherein the hash table includes: a hash ID field, a state type field and a state data field;

[0039] If the data in the storage location is empty, and the data type of the status data is a string type of a fixed size, the status data is stored in the status data field;

[0040] If the data in the storage location is empty and the data type of the status data is a string type of dynamic size, calling the memory pool to store the status data, and storing the pointer in the status data field accordingly;

[0041] If the data in the storage location is not empty and the first network function node is in the target state, updating the data in the storage location based on the state data;

[0042] If the data in the storage location is not empty, the first network function node is not in the target state, and the state name is not retrieved in the corresponding linked list, new memory is obtained from the memory pool as a new linked list node, and the linked list node is added to the linked list before the state data is stored;

[0043] If the data in the storage location is not empty, the first network function node is not in the target state, and the state name is retrieved from the linked list corresponding to the first network function node, the data in the storage location is updated based on the state data.

[0044] A network function state sharing system based on RDMA, comprising a network function node group based on RDMA communication, each network function node comprising:

[0045] Instruction module for:

[0046] Generate a hash ring according to the mapping relationship between the state name of the state data and the node address;

[0047] Acquire the status name of the status data; wherein the status data includes: status data to be stored or status data to be acquired;

[0048] Based on the state name, access the corresponding hash slot in the hash ring to obtain the corresponding node address;

[0049] generating a storage instruction based on the state data, or generating a read instruction based on the state name;

[0050] Sending the storage instruction or the read instruction to the network function node corresponding to the node address;

[0051] Storage modules for:

[0052] Generate a hash table for storing state data, the hash table comprising: a hash ID field, a state type field and a state data field;

[0053] Based on the storage instruction or the read instruction and using the hash table, a state sharing result is obtained.

[0054] A computer device, characterized in that it comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute any of the above methods.

[0055] A storage medium stores a computer program, wherein the computer program is configured to execute any of the above methods when running.

[0056] Compared with the prior art, the method proposed in the present invention has the following advantages and effects:

[0057] The present invention uses the RDMA protocol to transmit operation instructions, which reduces the number of memory copies and avoids the involvement of the kernel, thereby reducing the delay of state operations and correspondingly improving the processing capacity of network functions. In addition, by adding an instruction module to be responsible for addressing, the user's operation complexity is simplified, so that they do not need to care about the specific storage location of the state. Finally, through the design of consistent hashing, the scalability of the system is improved, so that it can be elastically scaled along with the network function. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 The system structure of the present invention.

[0059] Figure 2 A schematic diagram of the state acquisition process of the present invention.

[0060] Figure 3 A schematic diagram of the state storage process of the present invention. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only specific embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] In a first aspect, the present invention provides a network function state sharing system based on RDMA, such as Figure 1 As shown in the figure, each network function node includes an instruction module and a storage module. The instruction module is used to provide the network function with an interface for shared state operations, and to address the storage location of the state in the cluster according to the name of the state, and is responsible for the RDMA connection maintenance and instruction transmission between the current node and other nodes in the cluster. The storage module is responsible for the persistent storage of the state, and processes the operations issued by the instruction layer, and writes the state to the memory or returns the state stored in the memory according to the operation type.

[0063] Specifically, during the initialization process, the instruction module will initialize a piece of memory to generate a consistent hash ring of a fixed size, which consists of two fields: slot id (hash slot ID) and owner (node ​​ID). When a new node establishes a connection with it, the IP address of the new node will be hashed and mapped to the hash ring, and the hash ring will be redivided into different areas, and the owner field of each area will be updated. At the same time, a buffer for transmitting instructions will be allocated in the memory, and the buffer address raddr and the key rkey for RDMA remote operation will be exchanged with each other. During the initialization process, the storage module will initialize a piece of memory for storing the hash table of the state. The hash table includes three fields: the key field key for recording the state name, the type field type for giving the data type corresponding to the state, and the data field value for recording the data corresponding to the state. The value field uses a common data structure to support the storage of data and pointers of multiple data types. At the same time, the storage module will also initialize a memory pool to generate new nodes to resolve hash conflicts and store dynamic length string data.

[0064] The instruction module provides four state operations to the network function: Get(key) is used by the network function to obtain the state data corresponding to a state name; Set(key,value) is used by the network function to store or update the state data corresponding to a state name. When the state name is repeated, the original data will be updated with the new data by default; Add(key,value) is used to increase or decrease the state data, which is mainly used for the traffic counter in the network function; Delete(key) is used to delete the state data corresponding to a state name.

[0065] In summary, the network function status sharing system of the present invention is embedded in each network function node, and each network function node includes an instruction module and a storage module. The instruction module is used to generate a hash ring according to the mapping relationship between the state name of the state data and the node address; obtain the state name of the state data; wherein the state data includes: the state data to be stored or the state data to be obtained; based on the state name, access the corresponding hash slot in the hash ring to obtain the corresponding node address; generate a storage instruction based on the state data, or generate a read instruction based on the state name; send the storage instruction or the read instruction to the network function node corresponding to the node address. The storage module is used to generate a hash table for storing state data, and the hash table includes: a hash ID field, a state type field and a state data field; based on the storage instruction or the read instruction, and using the hash table, obtain the state sharing result.

[0066] In a second aspect, the present invention provides a method for sharing network function status based on RDMA, the method comprising: RDMA connection maintenance and instruction transmission process, addressing process, hash table operation process and status operation process.

[0067] RDMA connection maintenance and instruction transmission process, the instruction module will establish an RDMA connection in RC mode with other nodes in the cluster during initialization, and allocate a buffer for transmitting instructions when each connection is initialized, and exchange the addresses of both buffers and the key for RDMA remote operation. Later, when the instruction module needs to transmit instructions, it can use the RDMA Write primitive with the exchanged address and key to directly write the instructions into the other party's buffer, and this process does not require the other party's CPU to participate.

[0068] Addressing process, first the instruction module uses the consistent hashing algorithm to maintain a hash ring, which includes a fixed number of hash slots, and then maps the nodes in the cluster to the slots through the hash algorithm. The slots between the slot mapped to the node and the next slot mapped to other nodes belong to this node. After that, after the instruction module receives the operation of the network function on the specific state, it calculates the hash value corresponding to the state name through the hash function and obtains the slot corresponding to the hash value and the node to which it belongs.

[0069] Hash table operation process, first, when the storage module is initialized, it will build a hash table in memory based on the hash function, using the state name as the key of the table and the state data as the value. When storing a variable-size string, it will allocate a separate memory for it and store its address as the value in the hash table. When the storage module encounters a hash conflict during storage, it uses the zipper method to store the data in the new memory and store its address on the conflicting node.

[0070] Status operation process:

[0071] a) First, the instruction module uses the addressing process to obtain the node to which the state belongs.

[0072] b) If the node belongs to itself, it will pass the instruction to its own storage module; if not, it will pass the instruction to the instruction module of the corresponding node through the instruction transmission process, and the instruction module of the corresponding node will then pass the instruction to the corresponding storage module.

[0073] c) The storage module executes the corresponding hash table operation according to the instruction.

[0074] Figure 2 This is the state acquisition process in the present invention. First, the network function will use the name of the state to be acquired as the key, and call the instruction module of this system by calling the Get(key) function. After that, the instruction module will first determine whether the size and type of the key conform to the rules in the system. If not, an error will be returned. If it conforms, the key will be hashed and calculated slot_id = hash(key, slot_seed) % slot_num. The corresponding position of the hash ring is accessed through the calculated slot_id, and then the storage location of the key is obtained through the owner field, where slot_seed represents the hash slot mapping seed, and slot_num represents the total number of hash slots.

[0075] If it is this node, the instruction module will call the storage module to read the data. After receiving the storage instruction, the data module will perform another hash operation pos=hash(key,t_seed)%t_num, thereby obtaining the storage position of the state data in the hash table. After that, it will determine whether the node is the target state by comparing the key field with the key field of the existing data at the target position, where t_seed represents the hash table seed and t_num represents the hash table capacity.

[0076] If it is not the current node, it means that a hash conflict has occurred, and it is necessary to traverse the linked list corresponding to the node to find the target state. If it is found, its type field will be checked. If it is a fixed-size type, the data stored in the value field will be directly returned. If it is a dynamic-size string type, the pointer stored in the value field needs to be used to access the real data stored in the memory pool and return the data. If the instruction module finds that the storage location is not the current node, the instruction module will send the corresponding read instruction through the RDMA connection established with the corresponding node, and will use the buffer address raddr and remote operation key rkey exchanged when establishing the connection to write the instruction to the other party's buffer through the RDMA Write-IMM operation. At the same time, if the developer enables the cache function, the instruction module will call the storage module to check whether there is cached data before sending the instruction. If not, it will send the instruction to the corresponding node. After receiving the instruction, the instruction module of the corresponding node will determine whether the state corresponding to the instruction is stored in the current node. If it is, it will execute according to the above process stored in the current node, and return it to the original node through the RDMA connection after obtaining the data.

[0077] Figure 3This is the state storage process in the present invention. First, the network function will use the name of the state to be stored as the key key and the data to be stored as the value value, and call the instruction module of this system by calling the Set(key, value) function. After that, the instruction module will first determine whether the size and type of key and value conform to the rules in the system. If not, an error will be returned. If it conforms, the key will be hashed and calculated slot_id = hash(key, slot_seed)% slot_num. The corresponding position of the hash ring is accessed through the calculated slot_id, and then the storage position of the key is obtained through the owner field. If it is this node, the storage module will be called to store the state data. After the data module receives the storage instruction, it will perform another hash operation pos = hash(key, t_seed)% t_num, thereby obtaining the storage position of the state data in the hash table. If there is no data storage, the data only needs to be stored in the value field. At the same time, if the data type is a string of dynamic size, the memory pool needs to be called to obtain free memory for storage, and the address of the memory is stored in the value field. If there is already data storage, the key field will be used to determine whether the node is in the target state. If it is, it means that this is an update operation, and the corresponding value field can be directly updated. If it is not the target state, it means that the node may have a hash conflict, so it is necessary to traverse the linked list corresponding to the node. If the target state is found in the linked list, the corresponding value field can be directly updated. If it is not found, it means that this is a new storage, and a new memory needs to be obtained from the memory pool as a new node, and the node is added to the linked list before the data is stored. If the instruction module finds that the location where it should be stored is not this node, it will send the corresponding storage instruction through the RDMA connection established with the corresponding node. At the same time, if the developer enables non-blocking storage, the instruction module here will return directly, and execute a callback function to process the execution result after receiving the reply from the corresponding node. If it is not enabled, the instruction module will wait for the reply from the corresponding node. After receiving the instruction, the instruction module of the corresponding node will determine whether the state corresponding to the instruction is stored in this node. If it is, it will execute according to the above process of storing in this node, and notify the original node through the RDMA connection after the data is stored.

[0078] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. A person skilled in the art may modify or make equivalent substitutions for the technical solutions of the present invention without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A network function state sharing system based on RDMA, characterized in that: It consists of a group of network function nodes based on RDMA communication. Each network function node includes: Instruction module for: Generate a hash ring according to the mapping relationship between the state name of the state data and the node address; the hash ring consists of two fields: the hash slot ID and the node ID; Acquire the status name of the status data; wherein the status data includes: status data to be stored or status data to be acquired; Based on the state name, access the corresponding hash slot in the hash ring to obtain the corresponding node address; generating a storage instruction based on the state data, or generating a read instruction based on the state name; Sending the storage instruction or the read instruction to the network function node corresponding to the node address; Storage modules for: Generate a hash table for storing state data, the hash table comprising: a hash ID field, a state type field and a state data field; Based on the storage instruction or the read instruction and using the hash table, a state sharing result is obtained.

2. A RDMA-based network function status sharing method applicable to the system of claim 1, characterized in that: Applicable to a first network function node in a network function node group based on RDMA communication, the method comprising: Acquire the status name of the status data; wherein the status data includes: status data to be stored or status data to be acquired; Based on the state name, access the corresponding hash slot in the hash ring to obtain the corresponding node address, wherein the hash ring is used to maintain a mapping relationship between the hash value of the state name and the node address; In a case where the state data is the state data to be obtained, sending the state name to the second network function node corresponding to the node address, so that the second network function node obtains and returns the state data based on the state name; In a case where the status data is status data to be stored, the status data to be stored is sent to a second network function node corresponding to the node address, so that the second network function node stores the status data to be stored.

3. The method according to claim 2, characterized in that The accessing of the hash slot in the hash ring based on the state name to obtain the corresponding node address includes: Calculate a hash value of the state name; Using the hash value as a hash slot ID, accessing the hash ring; Based on the mapping relationship of <hash slot ID, node address> maintained by the hash ring, the node address storing the status data is obtained.

4. The method according to claim 2, characterized in that When the state data is the state data to be obtained, sending the state name to the second network function node corresponding to the node address, so that the second network function node obtains and returns the state data based on the state name, includes: Determining whether the second network function node is the first network function node; In the case where the second network function node is the first network function node, the first network function node uses the state name as a key value to search in a local hash table to obtain the state data; wherein the hash table maintains a mapping relationship between the state name, the state type and the state data; In the case where the second network function node is not the first network function node, the second network function node obtains the state data, and uses the state name in the read instruction as a key value to search in a local hash table to obtain and return the state data.

5. The method according to claim 4, characterized in that The first network function node uses the state name as a key value to search in a local hash table to obtain the state data, including: Calculate a hash value of the state name; Using the hash value as the hash ID of the hash table, obtaining the storage location of the state data in the hash table; wherein the hash table includes: a hash ID field, a state type field and a state data field; Determine whether the current data in the storage location is in the target state; In the case where the current data is the target state, using the current data as the state data; In the case where the current data is not in the target state, the linked list corresponding to the storage location is traversed to find the target state, and the state type of the target state is obtained: When the status type is a string type of fixed size, the status data stored in the status data field is returned; When the state type is a string type of dynamic size, the state data stored in the memory pool is obtained and returned through the pointer stored in the state data field.

6. The method according to claim 4, characterized in that In the case where the second network function node is not the first network function node, the second network function node acquiring the state data includes: Checking, by state name, whether there is cached state data in the memory of the first network function node; In case that the state data corresponding to the state name is cached in the memory of the first network function node, using the state data in the memory; In a case where there is no status data corresponding to the status name cached in the memory of the first network function node, a read instruction is issued to the second network function node through RDMA communication, so that the second network function node queries and returns the status data based on the read instruction.

7. The method according to claim 2, characterized in that When the state data is state data to be stored, sending the state data to be stored to a second network function node corresponding to the node address so that the second network function node stores the state data to be stored includes: Determining whether the second network function node is the first network function node; In the case where the second network function node is the first network function node, the first network function node stores the state data using the state name as a key value; In the case that the second network function node is not the first network function node, the second network function node obtains the state name, and uses the state name in the read instruction as a key value to store the state data.

8. The method according to claim 7, characterized in that The first network function node stores the state data using the state name as a key value, including: Calculate a hash value of the state name; Using the hash value as the hash ID of the hash table, obtaining the storage location of the state data in the hash table; wherein the hash table includes: a hash ID field, a state type field and a state data field; If the data in the storage location is empty, and the data type of the status data is a string type of a fixed size, the status data is stored in the status data field; If the data in the storage location is empty and the data type of the status data is a string type of dynamic size, calling the memory pool to store the status data, and storing the pointer in the status data field accordingly; If the data in the storage location is not empty and the first network function node is in the target state, updating the data in the storage location based on the state data; If the data in the storage location is not empty, the first network function node is not in the target state, and the state name is not retrieved in the corresponding linked list, new memory is obtained from the memory pool as a new linked list node, and the linked list node is added to the linked list before the state data is stored; If the data in the storage location is not empty, the first network function node is not in the target state, and the state name is retrieved from the linked list corresponding to the first network function node, the data in the storage location is updated based on the state data.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method according to any one of claims 2 to 8.

10. A storage medium storing a computer program, wherein: The computer program is arranged to perform the method according to any one of claims 2 to 8 when executed.

Images