Message processing system and data plane equipment

Abstract

The embodiment of the invention discloses a message processing system and data plane equipment, which can configure resources for a host according to the actual demand of the host and improve the flexibility of resource configuration. The message processing system comprises a first virtual machine and a first data plane device, and the first virtual machine runs in a first host; the first data plane equipment is used for receiving a first message sent by the first virtual machine; analyzing the first message, and obtaining first feature information of the first message; first control plane equipment is determined according to the first feature information, the first control plane equipment is used for executing a control plane function of a first open source virtual switch OVS of the first virtual machine, the first control plane equipment is located in a first control plane set, and the first control plane equipment set is located outside the first host.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a message processing system and a data plane device. Background Technology

[0002] Virtual machines (VMs) communicate with other VMs running on other hosts or VMs on the same host through an open-source virtual switch (Open vSwitch, OVS). When OVS processes a large amount of data flow, it will consume a lot of hardware resources on the host, which can easily affect the operation of VMs on the host and reduce work efficiency.

[0003] To reduce the burden on server hardware, the OVS function is offloaded to the smart network interface card. See [link / reference]. Figure 1 A typical offloading model includes a host running virtual machines and a Data Processing Unit (DPU). The DPU contains two processors: a control plane processor and a data plane processor. The data plane processor can be a smart network interface card (NIC). The control plane processor manages and controls OVS behavior, such as issuing or updating forwarding flow tables to the smart NIC. The smart NIC is used to forward data. Specifically, when a packet enters the smart NIC, it queries the forwarding flow table. If the packet is not found, it forwards the received packet to the control plane processor. The control plane processor generates a forwarding flow table and sends it to the data plane processor. Subsequent packets can then refer to the forwarding flow table and be forwarded directly based on it, eliminating the need for further processing by the control plane processor. Since the smart NIC's packet forwarding capacity far exceeds the control plane processor's processing capacity, this significantly improves forwarding efficiency.

[0004] This offloading model typically involves configuring one DPU per host. When the host requires high performance, multiple DPUs need to be configured. However, when performance requirements are not high, configuring multiple DPUs would be wasteful. Therefore, it is impossible to flexibly configure the number of DPUs according to the actual needs of the host. Summary of the Invention

[0005] This application provides a message processing system and a data plane device that can configure resources for a host according to its actual needs, thereby improving the flexibility of resource configuration.

[0006] In a first aspect, embodiments of this application provide a packet processing system, including: a first host and a first data plane device. The first host runs a first virtual machine. The first data plane device may be a first network interface card (NIC). The first NIC receives a first packet sent by the first virtual machine and parses the first packet to obtain first characteristic information of the first packet. The first characteristic information includes at least the source IP address, destination IP address, and protocol type of the first packet. Based on one or more combinations of the parsed first characteristic information, a flow table matching the first packet is searched. If the search fails, the intervention of a control plane device is required. In this embodiment, a first set of control plane devices is configured externally to the first host. A suitable control plane device is matched to the virtual machine on the first host from the first set of control plane devices, improving the flexibility of control plane device configuration. Furthermore, the first control plane device is determined for the virtual machine from the set of control plane devices based on the first characteristic information, achieving load balancing by selecting the control plane device. The first control plane device is used to execute the control plane functions of a first Open Window System (OVS), that is, the control plane functions of the first OVS are offloaded from the host to the first control plane device, which can improve the host's packet processing performance.

[0007] In one possible implementation, a first set of data plane devices is configured externally to the first host. Suitable first data plane control devices can be configured for virtual machines on the first host from this set, improving the flexibility of data plane device configuration. Simultaneously, the first data plane control device is used to execute the data plane functions of the first OVS (Optical Version Controller), that is, the data plane functions of the first OVS are offloaded from the host to the first data plane control device. The first control plane device can be the first network interface card (NIC), thereby improving the host's packet processing performance.

[0008] In one possible implementation, the hash operation used in this application embodiment can ensure that the hash value obtained by hashing using the first source IP address, the first destination IP address, and the protocol type is the same as the hash value obtained by hashing using the first source IP address as the second destination IP address and the first destination IP address as the second source IP address. This ensures that the packets have the same source and destination, that is, it ensures that the forward and reverse packets select the same control plane device, thereby ensuring the normal forwarding of the reverse packets. The reverse packets are packets generated by the receiving end based on the received forward packets.

[0009] In one possible implementation, after the first control plane device generates the first flow table, it sends the first flow table to the first data plane device. In this way, the first data plane device can not only forward the first packet to the next-hop device according to the first flow table, but also directly query the first flow table to forward the packet after receiving the packet again, without the intervention of the control plane device, thereby achieving high-speed processing of the packet.

[0010] The first control plane device may encapsulate the first message according to the first flow table to obtain the encapsulated message, and send the encapsulated message and the forwarding policy in the first flow table to the first control plane device. The first control plane device forwards the encapsulated message according to the forwarding policy, and then sends the first flow table to the first control plane device later.

[0011] In this embodiment of the application, the first control plane device can generate a fourth flow table while generating the first flow table, and send the fourth flow table to the first data plane device so that the first data plane device can directly query the fourth flow table to forward the message when it receives the reverse message, thereby realizing high-speed processing of the message. The reverse message is a message generated by the receiving end based on the received first message.

[0012] In one possible implementation, after receiving the first packet, the second data plane device also queries the flow table based on the first feature information. If the query fails, it determines a second control plane device from the set of second control plane devices based on the first feature information. When the second control plane device confirms that forwarding the first packet from the second data plane device to the next hop requires network address translation, it needs to re-determine a control plane device from the set of second control plane devices based on the second feature information after network address translation. This ensures that forward and reverse packets select the same control plane device, further guaranteeing that they are from the same source and have the same destination. Here, the first packet is referred to as the forward packet, and the reverse packet is a packet generated by the receiving end based on the received forward packet. If the re-determined control plane device is different from the second control plane device, the first packet needs to be forwarded to the re-determined control plane device, which is the third control plane device.

[0013] In one possible implementation, the third control plane device establishes a second flow table, which includes at least the source tunnel IP address of the first packet, so that the third control plane device can determine the destination tunnel IP address of the packet sent from the second virtual machine based on the source tunnel IP address, thereby ensuring that forward packets and reverse packets can select the same data plane device.

[0014] In one possible implementation, the second flow table also includes the identifier of the second data plane device.

[0015] In one possible implementation, the third control plane device also establishes a third flow table and sends the third flow table to the second data plane device, so that the second data plane device can directly query the third flow table to forward the message when it receives a reverse message, thereby enabling high-speed processing of the message.

[0016] In one possible implementation, after the third control plane device receives the second message, since the second flow table includes the identifier of the second data plane device (meaning the data plane device processing the flow to which the second message belongs is the second data plane device), and the third flow table matching the second message is already stored in the second data plane device, in this case, the third control plane device sends the third flow table to the third data plane device and deletes the third flow table in the second data plane device. This ensures that the reverse message can select the same data plane device. The third data plane device is the data plane device that receives the second message sent by the second virtual machine, and the third data plane device is located in the set of second data plane devices.

[0017] In one possible implementation, after receiving the second message, the first data plane device determines whether the data plane device that sent the second message is the same data plane device that received the first message. If not, the identifier of the second data plane device in the first flow table is adjusted to the identifier of the third data plane device, so that forward and reverse messages can select the same data plane device. Alternatively, after receiving the second message, the first data plane device determines whether the data plane device that sent the second message is the same data plane device that received the first message. If not, the first flow table is sent to the first control plane device, which then adjusts the identifier of the second data plane device in the first flow table to the identifier of the third data plane device.

[0018] In one possible implementation, the first flow table may also include an upload flag, which indicates that the flow needs to be uploaded to the control plane device after the flow table is hit. The control plane device then updates the hit flow table, and deletes the upload flag after successful connection establishment. In this way, the connection establishment process is completed by the control plane device, and the data plane device does not need to handle the complex connection establishment process, thereby improving packet processing efficiency.

[0019] Secondly, embodiments of this application provide a message processing method applied to a first data plane device, comprising: receiving and parsing a first message sent by a first virtual machine, obtaining first feature information of the first message, wherein the first virtual machine runs in a first host; determining a first control plane device based on the first feature information, wherein the first control plane device is used to execute the control plane function of the first open source virtual switch (OVS) of the first virtual machine, the first control plane device is located in a first control plane set, and the first control plane device set is located outside the first host.

[0020] In one possible implementation, the first feature information includes at least a first source international protocol IP address, a first destination IP address, and a protocol type. Determining the first control plane device based on the first feature information includes: performing a hash operation on the value corresponding to the first source IP address, the value corresponding to the first destination IP address, and the value corresponding to the protocol type to obtain a hash value, wherein the hash operation ensures that the calculation result is the same when the first source IP address is used as the second destination IP address and when the first destination IP address is used as the second source IP address; and determining the first control plane device corresponding to the hash value based on the correspondence between the hash value and the first control plane device.

[0021] In one possible implementation, the method further includes: receiving a second message; determining whether the third data plane device that sent the second message is the second data plane device that received the first message at the last hop; the second message is a return message generated by a second virtual machine based on the received first message; the second virtual machine runs in a second host; wherein the third data plane device and the second data plane device are located in a set of second data plane devices, and the set of second data plane devices is located outside the second host; if they are not the same, adjusting the identifier of the second data plane device in the flow table to the identifier of the third data plane device; the flow table is used to instruct the data plane device to forward the first message to the next-hop device.

[0022] The functions or technical effects achieved by the first data plane device in executing the message processing method can be referred to the functions or technical effects brought about by the first aspect of the data and any possible implementation method of the first aspect.

[0023] Thirdly, embodiments of this application also provide a data plane device, comprising: an interface for receiving a first message sent by a first virtual machine, the first virtual machine running in a first host; a processor for parsing the first message, obtaining first feature information of the first message; and determining a control plane device from a first set of control plane devices based on the first feature information, wherein the first set of control plane devices is located outside the first host, and the control plane device is used to execute the control plane function of a first open-source virtual switch (OVS) of the first virtual machine. The functions or technical effects achieved by the processor in executing the above steps can be referred to the technical effects brought about by the implementation of the first aspect described above.

[0024] Fourthly, embodiments of this application also provide a data plane apparatus, comprising: a receiving module for receiving a first message sent by a first virtual machine, the first virtual machine running on a first host; a parsing module for parsing the first message and obtaining first feature information of the first message; and a determining module for determining a first control plane device from a first control plane device set based on the first feature information, wherein the first control plane device is used to execute the control plane function of a first open source virtual switch (OVS) of the first virtual machine, and the first control plane device set is located outside the first host.

[0025] Fifthly, embodiments of this application also provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the message processing method as described in the second aspect and any possible implementation thereof.

[0026] Sixthly, embodiments of this application also provide a computer program product that, when run on a computer, causes the computer to execute the message processing method involved in the second aspect and any possible implementation thereof. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of an offloading model in the prior art that offloads OVS functionality from the host to the smart network interface card;

[0028] Figure 2 This is a schematic diagram of virtualization in existing technologies;

[0029] Figure 3 This application provides a schematic diagram of the architecture of a message processing system.

[0030] Figure 4 A schematic diagram of the hardware structure of a smart network interface card (NIC) in a message processing system provided in this application embodiment;

[0031] Figure 5 A flowchart illustrating the message processing method performed by the message processing system provided in this embodiment of the application;

[0032] Figure 6 Another flowchart illustrating the message processing method performed by the message processing system provided in this application embodiment;

[0033] Figure 7 This is a schematic diagram of the structure of the data plane device in the message processing system provided in the embodiments of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the specific embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0035] Please see Figure 2 Virtualization is the logical representation of resources, freeing them from physical constraints. Any technology that maps one form of interface and resource to another can be called virtualization. It is typically implemented by adding a virtualization software layer to the system, abstracting lower-level resources into another form for use by the upper layer. The computer system simulated through virtualization is called a virtual machine. The top-level machine running the virtual machine is called the host. Each VM can run independently, with its own operating system, applications (APPs), and of course, its own independent Media Access Control (MAC) address and Internet Protocol (IP) address. They connect to external physical networks through the host's internal OVS (Optical Virtualization System).

[0036] OVS, running on the host machine, provides Layer 2 network access and some Layer 3 network functions for VMs on the host. VMs connect to the network through OVS, while OVS uses the physical network interface card (NIC) on the host as the uplink to connect to the outside world. Based on the concept of Software Defined Networking (SDN), OVS divides the entire core architecture into a control plane and a data plane. The data plane is responsible for data exchange, while the control plane implements the exchange policy and guides the data plane's work.

[0037] Since a host may run multiple virtual machines, and each virtual machine has limited hardware resources, if the OVS (On-Site Controller) that handles data exchange tasks consumes too many hardware resources, it will affect the operation of the virtual machines on the host and reduce work efficiency. Therefore, as mentioned in the background technology, in order to reduce the burden on the host hardware, the OVS services can be offloaded to the DPU (Data Processing Unit). However, in the existing technology, one DPU is usually configured per host, which makes the forwarding capacity affected by the DPU.

[0038] To address the aforementioned technical problems, this application provides a message processing system, please refer to [link to relevant documentation]. Figure 3 The message processing system includes multiple hosts 300, each running one or more VMs. The system also includes a control plane device set for performing OVS control plane functions, i.e., offloading OVS control plane functions from the hosts 300 to the control plane devices in the control plane device set to process control plane traffic; and a data plane device set for performing OVS data plane functions, i.e., offloading OVS data plane functions from the hosts 300 to the data plane devices in the data plane device set to process data plane traffic.

[0039] The control plane device set includes multiple processors. These processors can be located in a single control plane device or in different control plane devices. The control plane device can be a host, a server, or other device; there are no restrictions here. The control plane device set is located outside the host 300, meaning that the aforementioned control plane device is different from the host 300. In other words, the control plane device and the host 300 are independent of each other.

[0040] The processor can be a general-purpose processor, such as a Central Processing Unit (CPU), a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. In specific implementations, the control plane device may also include memory, which can be volatile memory, non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM) or static random access memory (SRAM).

[0041] A data plane device suite includes multiple network interface cards (NICs). In practice, each NIC can perform data read / write operations or other computational functions. In this case, the NIC is a smart NIC. A smart NIC can be built into the data plane device or act as an external device connected to the data plane device via an interface. This interface can be a Peripheral Component Interconnect Express (PCIe) interface or a standalone NIC. When smart NICs are built into the data plane device, multiple smart NICs can reside in one data plane device or in different data plane devices. This data plane device can be a host, a server, or a Data Processing Unit (DPU).

[0042] Regarding the hardware, please refer to Figure 4 The smart network interface card (NIC) may include at least one processor 400, a memory 401, and an interface 402, wherein the interface 402 is used to receive packets sent by a virtual machine running on the host 300. In some possible implementations, the smart NIC may also include a system bus 403. At least one processor 400 and the memory 401 are respectively connected to the system bus 403. The processor 400 can access the memory 401 through the system bus 403; for example, the processor 400 can perform data read / write operations or code execution in the memory through the system bus 403. The system bus 403 may be a Quick Path Interconnect (QPI) or an Ultra Path Interconnect (UPI), etc. The system bus 402 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one ancestor is used, but this does not mean that there is only one bus or one type of bus. The type of processor included in the smart NIC may be the processor types included in the control plane device set 310 mentioned above, which will not be elaborated further here.

[0043] In the embodiments of this application, memory 401 can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM) or static random access memory (SRAM).

[0044] In the specific implementation process, the host 300 communicates with the control plane device set and the data plane device set. The connection method can be through a switching device or a bus device. The bus device can be PCIe, Compute Express Link (CXL) or other bus devices for interconnection.

[0045] In this embodiment, the control plane devices in the control plane device set and the data plane devices in the data plane device set are shared by multiple hosts. A dedicated management device manages the control plane device set and the data plane device set, allocating the control plane devices in the control plane device set and the data plane devices in the data plane device set to each host. Here, taking hosts 300-1 and 300-2 as examples, the management device will configure multiple control plane devices for host 300-1 from the control plane device set. The set of multiple control plane devices configured for host 300-1 is called the first control plane device set 310. The management device will also configure multiple control plane devices for host 300-2 from the data plane device set 320. The set of multiple data plane devices configured for host 300-1 is called the first data plane device set 320. Similarly, the management device will configure multiple data plane devices for host 300-2 from the data plane device set. The set of multiple data plane devices configured for host 300-2 is called the second data plane device set 340.

[0046] Taking host 300-1 as an example, after the management device configures the control plane and data plane devices for host 300-1, it informs host 300-1 which control plane and data plane devices have been configured. Then, the host allocates the configured control plane and data plane devices to the VMs running on the host according to actual needs. The host informs the VMs of the allocation of control plane and data plane devices so that the VMs can establish a first mapping table. This first mapping table includes the mapping between queues and data plane devices, where queues are enabled by the VM to process packets to be forwarded. In this case, the VM should also establish a second mapping table, which includes the mapping between hash values ​​and queues, so that the VM can determine the data plane device used for packet forwarding based on the first and second mapping tables. The specific process of determining the smart network interface card is described in the packet processing section below. In some possible implementations, the VM may also establish a mapping table that includes the mapping between hash values, queues, and data plane devices.

[0047] Correspondingly, the management device will also inform each data plane device of the control plane device configured for host 300-1. In this way, a third correspondence table will be established on each data plane device. This third correspondence table includes the correspondence between hash values ​​and control plane devices, so that the data plane device that receives the message can determine the corresponding control plane device according to the third correspondence table.

[0048] When adding or removing a control plane device in the control plane device set 310, for example, if control plane device 1 is added or removed from control plane device set 310, the management device will send update information to the data plane device to inform it that control plane device 1 has been removed from control plane device set 310, or if control plane device 1 has been added to control plane device set 310, the management device will send update information to the data plane device to inform it that control plane device 1 has been added to control plane device set 310. This allows the data plane device to delete the relevant correspondence from the established third correspondence table based on the received update information, or to add a correspondence to the established third correspondence table. Similarly, when adding or removing a data plane device in data plane device set 320, the management device will also send update information to host 300.

[0049] based on Figure 3 The message processing system shown allows the management device to flexibly configure the number of control plane CPUs and data plane smart network cards for each host, thereby enabling flexible configuration of control plane processing capabilities and data plane processing capabilities.

[0050] Please see Figure 5 Continuing with the above example, host 300 includes host 300-1 and host 300-2. VM1 runs on host 300-1, and VM2 runs on host 300-2. The control plane device set 310 on the host 300-1 side includes CPU1 and CPU2, and the data plane device set 320 includes smart NIC 1 and smart NIC 2. The control plane device set 330 on the host 300-2 side includes CPUA and CPUB, and the data plane device set 340 includes smart NIC A and smart NIC B. When VM1 and VM2 need to communicate, the control plane CPU and data plane smart NIC can be flexibly configured from the control plane device set 310 and data plane device set 320 of host 300-1, and the control plane device set 330 and data plane device set 340 of host 300-2, according to the actual situation, for the packets to be forwarded. The specific forwarding process will be described in detail in the following embodiments. It should be noted that... Figure 5 To clearly illustrate the message forwarding process, the CPU and smart network interface card (NIC) are shown separately. In actual implementation, CPUA and smart NIC A can be located in the same processing device, and CPUB and smart NIC B can be located in another processing device. Similarly, smart NIC 1 and CPU1 can be located in the same processing device, and smart NIC 2 and CPU2 can be located in another processing device.

[0051] Before introducing the specific forwarding process, let's first introduce the concept of a flow table used in the forwarding process. Flow tables are used to control data flow, and can specifically use flow tables conforming to the OpenFlow protocol or other protocols. A flow table contains a series of flow entries. The rules for each flow entry can be divided into two parts: matching information and instruction information. The matching information is used to match the packet's feature information; for example, the matching information indicates the value of each field in the feature information of the matched packet. In some possible implementations, a successful match between the packet's feature information and the matching information can mean that the value of each field in the feature information is the same as the value of the corresponding field in the matching information. In some optional examples, the matching information is also called a matching field; the action information determines how the matched data packet should be processed. In some optional examples, the instruction information is also called an instruction field, instruction set, or action field. The message's characteristic information includes at least several fields used to identify the flow to which the message belongs. For example, at least one of these fields comes from the message header or message body. Users can also define these fields as needed. For example, defining the multiple fields to which the message belongs includes fields used to describe at least one of the following information types: source address, destination address, protocol type, source port number or destination port number, etc., which is the message 5-tuple.

[0052] Based on the type of matching information in flow table entries, current flow table entries generally include precise flow table entries and fuzzy flow table entries. Precise flow table entries use matching information to match multiple fields of a packet; the value of each field indicated by the matching information is an exact value. A single precise flow table entry can be used to match packets in one flow. Fuzzy flow table entries, on the other hand, indicate wildcard values ​​for some fields, where a wildcard represents any value. A single fuzzy flow table entry can be used to match packets in multiple flows.

[0053] Continue to refer to Figure 5 The VM's virtual network adapter enables multiple queues, each corresponding to... Figure 5 The data plane device set shown depicts a smart network interface card (NIC). When a packet needs to be sent in the VM, the NIC selected for packet forwarding is determined by choosing a queue. In the specific implementation, received packets are assigned to different queues according to certain rules, such as source address, destination address, and protocol type. For example, a hash operation can be performed on the source address, destination address, and protocol type to obtain a hash value. Selecting a queue based on this hash value ensures that packets of the same data flow are assigned to the same queue, meaning the same data flow will choose the same smart NIC. A "same data flow" indicates a series of packets carrying the same matching information. Specifically, the matching information of packets within the same data flow can match the matching field of the corresponding flow table entry for that data flow.

[0054] Virtual machine switching typically uses an overlay network. This involves encapsulating the original Layer 2 packets from the source host within the existing network using tunneling encapsulation, then transparently transmitting them. Upon arrival at the destination, the packets are decapsulated to obtain the original packets, which are then forwarded to the target host, thus enabling Layer 2 communication between hosts. The underlying physical network corresponding to the overlay network is called the underlay network. Communication between the overlay and underlay networks is achieved through... Figure 2 The smart network interface card shown is used for communication. Overlay networks support various network protocols and standards, such as Virtual Extended Local Area Network (VXLAN) and Network Virtualization using Generic Routing Encapsulation (NVGRE). Here, we take VXLAN as an example. Figure 2 The smart network interface card shown can be called an edge device of a VXLAN network, used to process VXLAN packets, including tunnel encapsulation and tunnel decapsulation.

[0055] Please see Figure 5 Assuming that VM2 sends message 1 (which refers to the first message) to VM1, after VM2 determines the smart network interface card A through a queue, it sends message 1 to the smart network interface card A. This step is marked as step ①. After receiving message 1, the smart network interface card A needs to encapsulate message 1 through a tunnel and transmit the encapsulated message through the tunnel. After reaching the destination smart network interface card 1, the smart network interface card 1 decapsulates the received message to obtain the original message 1 and forwards message 1 to VM1.

[0056] In this embodiment, the type of message received by the smart network card is not limited. For example, the received message may be from an audio / video application, a game application, or an instant messaging application. As an example, message 1 includes a message header and a message body. The message header includes an IP header and a TCP header, which includes the source IP address, destination IP address, protocol type, source port number, and destination port number. The source address refers to the IP address of the source device that sends the message, the destination address refers to the IP address of the destination device that receives the message, the protocol type is the communication protocol used when forwarding the message, such as Transmission Control Protocol (TCP) or User Datagram Protocol (UDP), the source port number is the port number used by the source device when sending the message, and the destination port number is the port number used by the destination device when receiving the message. Typically, the source IP address, destination IP address, and protocol type of a message are called the message's 3-tuple, while the source IP address, destination IP address, protocol type, source port number, and destination port number are called the message's 5-tuple. Here, the 3-tuple for message 1 could be the source IP address IP2, destination IP address IP1, and protocol type TCP. The 5-tuple for message 1 could be the source IP address IP2, destination IP address IP1, protocol type TCP, source port number 2, and destination port number 49.

[0057] After receiving packet 1, the smart network interface card A parses packet 1 to obtain the 3-tuple of packet 1. Then, it queries the flow table based on the 3-tuple of packet 1. In some optional implementations, the flow table can also be queried based on the 5-tuple of the packet.

[0058] If the flow table lookup for smart NIC A does not find the packet, then the control plane CPU needs to process packet 1 to determine its forwarding strategy. In the specific implementation, a mapping table is maintained on the smart NIC, which includes the correspondence between hash values ​​and control plane CPU identifiers. If smart NIC A fails to query the flow table, it can perform a hash operation on the 3-tuple of packet 1 to obtain a hash value. Based on this hash value and the correspondence between the hash value and the control plane CPU identifier, the corresponding control plane CPU identifier is determined, thus identifying the control plane CPU corresponding to that hash value. Here, control plane CPU A is used as an example. In some optional implementations, a hash operation can also be performed on the 5-tuple of packet 1.

[0059] In this embodiment of the application, the first control plane device can be determined from the first control plane device set by the 3-tuple or 5-tuple of the message, thereby enabling load-balanced selection of the control plane device.

[0060] In the specific implementation process, the message 1 sent by VM2 is called the forward message, and the message returned by VM1 based on the received message 1 is called the directional message or the reverse message. In order to ensure that both the forward message and the directional message select the same control plane CPU, a 3-tuple can be used, that is, a hash operation is performed using the source IP address, the destination IP address, and the protocol type. As an example, the value corresponding to the source IP address and the value corresponding to the destination IP address are XORed to obtain the first operation result. The first operation result is XORed with the value corresponding to the protocol type to obtain the hash value. In this case, the source IP address IP_VM2 and the destination IP address IP_VM1 are XORed to obtain the first operation result. Then, the first operation result is XORed with the protocol type TCP. In some possible implementations, a 5-tuple is used, that is, a hash algorithm is performed using the source IP address, destination IP address, source port, destination port, and protocol type. For example, the value corresponding to the source IP address is XORed with the value corresponding to the destination IP address to obtain the first result; the value corresponding to the source port number is XORed with the value corresponding to the destination port number to obtain the second result; then the first result, the second result, and the value corresponding to the protocol type are XORed to obtain the hash value. This hash algorithm ensures that forward and reverse packets select the same control plane CPU, meaning packets belonging to the same data stream will select the same control plane CPU, thus guaranteeing normal transmission in the reverse direction. In the embodiments of this application, the above hash algorithm can be referred to as a symmetric hash algorithm or a forward / reverse independent hash algorithm.

[0061] Taking the determination of CPUA by smart NIC A as an example, smart NIC A sends packet 1 and its identifier to CPUA. This step is marked as step ②. CPUA generates flow table A (flow table A refers to the first flow table) based on the pre-set routing table and the received packet 1 and the identifier of smart NIC A. The matching information included in flow table A can be at least one or more combinations of source IP address IP_VM2, destination IP address IP_VM1, protocol type TCP, source port number 2, or destination port number, or it can also include other information, such as the identifier of smart NIC A. The instruction information at least includes the IP address of the next hop of packet 1. In some possible implementations, flow table A may also include other flow table entries, such as the IP address of CPUA, IP_CPUA. It should be noted that the pre-set routing table is stored in the memory of the host or server where CPUA is located. When a flow table needs to be built, CPUA retrieves it from the memory. Similarly, the flow table built by CPUA can also be stored in the memory of the host or server where CPUA is located. Of course, in some possible implementations, the flow table established by CPUA can also be temporarily stored in the data register of CPUA.

[0062] Furthermore, after generating flow table A, CPUA performs tunnel encapsulation on packet 1, marking this step as step ③. Continuing with the example above, this involves adding a VXLAN header and a UDP header to the original packet 1. The UDP header, along with the original packet, serves as UDP data. In the UDP header, the destination port number is fixed at 4789, and the source port number is the value calculated from the original packet 1 using a hash algorithm. Further encapsulation is performed on an outer IP layer. The source IP address is the IP address of the VTEP to which the source VM belongs, which is the IP address of smart network interface card A. The destination IP address is the IP address of the VTEP to which the destination VM belongs, which is the IP address of the smart network interface card receiving packet 1. Additionally, an outer Ethernet header is encapsulated. The source MAC address is the MAC address of the VTEP to which the source VM belongs, and the destination MAC address is the MAC address of the VTEP to which the packet arrives. The packet after tunnel encapsulation is referred to as packet 2.

[0063] CPUA sends packet 2 and its forwarding information to smart NIC A. This step is marked as step ④. Smart NIC A then forwards packet 2 to the next-hop device. Taking smart NIC 1 as an example, this step is marked as step ⑤. Simultaneously with sending packet 2 to smart NIC A, CPUA also sends flow table A to smart NIC A. In some other examples, flow table A may be sent to smart NIC A after CPUA sends packet 2. In some possible implementations, after generating flow table A, CPUA can send flow table A and packet 1 to smart NIC A simultaneously. Smart NIC A processes packet 1 according to flow table A to obtain the tunnel-encapsulated packet 2, and then forwards packet 2 to the next-hop device, such as smart NIC 1.

[0064] It should be noted here that while generating flow table A, CPU A can also generate flow table B and send flow table B to smart NIC A. If flow table A is called forward flow table A, then flow table B can be called the reverse flow table B corresponding to flow table A. In this way, when smart NIC A receives packets from the peer (smart NIC 1), it can directly query the reverse flow table B to forward the packets without the need for control plane CPU intervention, thereby speeding up packet processing efficiency.

[0065] After receiving packet 2, Smart NIC 1 decapsulates packet 2 to obtain packet 1. It then parses packet 1 to obtain its 5-tuple or 3-tuple. This step is marked as step ⑥. Similarly, Smart NIC 1 can query the flow table based on the parsed 3-tuple. If the query fails, it indicates that the control plane CPU needs to determine the forwarding strategy for the received packet 1. The method by which Smart NIC 1 determines the control plane CPU is the same as that by Smart NIC A, and will not be repeated here. Here, we take CPU1 (CPU1 refers to the second control plane device) as an example. After identifying CPU1, Smart NIC 1 uploads packet 1 and the identifier of Smart NIC 1 to CPU1. This step is marked as step ⑦.

[0066] CPU1 parses the received packet 1, obtaining either a 3-tuple or a 5-tuple. Based on this 3-tuple or 5-tuple, control plane CPU1 determines that forwarding packet 1 requires Network Address Translation (NAT). When NAT is needed, the source IP address of packet 1 needs to change. This would cause forward and indirect packets to select different control plane CPUs. To ensure that forward and indirect packets select the same control plane CPU during NAT, a new hash operation is performed based on the NAT-translated 3-tuple or 5-tuple. The resulting hash value is then used to reselect the control plane CPU. In the specific implementation, the source address IP_VM2 is translated using NAT to obtain the translated source address IP_VM2-1. This yields a new 5-tuple, including the source address IP_VM2-1, the destination address IP_VM1, TCP, source port number 2, and destination port number 49. A hash operation is performed on this new 5-tuple to obtain a hash value. The control plane CPU is then re-determined based on this hash value. For example, if the control plane CPU is control plane CPU2 (CPU2 refers to the third control plane device), in some possible implementations, the control plane CPU can also be re-selected based on the new 3-tuple obtained after network address translation.

[0067] After determining the new control plane CPU, CPU1 forwards packet 1, the identifier of smart NIC 1, and the address IP_A of smart NIC A to CPU2. This step is marked as step ⑧. Upon receiving packet 1, CPU2 creates flow table C, also known as the second flow table. The second flow table includes at least the source tunnel IP address of packet 1, which is the IP address of smart NIC A, so that CPU2 can determine the destination tunnel IP address of the packet originating from the second virtual machine based on the IP address of smart NIC A. In some possible implementations, the second flow table also includes the identifier of smart NIC 1 to indicate that the smart NIC processing the data flow to which packet 1 belongs is smart NIC 1.

[0068] After CPU2 creates flow table C, it sends flow table C to smart NIC 1. This step is marked as step 9. Smart NIC 1 then forwards packet 1 to the next-hop device, which is the destination device VM1. This step is marked as step 10. In the specific implementation, CPU2 can also create flow table D at the same time as creating flow table C. Flow table C is called the forward flow table C, and flow table D can be called the reverse flow table D (the reverse flow table refers to the third flow table). The reverse flow table D is sent to smart NIC 1 so that when smart NIC 1 receives a reverse packet from the other end, that is, VM1, it can directly query the reverse flow table D and forward the packet without the intervention of the control plane CPU, thereby improving the packet processing efficiency.

[0069] Please see Figure 6 After VM1 receives packet 1 forwarded by smart NIC 1, it generates a reverse packet / direction packet based on packet 1. This reverse packet is referred to as packet 3 (packet 3 refers to the second packet). Packet 3 also includes a header and a body. The header includes the source IP address IP_VM1, the destination IP address IP_VM2, the protocol type, the source port number, and the destination port number. VM1 also selects a smart NIC through a selection queue. Normally, packets initiated by VM are sent to the same smart NIC, but the smart NIC that initiates the packet and the smart NIC that receives the packet are not the same. That is, the smart NIC that sends packet 1 to VM1 is different from the smart NIC selected by VM1 when sending the packet. Here, the smart NIC that sends packet 1 to VM1 is smart NIC 1, while the smart NIC selected by VM1 when initiating the packet is smart NIC 2, which is different from smart NIC 1. Therefore, the step of VM1 sending packet 3 to smart NIC 2 is marked as step ①.

[0070] In this scenario, since Smart NIC 1 performs NAT translation when forwarding packet 1 to VM1, NAT address translation is also required when VM1 sends a packet to Smart NIC 2, converting the destination address IP_VM2 to IP_VM2-1. The resulting 5-tuple after NAT translation contains the source IP address IP_VM1, the destination IP address IP_VM2-1, the protocol type, the source port number, and the destination port number. This is identical to the 5-tuple information obtained after NAT by CPU1. Therefore, based on the forward / reverse independent hash algorithm described above, Smart NIC 2 can select the same control plane CPU as VM1 when sending packet 1, which is CPU2.

[0071] After identifying CPU2, Smart NIC 2 sends packet 3 and its identifier to CPU2. This step is marked as step ②. When CPU2 queries the flow table, since the established flow table C already records that the Smart NIC used to process the flow to which packet 3 belongs is Smart NIC 1 (meaning the network card previously processed by the data flow to which packet 3 belongs was Smart NIC 1, not Smart NIC 2), and a corresponding flow table D already exists on Smart NIC 1, CPU2 sends flow table D to Smart NIC 2. This step is marked as step ③. CPU2 then deletes flow table D from Smart NIC 1. This ensures that packets sent by VM1 can select the same Smart NIC.

[0072] After receiving flow table D, smart NIC 2 encapsulates packet 3 using tunnel encapsulation based on flow table D, resulting in tunnel-encapsulated packet 4. Packet 4 is then forwarded to the next-hop device, namely smart NIC A. This step is marked as step ④. After receiving packet 4, smart NIC A decapsulates packet 4, obtaining packet 3. It then queries the flow table matching packet 3, specifically flow table B described above. Based on flow table B, it determines that the next-hop device for packet 3 is VM2, and forwards packet 3 to VM2. This step is marked as step ⑤.

[0073] Simultaneously, smart NIC A queries the flow table and discovers a change in the tunnel's exit destination IP address. Specifically, the exit destination IP address recorded in flow table A was the IP address of smart NIC 1, but it has now changed to the IP address of smart NIC 2. In this case, smart NIC A updates flow table A. In some possible implementations, smart NIC A uploads the update information to CPUA, which then refreshes flow table A and distributes the refreshed flow table A back to smart NIC A. This ensures that the link establishment process is performed entirely by control plane devices, while data plane devices are only used for packet forwarding, thereby improving packet processing efficiency.

[0074] Furthermore, in this embodiment, a flow table entry can be added to the flow table. This entry is used to identify whether a hit flow table needs to be uploaded to the control plane CPU for processing, such as updating flow table A. This flow table entry can be deleted after the link is established.

[0075] In this embodiment of the application, after VM1 generates a reverse packet based on the received packet, if the smart network card selected by VM1 is different from the network card that sent the packet to VM1, the tunnel exit IP address in flow table A is refreshed after smart network card A receives the reverse packet. This can ensure that the forward packet and the reverse packet can select the same smart network card, thereby ensuring the normal forwarding of the reverse packet.

[0076] In the specific implementation, when VM2 sends packet 1 to VM1, VM1 receives packet 1 and returns packet 3 to VM2. Once VM2 receives this packet, it indicates that the link between VM2 and VM1 is established. Subsequently, when VM2 sends packets to VM1, it can directly forward the packets based on the established flow table on the smart network interface card, without needing processing by the control plane CPU. Similarly, when VM1 sends packets to VM2, it also does not need to go through the control plane CPU, thus improving packet forwarding efficiency.

[0077] Secondly, embodiments of this application also provide a message processing method, which is executed by a data plane device, the data plane device being... Figure 5The smart network interface card A shown in the figure, the functional or technical effects produced by the smart network interface card A in the process of executing the message processing method can be referred to the functional or technical effects brought about in the first aspect.

[0078] Thirdly, please see Figure 7 This application also provides a data plane device applied to a message processing system, capable of implementing the functions of the message processing system in the first aspect of this application. The data plane device can be a hardware structure, a software unit, or a hardware structure plus a software unit. The data plane device may include a receiving module 701, a parsing module 702, and a determining module 703, wherein the receiving module 701 is used to execute... Figure 5 Step ①; the parsing module 702 can execute. Figure 5 After step ①, message 1 is parsed to obtain the first characteristic information of message 1; the determination module 703 can execute. Figure 5 After parsing and obtaining the first feature information of message 1, the first control plane device is determined based on the first feature information, that is, the control plane CPUA is determined. All relevant content of each step involved in the method embodiment provided in the first aspect above can be referenced from the functional description of the corresponding functional unit, and will not be repeated here.

[0079] The unit division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into a single processor, exist as separate physical units, or be integrated into a single unit. The integrated units can be implemented in hardware or as software functional units.

[0080] Fourthly, embodiments of this application also provide a computer-readable storage medium, including instructions that, when executed on a computer, cause the computer to perform the message processing method as described in the second aspect above.

[0081] Fifthly, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to perform the message processing method as described in the second aspect above.

[0082] The methods provided in this application can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium, etc.

[0083] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A message processing system, characterized in that, include: A first virtual machine and a first data plane device, wherein the first virtual machine runs on a first host; the first data plane device is used for: Receive the first message sent by the first virtual machine; Parse the first message to obtain the first feature information of the first message; A first control plane device is determined based on the first feature information, wherein the first control plane device is used to execute the control plane function of the first open source virtual switch (OVS) of the first virtual machine, the first control plane device is located in a first control plane set, and the first control plane device set is located outside the first host.

2. The system according to claim 1, characterized in that, The first data plane device is one of the data plane devices in a first data plane device set located outside the first host. The first data plane device is used to perform the data plane function of the first OVS.

3. The system according to claim 1, characterized in that, The first feature information includes at least a first source international protocol IP address, a first destination IP address, and a protocol type; when the first data plane device determines the first control plane device based on the first feature information, it is specifically used for: A hash operation is performed on the value corresponding to the first source IP address, the value corresponding to the first destination IP address, and the value corresponding to the protocol type to obtain a hash value. The hash operation ensures that the calculation result is the same when the first source IP address is used as the second destination IP address and when the first destination IP address is used as the second source IP address. Based on the correspondence between the hash value and the first control plane device, the first control plane device corresponding to the hash value is determined.

4. The system according to claim 1, characterized in that, The first control plane device is also used for: Based on the first message, determine the forwarding strategy for the first message; A first flow table is established based on the forwarding policy and the identifier of the first data plane device. The first flow table includes at least the forwarding policy, the identifier of the first data plane device, and the identifier of the first control plane device. The first flow table is sent to the first data plane device so that the first data plane device forwards the first packet to the next-hop device according to the first flow table. The next-hop device is a second data plane device in the second data plane device set, which is located outside the second host. The second data plane device is used to execute the second OVS data plane function of the second virtual machine, which runs in the second host.

5. The system according to claim 4, characterized in that, The second data plane device is used for: Based on the first feature information, a second control plane device is determined from the second control plane device set, the second control plane device set being located outside the second host, and the second control plane device being used to execute the control plane function of the second OVS; The second control plane device is configured to, when it is determined that network address translation is required to forward the first packet from the second data plane device to the next hop, perform network address translation based on the first feature information to obtain second feature information; and determine a third control plane device from the set of second control plane devices based on the second feature information.

6. The system according to claim 5, characterized in that, The third control plane device is used for: A second flow table is established, which includes the source tunnel IP address of the first packet, so that the third control plane device can determine the destination tunnel IP address of the packet sent from the second virtual machine based on the source tunnel IP address.

7. The system according to claim 6, characterized in that, The third control plane device is also used for: Receive a second message, which is a return message generated by the second virtual machine based on the received first message; Based on the second flow table, a third flow table matching the second message is stored in the second data plane device; The third flow table is sent to the third data plane device so that the third data plane device forwards the second message according to the third flow table. The third data plane device is a device that receives the second message sent by the second virtual machine and is located in the set of second data plane devices.

8. The system according to claim 7, characterized in that, The first data plane device is also used for: After receiving the second message, confirm whether the data plane device that sent the second message is the same data plane device that received the first message; If they are different, the identifier of the second data plane device in the first flow table is changed to the identifier of the third data plane device.

9. A data plane device, characterized in that, include: An interface for receiving a first message sent by a first virtual machine, which runs on a first host; The processor is configured to parse the first packet, obtain first feature information of the first packet, and determine a first control plane device from a first control plane device set based on the first feature information, wherein the first control plane device is configured to execute the control plane function of the first open source virtual switch (OVS) of the first virtual machine, and the first control plane device set is located outside the first host.

10. The data plane device according to claim 9, characterized in that, The data plane device is the first data plane device in the first data plane device set, which is located outside the first host. The first data plane device is used to perform the data plane function of the first OVS.

11. The data plane device according to claim 10, characterized in that, The first feature information includes at least a first source international protocol IP address, a first destination IP address, and a protocol type; when the processor determines a control plane device from the first control plane device set based on the first feature information, it is specifically used for: A hash operation is performed on the value corresponding to the first source IP address, the value corresponding to the first destination IP address, and the value corresponding to the protocol type to obtain a hash value. The hash operation ensures that the calculation result is the same when the first source IP address is used as the second destination IP address and when the first destination IP address is used as the second source IP address. Based on the correspondence between the hash value and the control plane device, the control plane device corresponding to the hash value is determined.

12. The data plane device according to claim 11, characterized in that, The data plane device is also used for: Receive a second message, determine whether the third data plane device that sent the second message is the second data plane device that received the first message at the last hop, the second message is a return message generated by the second virtual machine based on the received first message, the second virtual machine runs in the second host, wherein the third data plane device and the second data plane device are located in the second data plane device set, and the second data plane device set is located outside the second host; If they are different, the identifier of the second data plane device in the flow table is adjusted to the identifier of the third data plane device. The flow table is used to instruct the data plane device to forward the first packet to the next-hop device.

Images