Message processing method for multiple NUMA units and network equipment

Abstract

The invention provides a message processing method for multiple NUMA units and network equipment, and the method comprises the steps: receiving a first message sent by an EP node of a PCIE system, obtaining the first feature information of the first message, determining a target EP for sending the first message through the first feature information, and sending the first message to the target EP. And obtaining a target NUMA unit according to a matching result of the target EP and the corresponding relationship between the EP and the NUMA unit, and sending the first message to the target NUMA unit. Through the method, the situation of cross-NUMA unit communication can be effectively avoided, and the service processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a message processing method and network device for multiple NUMA units. Background Art

[0002] Network equipment has been widely used in all walks of life to achieve interconnection within various backbone networks, between backbone networks, and between backbone networks and the Internet. The PCI Express bus provides a high-speed, high-performance, full-duplex point-to-point, multi-channel bundled differential link connection universal I / O interconnect bus for network device interconnection. The main features of the PCI Express bus, such as application model, storage structure, software interface, etc. are consistent with the traditional PCI / PCI-X bus, and make full use of advanced point-to-point interconnection, switching and data packet-based protocols to achieve new bus performance and characteristics. Since the PCI-Express bus is a point-to-point switching bus architecture. Each device in the system has its own bus, and point-to-point communication is established by directly connecting to the Switch device.

[0003] Data transfer between PCI Express devices is organized in transactions. The root complex can send transactions to endpoints, endpoints can initiate transactions to the root complex, and endpoints can initiate transactions to another endpoint. The execution or completion of a transaction is implemented by sending and receiving transaction layer packers (TLPs). A transaction is defined as a sequence of one or more packets required to transfer certain information between a requester and a completer.

[0004] With the development of hardware technology and chip design technology, PCIE speed and CPU number are increasing. The bottleneck of network equipment in processing messages is mainly the reading rate of CPU, memory and PCIE system data interaction. Non-uniform memory access (NUMA) is a memory access design for multiple processors. The memory access efficiency depends on the memory location of the processor. In network devices with multi-core CPUs, there are often access messages between NUMAs. However, the efficiency of access messages between NUMAs is low, which affects the efficiency of business processing. Summary of the invention

[0005] In order to overcome the problems existing in the related art, this specification provides a message processing method and network device for multiple NUMA units.

[0006] According to a first aspect of an embodiment of this specification, a method for processing a message of a multi-NUMA unit is provided, the method comprising:

[0007] Receive a first message sent by an EP node of a PCIE system, and obtain first characteristic information of the first message;

[0008] Determine a target EP to send the first message by using the first characteristic information, and obtain a target NUMA unit according to a matching result between the target EP and a correspondence between the EP and the NUMA unit, wherein the target NUMA unit is connected to the target EP;

[0009] The first message is sent to the target NUMA unit.

[0010] The obtaining of first characteristic information of the first message includes:

[0011] Acquire quintuple information and / or message attribute information of the first message.

[0012] The target NUMA unit is obtained according to the matching result of the target EP and the correspondence between the EP and the NUMA unit, including:

[0013] Performing a hash operation on the first characteristic information to obtain a first hash value;

[0014] The target NUMA unit is obtained by using the matching result of the first hash value and the correspondence between the EP and the NUMA unit.

[0015] The target NUMA unit is obtained according to the matching result of the target EP and the correspondence between the EP and the NUMA unit, including:

[0016] When it is identified according to the first characteristic information that the message size of the first message is greater than a preset value, multiple target NUMA units are acquired, and the first message is sent to the multiple target NUMA units in an evenly distributed manner.

[0017] The sending of the first message to the target NUMA unit includes:

[0018] The first message is linearly addressed and converted into an address space of a target NUMA unit, and the first message is loaded into a memory in the target NUMA unit through the NUMA unit.

[0019] Wherein, the method further comprises:

[0020] Monitor the operating status of each NUMA unit;

[0021] When it is found that the second NUMA unit has a fault, the second NUMA unit is removed from the correspondence between the EP and the NUMA unit.

[0022] It can be seen from the above embodiments that by obtaining the first characteristic information of the first message and determining the target NUMA unit corresponding to (connected to) the target EP sending the first message based on the first characteristic information, cross-NUMA unit communication can be effectively avoided, thereby improving business processing efficiency.

[0023] According to a second aspect of an embodiment of this specification, a network device is provided, the network device comprising a plurality of non-uniform memory access NUMA units, the network device comprising:

[0024] A receiving module, used for receiving a first message sent by an EP node of a PCIE system, and acquiring first characteristic information of the first message;

[0025] a processing module, configured to determine a target EP to which the first message is sent by using the first characteristic information, and obtain a target NUMA unit according to a matching result between the target EP and a correspondence between the EP and the NUMA unit, wherein the target NUMA unit is connected to the target EP;

[0026] A sending module is used to send the first message to a target NUMA unit.

[0027] The receiving module is specifically used to obtain the quintuple information and / or message attribute information of the first message.

[0028] The processing module is specifically used to perform a hash operation on the first feature information to obtain a first hash value, and obtain the target NUMA unit using a matching result of the first hash value with the corresponding relationship between the EP and the NUMA unit.

[0029] The processing module is specifically used to obtain multiple target NUMA units and send the first message to the multiple target NUMA units in an evenly distributed manner when it is identified according to the first characteristic information that the message size of the first message is greater than a preset value.

[0030] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0032] Figure 1 This is a schematic diagram of message forwarding in a PCIE system of a non-NUMA network device according to an exemplary embodiment of the present specification.

[0033] Figure 2This is a schematic diagram of message forwarding in a PCIE system of a NUMA network device according to an exemplary embodiment of the present specification.

[0034] Figure 3 It is a flowchart of a message processing method for multiple NUMA units shown in this specification according to an exemplary embodiment.

[0035] Figure 4 This is a schematic diagram of message forwarding in a PCIE system of a NUMA network device according to an exemplary embodiment of the present specification.

[0036] Figure 5 This is a schematic diagram of message forwarding in a PCIE system of a NUMA network device according to an exemplary embodiment of the present specification. DETAILED DESCRIPTION

[0037] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0038] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0039] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0040] like Figure 1 The diagram shows a message forwarding diagram of a non-NUMA network device PCIE system. In this architecture, more transceiver modules can be supported under the condition of effective computing unit resources, the device transceiver capability can be expanded, the forwarding performance of the network device can be improved, and the flexible expansion of the transceiver modules can be enhanced.

[0041] In this example, the received message is stored in the memory through the computing unit PCIE message processing module and the memory controller for computing unit CPU message processing, and the processed message to be forwarded is transferred from the memory to the PCIE endpoint transceiver module for forwarding through the DMA controller.

[0042] However, in this implementation, when the computing unit processes a message, it needs to traverse the entire memory management linked list to obtain the corresponding data. This implementation greatly increases the overhead of the computing unit and affects the forwarding efficiency of the message.

[0043] In the distributed network device computing unit supporting NUMA function instance, the NUMA computing unit CPU reads and writes the memory space mapped by the unit with high efficiency. This design scheme can improve the efficiency of multi-computing units in processing messages. The specific implementation diagram of NUMA network device PCIE system message forwarding is shown.

[0044] like Figure 2 As shown, the message transceiver module is aggregated through the bridge chip and bound to the NUMA computing unit module. The performance of each NUMA computing unit CPU accessing the local memory is high, but the data of other computing units can only be read and written through the QPI, a fast bus channel for interaction between CPUs, which has low performance and affects the business processing efficiency.

[0045] To solve the above technical problems, the present disclosure provides a method for processing packets in multiple NUMA units. Figure 3 As shown, the method includes:

[0046] S301 receives a first message sent by an EP node of a PCIE system, and obtains first characteristic information of the first message;

[0047] S302 determines a target EP for sending the first message by using the first characteristic information, and obtains a target NUMA unit according to a matching result between the target EP and a correspondence between the EP and the NUMA unit, wherein the target NUMA unit is connected to the target EP;

[0048] S303 sends the first message to the target NUMA unit.

[0049] In this embodiment, the solution in the present disclosure can be implemented by setting a message identification module and a message mapping module between the NUMA unit and the EP (PCIE EP).

[0050] The administrator can configure the mapping relationship between each EP and each NUMA unit in the message mapping module, such as Figure 4 As shown in , the PCIE node 1 (EP1) can be configured to correspond to NUMA1, the PCIE node 2 (EP2) can correspond to NUMA2, and the PCIE node N can correspond to NUMAN.

[0051] In this embodiment, after receiving the first message sent by the EP node of the PCIE system, the first message is parsed to obtain first characteristic information of the first message, wherein the first characteristic information may be five-tuple information and / or message attribute information of the first message.

[0052] When it is quintuple information of the first message, the EP node of the PCIE that sends the first message can be identified according to the quintuple information of the first message.

[0053] like Figure 4 As shown, the message identification module obtains the first message and obtains the five-tuple information of the first message, and identifies the PICE node sending the first message as PCIE node 1 or PCIE node 2 through the five-tuple information.

[0054] In another embodiment, it is also possible to identify whether the size of the first message is greater than a preset value by identifying the message attribute information of the first message. For example, when the size of the first message is greater than the preset value, the message can be considered to be an "elephant" message.

[0055] When the message is an "elephant" message, the first message can be sent to multiple target NUMA units in an evenly distributed manner to reduce the situation where a certain NUMA unit has insufficient processing resources due to processing an "elephant" message.

[0056] like Figure 5 As shown, in another embodiment, message identification, flow control, NUMA mapping, address space management, PCIE system link detection and message fast forwarding modules can also be configured.

[0057] Among them, the message identification module and the traffic control module mainly identify the messages sent by the PCIE system endpoint and obtain the mappable NUMA computing unit to establish a mapping relationship, and send the message to the corresponding NUMA unit for processing the message; the PCIE link detection module mainly detects the link status between the NUMA system and the PCIE endpoint, and the RC and the network device PCIE EP node, and isolates the abnormal links; the NUMA mapping management is mainly used to establish a mapping relationship between the PCIE system port PCIE node and the computing unit PCIE RC, which is used to guide the NUMA unit sent by the message sending and receiving port. The mapping relationship is mainly based on the hash mapping of the ordinary message quintuple, and the elephant traffic with large bandwidth and long duration is established. The elephant traffic sending table entry is mainly used to evenly distribute NUMA, and the computing unit NUMA table entry is established. The table entry needs to be aged to avoid the imbalance of NUMA computing unit processing elephant traffic; the address management space is mainly used to manage and linearly map the computing unit and the transceiver unit message address conversion relationship.

[0058] In this embodiment, the message receiving module sends the message to the message identification module, and according to the five-tuple, the protocol attributes of the message and the mappable NUMA computing unit, the computing module to which the message is sent directly converts the linear address of the message into the address space of the corresponding computing unit according to the corresponding conversion relationship recorded in the address space mapping, and loads the message into the corresponding local memory through the computing unit memory controller. Similarly, after the NUMA computing unit processes the message, it transmits it to the NUMA forwarding PCIE message node index through the memory controller, and then sends it to the message sending unit through the RC node through linear mapping.

[0059] It can be seen from the above embodiments that a message mapping relationship is established based on the number of NUMA computing units of the network device and the message quintuple. In particular, for elephant traffic messages, they are no longer sent to the NUMA computing unit based on the message quintuple. Instead, a NUMA computing unit large traffic balancing mapping table entry is established to balance the sending of unreachable elephant messages to different NUMA units, and the table entry is recorded.

[0060] Through the embodiments provided by the present disclosure, a mapping relationship between the PCIE system transceiver endpoints and the NUMA computing units can be established according to the message attributes and characteristics, thereby avoiding the problem of low efficiency in mutual access between different memory spaces of different NUMA computing units to a great extent, thereby improving the forwarding efficiency of network devices.

[0061] For the time or usage scenarios when the bandwidth of the whole machine is not high, the operation status of some computing units is dynamically adjusted to idle, and the mapping relationship of the message mapping computing unit is adjusted synchronously. The idle computing unit does not receive and process the message temporarily, saving the power consumption of the whole network equipment. In addition, for ordinary and simple forwarding messages, a direct mapping relationship between the PCIE transceiver unit node is established, and the table items issued by the computing unit are used to guide different PCIE nodes to quickly forward messages.

[0062] Based on the above method embodiments, the embodiment of the present disclosure further provides a network device, wherein the network device includes a plurality of non-uniform memory access NUMA units, and the network device includes:

[0063] A receiving module, used for receiving a first message sent by an EP node of a PCIE system, and acquiring first characteristic information of the first message;

[0064] a processing module, configured to determine a target EP to which the first message is sent by using the first characteristic information, and obtain a target NUMA unit according to a matching result between the target EP and a correspondence between the EP and the NUMA unit, wherein the target NUMA unit is connected to the target EP;

[0065] A sending module is used to send the first message to a target NUMA unit.

[0066] The receiving module is specifically used to obtain the quintuple information and / or message attribute information of the first message.

[0067] The processing module is specifically used to perform a hash operation on the first feature information to obtain a first hash value, and obtain the target NUMA unit using a matching result of the first hash value with the corresponding relationship between the EP and the NUMA unit.

[0068] The processing module is specifically used to obtain multiple target NUMA units and send the first message to the multiple target NUMA units in an evenly distributed manner when it is identified according to the first characteristic information that the message size of the first message is greater than a preset value.

[0069] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0070] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0071] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.

[0072] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0073] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A message processing method for multiple NUMA units, characterized in that: The method comprises: Receive a first message sent by an EP node of a PCIE system, and obtain first characteristic information of the first message; Determine a target EP to send the first message by using the first characteristic information, and obtain a target NUMA unit according to a matching result between the target EP and a correspondence between the EP and the NUMA unit, wherein the target NUMA unit is connected to the target EP; The first message is sent to the target NUMA unit.

2. The method according to claim 1, characterized in that The obtaining of first characteristic information of the first message includes: Acquire quintuple information and / or message attribute information of the first message.

3. The method according to claim 1, characterized in that The target NUMA unit is obtained according to the matching result of the target EP and the correspondence between the EP and the NUMA unit, including: Performing a hash operation on the first characteristic information to obtain a first hash value; The target NUMA unit is obtained by using the matching result of the first hash value and the correspondence between the EP and the NUMA unit.

4. The method according to claim 1, characterized in that: The target NUMA unit is obtained according to the matching result of the target EP and the correspondence between the EP and the NUMA unit, including: When it is identified according to the first characteristic information that the message size of the first message is greater than a preset value, multiple target NUMA units are acquired, and the first message is sent to the multiple target NUMA units in an evenly distributed manner.

5. The method according to claim 1, characterized in that The sending the first message to the target NUMA unit includes: The first message is linearly addressed and converted into an address space of a target NUMA unit, and the first message is loaded into a memory in the target NUMA unit through the NUMA unit.

6. The method according to claim 1, characterized in that The method further comprises: Monitor the operating status of each NUMA unit; When it is found that the second NUMA unit has a fault, the second NUMA unit is removed from the correspondence between the EP and the NUMA unit.

7. A network device, characterized in that: The network device includes a plurality of non-uniform memory access NUMA units, and the network device includes: A receiving module, used for receiving a first message sent by an EP node of a PCIE system, and acquiring first characteristic information of the first message; a processing module, configured to determine a target EP to which the first message is sent by using the first characteristic information, and obtain a target NUMA unit according to a matching result between the target EP and a correspondence between the EP and the NUMA unit, wherein the target NUMA unit is connected to the target EP; A sending module is used to send the first message to a target NUMA unit.

8. The network device according to claim 7, characterized in that: The receiving module is specifically used to obtain the quintuple information and / or message attribute information of the first message.

9. The network device according to claim 7, characterized in that: The processing module is specifically configured to perform a hash operation on the first feature information to obtain a first hash value, and obtain a target NUMA unit using a matching result of the first hash value with a corresponding relationship between the EP and the NUMA unit.

10. The network device according to claim 7, characterized in that: The processing module is specifically used to obtain multiple target NUMA units and send the first message to the multiple target NUMA units in an evenly distributed manner when it is identified according to the first characteristic information that the message size of the first message is greater than a preset value.

Images