A message processing method and device

Abstract

Embodiments of the present application provide a packet processing method and device, and relate to the technical field of IPFIX, and are applied to each CPU data core included in a network device, each CPU data core corresponding to a different flow record table, the method comprising: obtaining a plurality of network packets; using a CPU pipeline to batch process the plurality of network packets, and updating a flow record corresponding to a service flow to which each network packet belongs in the flow record table corresponding to each CPU data core; and sending an IPFIX packet to a collector, the IPFIX packet comprising the flow record. The scheme can solve the problem that the IPFIX flow monitoring function on the network device is unavailable when facing network traffic with a super large bandwidth capacity.

Description

Technical Field

[0001] This application relates to the field of IPFIX technology, and in particular to a message processing method and apparatus. Background Technology

[0002] In Internet Protocol Flow Information Export (IPFIX) systems, the exporter is typically implemented in software. For example, it samples network packets at a sampling ratio of 1000:1 and reports the sampled network packets to the Central Processing Unit (CPU). The CPU then performs software processing such as parsing, classifying, aggregating, and caching on the network packets to meet the user's traffic monitoring needs.

[0003] However, with the rapid development of the Internet, network devices in the core and backbone networks need to forward traffic in the millions or more, with bandwidth easily reaching the terabit (Tb) level or higher. Currently, the single-core processing performance and number of CPU cores are nearing their physical limits, and significant improvements are unlikely for a considerable period. When facing network traffic volumes of 2Tb or higher, implementing IPFIX traffic monitoring becomes increasingly difficult, rendering this function unusable on these network devices. Summary of the Invention

[0004] The purpose of this application is to provide a message processing method and apparatus to address the unavailability of IPFIX traffic monitoring functionality on network devices when facing network traffic with extremely high bandwidth capacity. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide a packet processing method applied to each CPU data core of a network device, wherein each CPU data core corresponds to a different flow record table. The method includes: acquiring multiple network packets; using a CPU pipeline to batch process the multiple network packets, updating the flow record corresponding to the service flow to which each network packet belongs in the flow record table corresponding to each CPU data core; and sending an IPFIX packet to a collector, wherein the IPFIX packet includes the flow record.

[0006] In some embodiments, the step of acquiring multiple network packets includes: reading a first number of network packets from the cache queue corresponding to the CPU data core; caching the first number of network packets in a cache; and reading the first number of network packets from the cache.

[0007] In some embodiments, during the process of batch processing the plurality of network packets using a CPU pipeline, the method further includes: reading a second number of network packets after the first number of network packets from the cache queue; and caching the second number of network packets into the cache.

[0008] In some embodiments, reading a first number of network packets from the cache queue corresponding to the CPU data core includes: if the number of network packets in the cache queue corresponding to the CPU data core is greater than or equal to a preset value, then reading the preset number of network packets from the cache queue corresponding to the CPU data core; if the number of network packets in the cache queue corresponding to the CPU data core is less than the preset value, then reading all network packets in the cache queue corresponding to the CPU data core.

[0009] In some embodiments, the batch processing includes multiple processing stages, each processing stage includes multiple processing sub-stages, and each processing sub-stage consists of multiple processing logics included in the CPU pipeline;

[0010] The process of using a CPU pipeline to batch process the multiple network packets includes: performing CPU pipeline processing on the multiple network packets according to the order of the multiple processing stages and the order of the multiple processing sub-stages in each processing stage.

[0011] In some embodiments, the network device includes multiple CPU data cores; the multiple CPU data cores satisfy at least one of the following conditions: each CPU data core corresponds to a different cache queue; each CPU data core corresponds to a lock-free circular queue; each CPU data core processes different service flows; each CPU data core corresponds to a different IPFIX thread.

[0012] In some embodiments, the method further includes: receiving an IPFIX service request, the IPFIX service request being used to update the flow record table; and invoking the IPFIX thread corresponding to the CPU data core to update the flow record table.

[0013] Secondly, embodiments of this application provide a message processing apparatus applied to each CPU data core included in a network device, wherein each CPU data core corresponds to a different flow record table, and the apparatus includes:

[0014] The acquisition module is used to acquire multiple network packets;

[0015] The processing module is used to batch process the multiple network packets using a CPU pipeline and update the flow record corresponding to the service flow of each network packet in the flow record table corresponding to each CPU data core.

[0016] The sending module is used to send IPFIX messages to the collector, the IPFIX messages including the flow records.

[0017] In some embodiments, the acquisition module is specifically configured to read a first number of network packets from the cache queue corresponding to the CPU data core; cache the multiple network packets in a cache; and read the first number of network packets from the cache.

[0018] In some embodiments, the acquisition module is further configured to, during the process of batch processing the plurality of network packets using a CPU pipeline, read a second number of new network packets after the first number of network packets from the cache queue; and cache the second number of network packets in the cache.

[0019] In some embodiments, the acquisition module is specifically configured to: if the number of network packets in the cache queue corresponding to the CPU data core is greater than or equal to a preset value, then read the preset value of network packets from the cache queue corresponding to the CPU data core; if the number of network packets in the cache queue corresponding to the CPU data core is less than the preset value, then read all network packets in the cache queue corresponding to the CPU data core.

[0020] In some embodiments, the batch processing includes multiple processing stages, each processing stage includes multiple processing sub-stages, and each processing sub-stage consists of multiple processing logics included in the CPU pipeline; the processing module is specifically used to perform CPU pipeline processing on the multiple network packets in the order of the multiple processing stages and the order of the multiple processing sub-stages in each processing stage.

[0021] In some embodiments, the network device includes multiple CPU data cores; the multiple CPU data cores satisfy at least one of the following conditions: each CPU data core corresponds to a different cache queue; each CPU data core corresponds to a lock-free circular queue; each CPU data core processes different service flows; each CPU data core corresponds to a different IPFIX thread.

[0022] In some embodiments, the apparatus further includes: a receiving module, configured to receive an IPFIX service request, the IPFIX service request being used to update the flow record table; and a calling module, configured to call the IPFIX thread corresponding to the CPU data core to update the flow record table.

[0023] Thirdly, embodiments of this application provide a network device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor is used to implement any of the methods provided in the first aspect when executing the programs stored in the memory.

[0024] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the methods provided in the first aspect.

[0025] Fifthly, embodiments of this application also provide a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the methods provided in the first aspect.

[0026] Beneficial effects of the embodiments in this application:

[0027] In the technical solution provided in this application embodiment, for each CPU data core of the network device, after the CPU data core acquires multiple network packets, it processes these multiple network packets in batches. Thus, after performing an operation on the first network packet among these multiple network packets, the data / instructions required for that operation are cached in the cache. When performing the same operation on other network packets among these multiple network packets, the data / instructions in the cache can be hit, and thus the data / instructions in the cache can be directly used to process other network packets. Compared to accessing main memory to obtain the required data / instructions, this greatly improves the processing efficiency of the network device CPU and increases the single-core traffic bandwidth supported by the network device CPU. Furthermore, each CPU data core processes multiple network packets using a CPU pipeline and updates its corresponding flow record table. Operations on the flow record table between different CPU data cores do not conflict, enabling parallel processing of these multiple network packets, accelerating the program execution process, further improving the processing efficiency of the network device CPU, and increasing the single-core traffic bandwidth supported by the network device CPU. This effectively solves the problem of the IPFIX traffic monitoring function being unavailable on network devices when facing extremely large bandwidth network traffic.

[0028] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0030] Figure 1 A schematic diagram illustrating the working principle of an IPFIX system;

[0031] Figure 2 This is a schematic diagram of a network device;

[0032] Figure 3 This is a first schematic diagram of a message processing method provided in an embodiment of this application;

[0033] Figure 4 A schematic diagram of a CPU pipeline provided in an embodiment of this application;

[0034] Figure 5 A schematic diagram of the message processing flow provided in the embodiments of this application;

[0035] Figure 6 This is a second schematic diagram of a message processing method provided in an embodiment of this application;

[0036] Figure 7 A schematic diagram of batch processing provided in an embodiment of this application;

[0037] Figure 8 This is a first schematic diagram of the IPFIX service processing flow provided in an embodiment of this application;

[0038] Figure 9 This is a second schematic diagram of the IPFIX service processing flow provided in the embodiments of this application;

[0039] Figure 10 A schematic diagram of a message processing apparatus provided in an embodiment of this application;

[0040] Figure 11 This is a schematic diagram of a network device provided in an embodiment of this application. Detailed Implementation

[0041] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0042] The Internet Protocol Flow Information Export (IPFIX) protocol is a protocol standardized by the Internet Engineering Task Force (IETF). IPFIX is used by network devices such as routers, switches, and probes to efficiently and flexibly transmit network traffic-related measurement and statistical data to collectors.

[0043] The IPFIX system consists of three parts: an exporter, a collector, and an analyzer. The working principle of the IPFIX system is as follows: Figure 1 As shown, an exporter is configured on the network device. After capturing network packets from network traffic, the exporter performs IPFIX processing on these packets, including sampling, classification, parsing, aggregation, and caching, to obtain flow records. When a flow record ages, the exporter encapsulates the flow record in an IPFIX packet and sends it to the collector. The collector parses the received IPFIX packets using IPFIX template packets to obtain network traffic statistics, which are then stored in a database. The analyzer analyzes the data collected by the collector and presents the network traffic to the user in the form of graphs, reports, and other formats.

[0044] In existing technologies, the exporter can be implemented in hardware, such as using an IPFIX hardware chip. Alternatively, it can be implemented in software, where the sampled network packets are reported to a central processing unit (CPU). The CPU then performs software processing on the network packets, including parsing, classifying, aggregating, and caching, to meet the user's traffic monitoring needs. Figure 2 The network device shown includes a forwarding plane, multiple CPU data cores, and a CPU control core. The forwarding plane can be a network interface card (NIC) or a forwarding chip. Figure 2 When the network device shown uses a software-implemented exporter, the packet processing flow may include:

[0045] (1) The forwarding plane samples network packets at a sampling ratio of 1000:1, and forwards the sampled network packets (such as...) Figure 2 Messages P1 to P5 in the buffer queue are written to the buffer queue.

[0046] (2) Each CPU data core reads network packets from the cache queue, parses the network packets, obtains the key fields included in the network packets, queries the flow record table based on the key fields, determines the flow record corresponding to the network packets, and then updates the flow record. The flow record includes statistical information of network traffic. Multiple flow records are aggregated according to the configuration and cached.

[0047] (3) When the cached stream records age, the CPU control core encapsulates the stream records in an IPFIX message and sends it to the collector.

[0048] With the rapid development of the Internet, network devices in the core and backbone networks need to forward traffic in the millions or more terabit (Tb) range. However, due to technical limitations, ultra-high-capacity IPFIX hardware chips that can match Tb-level or higher traffic bandwidth, achieve a 1:1 sampling ratio, and not affect packet forwarding requirements are currently unavailable. Therefore, the current mainstream solution is to sample network packets at a 1000:1 sampling ratio and report them to the CPU for software processing. Even so, limited by physical laws such as nanometer manufacturing processes and quantum tunneling, Moore's Law has failed. For a considerable period, the single-core processing performance of modern CPUs has essentially reached its physical limits, making significant further improvements difficult. When facing network traffic of 2Tb or higher, the aforementioned approach is insufficient. Figure 2 The method shown, which sends network packets one by one to each CPU core and utilizes the parallel processing capabilities of multiple CPU cores to implement IPFIX traffic monitoring, is becoming increasingly difficult. Currently, there are two technical approaches to alleviate this problem:

[0049] Option 1: Use a higher sampling ratio. This reduces the number of network packets sent to the CPU per unit time, alleviating the CPU's burden. However, the downside is that network traffic reconstruction becomes less accurate.

[0050] Direction 2: Optimize the software processes of uploading, parsing, caching, and sending network packets to improve the packet processing capability of a single CPU core.

[0051] This application embodiment, for technical direction 2, fully explores the potential of a single CPU core, which can significantly improve the single-core traffic bandwidth supported by the network device CPU, and there is no synchronous blocking between cores, so the overall bandwidth of the device board can be linearly expanded according to the number of CPU cores.

[0052] like Figure 3 As shown in the figure, this application provides a message processing method, which is applied to each CPU data core of a network device. Each CPU data core corresponds to a different flow record table. The method includes the following steps S301 to S303.

[0053] Step S301: Obtain multiple network packets;

[0054] Step S302: Using a CPU pipeline, multiple network packets are processed in batches, and the flow record corresponding to the service flow of each network packet in the flow record table corresponding to each CPU data core is updated.

[0055] Step S303: Send an IPFIX message to the collector. The IPFIX message includes flow records.

[0056] In the technical solution provided in this application embodiment, for each CPU data core of the network device, after the CPU data core acquires multiple network packets, it processes these multiple network packets in batches. Thus, after performing an operation on the first network packet among these multiple network packets, the data / instructions required for that operation are cached in the cache. When performing the same operation on other network packets among these multiple network packets, the data / instructions in the cache can be hit, and thus the data / instructions in the cache can be directly used to process other network packets. Compared to accessing main memory to obtain the required data / instructions, this greatly improves the processing efficiency of the network device CPU and increases the single-core traffic bandwidth supported by the network device CPU. Furthermore, each CPU data core processes multiple network packets using a CPU pipeline and updates its corresponding flow record table. Operations on the flow record table between different CPU data cores do not conflict, enabling parallel processing of these multiple network packets, accelerating the program execution process, further improving the processing efficiency of the network device CPU, and increasing the single-core traffic bandwidth supported by the network device CPU. This effectively solves the problem of the IPFIX traffic monitoring function being unavailable on network devices when facing extremely large bandwidth network traffic.

[0057] In this embodiment, the network device may include multiple CPU data cores. Each CPU data core performs software processing such as parsing, classifying, aggregating, and caching of sampled network packets; that is, it is the CPU core that executes IPFIX services. A flow record table may include multiple flow records, and different CPU data cores correspond to different flow record tables. The operation of IPFIX services is the same for each CPU data core. The following description uses one CPU data core as an example and is not intended to be limiting.

[0058] In step S301 above, multiple network packets are obtained by sampling network packets according to a preset sampling ratio by the forwarding plane. The preset sampling ratio can be set according to actual needs. For example, the preset sampling ratio can be 1000:1. When the exporter needs to collect statistical information on network traffic (i.e., service flow), for the sampled network packets, the CPU data core can read multiple network packets that need to be processed in batches into the cache, and then read these multiple network packets from the cache.

[0059] In step S302 above, the flow record may include statistical information about network traffic, such as the cumulative number of packets, the cumulative number of bytes, the start timestamp of the network traffic, and the end timestamp of the network traffic. The flow record may also include routing-related information about the network traffic. This is not limited. The CPU pipeline may include multiple processing logics, such as... Figure 4 The diagram shows four processing logics: fetch, decode, schedule, and execute. These multiple processing logics are executed sequentially, much like a pipeline. The execution logic can include operations such as integer calculations, logical judgments, and load / write-back.

[0060] After acquiring multiple network packets, the CPU data core performs batch processing on these packets using the CPU pipeline. For example, the CPU pipeline includes four processing logics: fetch, decode, schedule, and execute. The CPU data core acquires four network packets, such as packets 1 through 4. After fetching packet 1, the CPU data core performs decoding on packet 1. While decoding packet 1, the CPU data core fetches instructions for packet 2. This process continues, with the CPU data core executing packet 1, scheduling packet 2, decoding packet 3, and fetching instructions for packet 4. This achieves parallel batch processing of packets 1 through 4.

[0061] In this embodiment of the application, the CPU data core batching multiple network packets has the following beneficial effects:

[0062] 1) If these multiple network packets come from the same Virtual Private Network (VPN) and have the same destination Internet Protocol (IP), then when the CPU data core processes these multiple network packets, it only needs to query the routing-related storage structure once using the destination IP of the first network packet to obtain the corresponding routing information. Subsequent network packets can directly use the routing information obtained this time without having to look up the routing-related storage structure again, thus reducing unnecessary processes.

[0063] 2) The subsequent processing of a single network packet is lengthy and involves accessing a large amount of data. By batch processing multiple network packets, the number of cache misses when the CPU data core accesses memory data level is reduced when processing network packets after the first network packet. This can effectively utilize the CPU's cache and improve memory access performance.

[0064] For example, the CPU data core receives two network packets, such as packet 1 and packet 2. The packet processing flow includes, for example, packet 2. Figure 5 The steps A through E are shown. Executing steps A through E requires accessing data a through data e, respectively.

[0065] In the independent processing flow of a single network packet, the CPU core executes steps A through E for packet 1. During this process, the CPU core encounters a cache miss when accessing memory, and then reads data a through e from main memory into the cache, completing the processing of packet 1. Because the subsequent processing flow of a single network packet is lengthy, the cache is full after data a through d are cached. When data e is read into the cache, data a is cleared from the cache. After executing steps A through E for packet 1, the CPU core then executes steps A through E for packet 2. In this process, the CPU core first accesses the cache to retrieve data a, encountering a cache miss, and then reads data a from main memory into the cache. At this time, data b is cleared from the cache. It then accesses the cache again to retrieve data b, encountering a cache miss, and then reads data b from main memory into the cache. At this time, data c is cleared from the cache. This process continues. The CPU core encounters 5 cache misses when executing steps A through E for packet 2.

[0066] In a batch processing flow of multiple network packets, the CPU data checks packets 1 and 2 and executes step A respectively. Specifically, for packet 1, step A involves a cache miss, prompting the CPU to read data 'a' from main memory into the cache, completing step A for packet 1. Then, for packet 2, step A is executed; data 'a' is not evicted due to a full cache and can be read directly from the cache, completing step A for packet 2. Next, the CPU data checks packets 1 and 2 and executes step B respectively. For packet 1, step B involves a cache miss, prompting the CPU to read data 'b' from main memory into the cache, completing step B for packet 1. Then, for packet 2, step B is executed; data 'b' is not evicted due to a full cache and can be read directly from the cache, completing step B for packet 2. This process continues. Steps A through E for packet 2 are processed without cache misses.

[0067] 3) The subsequent processing flow of a single network packet is lengthy, involving numerous and deeply nested function calls. Because these functions have different functionalities, the CPU core needs to access different code segments. When multiple network packets are of the same type, such as both being Internet Protocol Version 4 (IPv4) or Internet Protocol Version 6 (IPv6) packets, the instructions for processing the packets in the functions corresponding to the parsing, classification, aggregation, and caching steps are identical. By batch processing multiple network packets, the number of cache misses at the memory access level by the CPU core is reduced when processing network packets after the first one. This effectively utilizes the CPU's cache and improves memory access performance.

[0068] After batch processing of these multiple network packets using the CPU pipeline, the CPU data core can obtain the statistical information corresponding to the service flow to which each network packet belongs, and complete the flow record update for each service flow to which the network packet belongs. For example, if the CPU data core obtains 4 network packets, all of which belong to service flow 1, after batch processing of these 4 network packets using the CPU pipeline, the cumulative number of packets included in flow record 1 corresponding to service flow 1 in the flow record table corresponding to the CPU data core is incremented by 4, and the cumulative number of bytes is added to the sum of the number of bytes and the value of these 4 network packets.

[0069] In step S303 above, each flow record can be configured with a corresponding aging timer. When the aging timer expires, the CPU data core can encapsulate the corresponding flow record in an IPFIX packet and send it to the collector. The collector uses the IPFIX template packet to parse the received IPFIX packet, obtains network traffic statistics, and stores the network traffic statistics in the database. The analyzer analyzes the data collected by the collector and displays the network traffic to the user in the form of graphs, reports, and other formats.

[0070] In some embodiments, step S301 may be: reading a first number of network packets from the cache queue corresponding to the CPU data core; caching the first number of network packets in the cache; and reading the first number of network packets from the cache.

[0071] In this embodiment, the forwarding plane stores the sampled network packets in a buffer queue. That is, the sampled network packets that the CPU data core needs to process are stored in the buffer queue corresponding to that CPU data core. The first quantity is less than or equal to a preset value, which is the maximum number of packets read at one time. The preset value can be set according to actual needs. For example, the preset value can be 4, 50, 100, 256, etc.

[0072] When the exporter needs to collect network traffic statistics, the CPU data core first reads network packets from the cache. If a cache miss occurs, that is, the cache does not store the first number of network packets to be processed, the CPU data core reads the first number of network packets from the corresponding cache queue and caches these first number of network packets in the cache. After that, when the CPU data core reads network packets from the cache, it will hit the network packets and read these first number of network packets. For these first number of network packets, step S302 is executed.

[0073] In the technical solution provided in this application embodiment, network packets can be pre-diverted, and the network packets that the CPU data core needs to process can be stored in the cache queue corresponding to the CPU data core, which reduces the lack of synchronization and blocking between CPU data cores and improves data processing performance.

[0074] In this embodiment, when the CPU data core reads network packets into the cache, it can read them one cache line per network packet. That is, it reads the first cache line size of each network packet into the cache, with each read data occupying one cache line. Taking a cache line size of 64 bytes as an example, the first 64 bytes of data in the network packet can include relevant information for network traffic statistics. In other words, based on the first 64 bytes of data in the network packet, network traffic statistics can be completed, and IPFIX flow records can be updated. In this embodiment, reading the first cache line size of each network packet into the cache reduces the cache space occupied by each network packet, increases the number of packets read at once, and thus increases the number of packets processed in batches. This further improves memory access performance and increases the single-core bandwidth supported by the network device's CPU. The size of a cache line can be set according to actual needs; for example, a cache line size can also be 32 bytes, 128 bytes, 256 bytes, etc.

[0075] In some embodiments, such as Figure 6 As shown in the embodiment of this application, a message processing method is also provided, which is applied to any CPU data core included in a network device. The method may include steps S601 to S607, wherein steps S606 to S607 are the same as steps S302 to S303 described above, and will not be repeated here.

[0076] Step S601: Read the first number of network packets from the cache queue corresponding to the CPU data core;

[0077] Step S602: Cache the first number of network packets into the cache;

[0078] Step S603: Read the first number of network packets from the cache;

[0079] Step S604: Read the second number of network packets after the first number of network packets from the cache queue corresponding to the CPU data core;

[0080] Step S605: Cache the second number of network packets into the cache.

[0081] In this embodiment of the application, the CPU data core reads a first number of network packets from the cache, and executes step S606 for these first number of network packets.

[0082] During step S606, when processing the first number of network packets, the CPU data core rereads the second number of network packets from the corresponding cache queue and caches these second number of network packets in the cache. The second number is less than or equal to a preset value, and the second number may be the same as or different from the first number.

[0083] After step S606 is executed on the first number of network packets, the CPU data core will read new network packets from the cache and hit the network packets, and then read the second number of network packets. For the second number of network packets, step S606 is executed, that is, the CPU pipeline is used to process the second number of network packets in batches and update the flow record table corresponding to the service flow to which the second number of network packets belong in the CPU data core.

[0084] In the technical solution provided in this application embodiment, during the process of the CPU data core processing multiple network packets in batches, the memory prefetch capability of the CPU data core is used to preload multiple network packets that need to be processed next into the cache, thereby reducing the number of cache misses when processing subsequent network packets, further improving memory access performance, and increasing the single-core traffic bandwidth supported by the network device CPU.

[0085] In some embodiments, step S601 can be as follows: if the number of network packets in the cache queue corresponding to the CPU data core is greater than or equal to a preset value, then a preset number of network packets are read from the cache queue corresponding to the CPU data core; if the number of network packets in the cache queue corresponding to the CPU data core is less than the preset value, then all network packets in the cache queue corresponding to the CPU data core are read. The preset value can be set according to actual needs. For example, the preset value can be 4, 50, 100, 256, etc.

[0086] In this embodiment, when the CPU data core reads network packets from the corresponding cache queue, it can detect whether the number of network packets in the cache queue is greater than or equal to a preset value. If yes, that is, the number of network packets in the cache queue is greater than or equal to the preset value, the CPU data core reads the preset value of network packets from the corresponding cache queue, i.e., the first number equals the preset value; if no, that is, the number of network packets in the cache queue is less than the preset value, the CPU data core reads all the current network packets in the corresponding cache queue, i.e., the first number is less than the preset value.

[0087] In this embodiment, the CPU data core can also directly read network packets from the corresponding cache queue; when a preset number of network packets are read, reading network packets from the corresponding cache queue stops; when the number of network packets read is less than the preset value, but the corresponding cache queue is empty, reading network packets from the corresponding cache queue stops; when the number of network packets read is less than the preset value, and the corresponding cache queue is not empty, reading network packets from the corresponding cache queue continues until the preset number of network packets are read, or the corresponding cache queue is empty.

[0088] In the technical solution provided in this application embodiment, the CPU data core does not need to wait for the number of packets in the cache queue to reach a preset value before reading and subsequent processing. Instead, it tries its best to receive packets, reading as many as possible, up to a maximum of the preset value. This reduces the waiting time of the CPU data core, further improves memory access performance, and increases the single-core traffic bandwidth supported by the network device CPU.

[0089] In some embodiments, batch processing may include multiple processing stages, each processing stage may include multiple processing sub-stages, and each processing sub-stage consists of multiple processing logics included in the CPU pipeline. In this case, step S302 above may be: performing CPU pipeline processing on multiple network packets according to the order of the multiple processing stages and the order of the multiple processing sub-stages in each processing stage, thereby updating the flow record corresponding to the service flow to which each network packet belongs in the flow record table corresponding to each CPU data core.

[0090] In this embodiment, the processing stages included in batch processing can be divided according to actual needs. For example, batch processing may include, for instance, the following stages: Figure 7The batch processing stages shown include batch packet reception, packet classification, batch filtering, batch parsing, batch querying, and batch updating. Batch processing may also include other processing stages, such as some features in IPFIX flow record table management, for example, processing packets corresponding to out-of-size flow records according to the pattern when the flow record table is out of specification. The processing stages included in batch processing are not limited, as long as the processing logic for the same batch of packets is the same and continuous, and the occurrence of branch judgments and other jump instructions is avoided as much as possible, so as to realize batch processing in the CPU pipeline.

[0091] Batch processing comprises multiple processing stages, which are executed sequentially to achieve the complete flow from packet reception to statistical stream records. Each processing stage can further include multiple sub-stages, which are executed sequentially to complete the corresponding processing stage. Each sub-stage consists of multiple processing logics within the CPU pipeline, which are executed sequentially to complete the corresponding sub-stage.

[0092] The CPU data core executes each processing stage in sequence. Within a single processing stage, it executes its sub-stages in the same order. During each sub-stage, the CPU data core performs batch processing of multiple network packets using a CPU pipeline. After completing all processing stages, the CPU data core updates the flow record in its flow record table for each network packet, corresponding to the relevant service flow.

[0093] Still with Figure 7 Let's take an example to illustrate. The CPU data core receives packets in batches from the cache queue, that is, it acquires multiple network packets, and then classifies each network packet, such as... Figure 7 As shown, packets are categorized into IPv4 ordinary flows, IPv6 ordinary flows, and Multi-Protocol Label Switching (MPLS) flows. These packet types are merely examples; the classification should be as detailed as possible based on actual network traffic conditions, as the processing logic (such as parsing packet header formats) for the same type of network traffic is essentially the same. This packet classification must also ensure the ability to generate batches of packets. The following example uses IPv6 ordinary flows.

[0094] For multiple network packets (i.e., IPv6 packets) in a normal IPv6 flow, the CPU data core performs batch filtering on these packets, that is, batch excluding multiple IPv6 packets that do not meet preset conditions, such as batch excluding illegal IPv6 packets. For IPv6 packets that do not meet the preset conditions, no processing is performed, and the current processing ends. IPv6 packets that meet the preset conditions are the IPv6 packets that actually need to be processed.

[0095] For the remaining IPv6 packets after filtering, the CPU data core performs batch parsing on these packets, extracting key fields such as the 7-tuple information. After parsing, the CPU data core queries the IPv6 flow record table in batches based on the extracted key fields. The query results are categorized into two types: flow record found and flow record not found. Since IPv6 packets are divided into inbound and outbound packets, the "no flow record found" category can be further subdivided into inbound flow record not found and outbound flow record not found.

[0096] For IPv6 packets for which no flow record can be found, they are considered new traffic. The CPU data core can query routing information according to the inbound and outbound directions and add the corresponding routing information to the IPv6 flow record table. After supplementing the routing information, the CPU data core updates the statistical information in the flow records corresponding to the IPv6 packets for both those with and without flow records.

[0097] In the technical solution provided in this application embodiment, the batch processing can be divided into processing stages, processing sub-stages, and processing logic according to the characteristics of network traffic in IPFIX services. This can ensure that the processing logic of the same batch of packets is the same and continuous, thereby realizing batch processing of the CPU pipeline and improving the single-core traffic bandwidth supported by the network device CPU.

[0098] In some embodiments, the network device may include a plurality of CPU data cores; the plurality of CPU data cores satisfy at least one of the following conditions 1) to 4).

[0099] 1) Each CPU data core corresponds to a different cache queue.

[0100] In this embodiment, one CPU data core corresponds to one cache queue. This allows multiple CPU data cores to read network packets from their respective cache queues in parallel. Compared to multiple CPU data cores corresponding to one cache queue, this embodiment eliminates the need for locks to ensure synchronization between different CPU data cores, preventing synchronization blockage between CPU data cores, improving data processing performance, and achieving lock-free communication.

[0101] 2) The cache queue corresponding to each CPU data core is a lock-free circular queue.

[0102] In this embodiment, the buffer queue is a lock-free ring queue. Thus, when the enqueue processing unit and the dequeue processing unit operate on the same buffer queue, they do not need to lock each other; each processes its own data, thereby achieving lock-free message transmission, improving data processing performance, and realizing lock-free communication.

[0103] 3) Each CPU data core processes different business flows.

[0104] In this embodiment of the application, one CPU data core corresponds to one flow record table, and the network packets included in different service flows are processed by different CPU data cores.

[0105] IPFIX service monitors and analyzes incoming and outgoing packets in communication on a "flow" basis. Typically, a service flow (i.e., network traffic) is defined using a seven-tuple of information, which includes the destination IP address, source IP address, destination port number, source port number, protocol number, type of service (ToS), and input or output interface of the network packet.

[0106] The network device's CPU traffic splitting core or forwarding plane performs preliminary parsing of network packets to obtain seven-tuple information (it can be less than seven tuples, but cannot exceed the aforementioned range); it calculates a hash value corresponding to the seven-tuple information, and this hash value corresponds to a CPU data core; then it sends the network packet to the buffer queue corresponding to that CPU data core. In this way, it can be ensured that network packets included in the same network traffic are always processed on the same CPU data core.

[0107] When a CPU core receives a network packet, since all network packets belonging to the same network traffic are processed on the same CPU core, the entry data for that network traffic will definitely only exist in the flow record table corresponding to that CPU core. In this way, the CPU core only needs to look up its own flow record table to perform flow record creation and update operations, without conflicting with query, creation, deletion, and update operations in the flow record tables of other CPU cores, thus achieving lock-free storage.

[0108] 4) Each CPU data core corresponds to a different IPFIX thread.

[0109] A thread is the basic unit for CPU to perform business processing. Within the same thread, various business processes are processed serially without conflict. A complete network traffic executing IPFIX services (referred to as an IPFIX flow) is processed by the same CPU core within the same thread (i.e., the IPFIX thread). In this embodiment, one CPU core corresponds to one IPFIX thread; that is, the number of IPFIX threads started corresponds to the number of CPU cores in the network device.

[0110] IPFIX services include sampling packets, uploading them to the CPU data core for storage and updating, as well as periodic reporting of flow record tables to the collector, aging and deletion of flow record tables, querying and displaying flow record tables, and other business processing logic.

[0111] In this embodiment, different CPU data cores use different IPFIX threads to execute different IPFIX services. If the above batch processing is executed separately, there will be no conflict between different CPU data cores, thus achieving lock-free service.

[0112] When a network device meets all four of the above conditions, it can achieve complete lock-free operation, with no competition or synchronization blocking between CPU cores, and the overall processing performance can be linearly scaled without loss according to the number of CPU cores.

[0113] In some embodiments, a CPU data core can receive an IPFIX service request, which is used to update the flow record table; and call the IPFIX thread corresponding to the CPU data core to update the flow record table.

[0114] In this embodiment, each CPU data core corresponds to a different IPFIX thread. IPFIX service requests can be requests for updating processes such as periodic reporting of the flow record table to the collector, aging and deletion of the flow record table, and querying and displaying the flow record table. IPFIX service requests can also be the aforementioned... Figure 3 The diagram shows the request to update the flow record and send an IPFIX message.

[0115] The CPU control plane (e.g., the user) sends IPFIX service requests to the corresponding CPU data core. Upon receiving the IPFIX service request, the CPU data core calls its corresponding IPFIX thread to execute the update processing indicated by the IPFIX service request, such as adding, deleting, modifying, and querying the IPFIX flow record table. Examples include periodic reporting of the flow record table to the collector, aging out and deleting the flow record table, and querying and displaying the flow record table.

[0116] 1) Periodic reporting collector for the stream record table.

[0117] Within a single IPFIX thread processing the flow record table, when the IPFIX system timer expires, the CPU control plane can add a timeout service request (i.e., an IPFIX service request) to the IPFIX thread via a lock-free circular queue and wake up the IPFIX thread to process the timeout service request. After being woken up, the IPFIX thread reads the timeout service request from the aforementioned lock-free circular queue, iterates through and compares the periodically reported timeout timestamp of the corresponding flow record with the current time; if the conditions are met, such as the periodically reported timeout timestamp being earlier than the current time, the statistical information in this flow record is assembled into an IPFIX packet and sent to the collector.

[0118] 2) Aging deletion of the stream record table.

[0119] Within a single IPFIX thread processing the flow record table, when the IPFIX system timer expires, the CPU control plane can add a timeout service request (i.e., an IPFIX service request) to the IPFIX thread via a lock-free circular queue and wake up the IPFIX thread to process the timeout service request. After being woken up, the IPFIX thread reads the timeout service request from the aforementioned lock-free circular queue and determines the duration of the flow record's lack of update. If the lack of update duration exceeds a preset duration, meaning the flow record has not received new packet refresh information for an extended period, the flow record needs to be deleted, and the memory corresponding to the flow record needs to be released.

[0120] 3) Query and display of the stream record table.

[0121] Administrators may need to view statistics on specific flow records at a particular moment on the device. In this case, the CPU control plane can send query requests (i.e., IPFIX requests) to each IPFIX thread. Upon receiving the query request, the IPFIX thread iterates through or queries the flow record table according to filtering conditions; the query results are then returned to the CPU control plane and displayed to the administrator.

[0122] Since all operations on the flow record table are performed by the same IPFIX thread, and only the flow record table corresponding to the CPU data core to which the IPFIX thread belongs is accessed, there are no conflicts between these IPFIX threads, thus achieving complete lock-free operation of IPFIX thread services.

[0123] The following is combined with Figures 8-9 The IPFIX service processing flow shown herein provides a detailed description of the message processing method provided in the embodiments of this application. Figure 8 In this network device, CPU data core 1, CPU data core 2, CPU traffic splitting core, and forwarding plane are included. Figure 9 In this embodiment, the network device includes CPU data core 1, CPU data core 2, and a forwarding plane. The CPU traffic splitting core is the CPU core that performs traffic splitting. The IPFIX service processing flow includes a packet sampling and traffic splitting sub-flow and a CPU software processing sub-flow. The CPU software processing sub-flow includes parsing, classification, aggregation, and caching. In this embodiment, the preset value is 4, i.e., processing 4 network packets in batches as an example.

[0124] (1) Message sampling and diversion processing sub-process.

[0125] Figure 8In the process, the forwarding plane samples network packets and writes them into a lock-free circular queue 3. The CPU traffic splitting core performs preliminary parsing on the network packets in the lock-free circular queue 3 to obtain key fields (such as 7-tuple information); it then uses these key fields for traffic splitting, determining the CPU data core corresponding to the network traffic to which the network packet belongs, and distributing the network packet to the lock-free circular queue corresponding to that CPU data core. Figure 8 In the process, the CPU distribution core distributes packets P1 to P12 to the lockless circular queue 1 corresponding to CPU data core 1, and distributes packets P13 to P20 to the lockless circular queue 2 corresponding to CPU data core 2.

[0126] Figure 9 In this process, the forwarding plane samples network packets and performs preliminary parsing to obtain key fields (such as 7-tuple information). These key fields are then used for traffic routing, specifically determining the CPU core to which the network traffic belongs and distributing the network packet to the corresponding lock-free circular queue. For example... Figure 9 In the process, the CPU distribution core distributes packets P1 to P12 to the lockless circular queue 1 corresponding to CPU data core 1, and distributes packets P13 to P20 to the lockless circular queue 2 corresponding to CPU data core 2.

[0127] (2) CPU software processing sub-process.

[0128] Figure 8 and Figure 9 In the process, CPU data core 1 reads packets P1 through P4 from the lock-free circular queue 1 into the cache. Each packet occupies one cache line; for example, packet P1 occupies cache line 1, packet P2 occupies cache line 2, packet P3 occupies cache line 3, and packet P4 occupies cache line 4. Then, CPU data core 1 performs batch processing of packets P1 through P4 using the CPU pipeline. Figure 8 and Figure 9In the CPU data core 1, the execution logic for message P1 is executed in parallel, the scheduling logic for message P2 is executed, the decoding logic for message P3 is executed, and the instruction fetching logic for message P4 is executed, updating the flow record table 1 corresponding to CPU data core 1. During the batch processing of messages P1 to P4 in the CPU pipeline, CPU data core 1 reads messages P5 to P8 from the lock-free circular queue 1 into the cache, i.e., prefetches messages P5 to P8 into the cache, such as message P5 occupying cache line 5, message P6 occupying cache line 6, message P7 occupying cache line 7, and message P8 occupying cache line 8. After the batch processing of messages P1 to P4 in the CPU pipeline is completed, CPU data core 1 performs batch processing of messages P5 to P8 in the CPU pipeline, updating the flow record table 1 corresponding to CPU data core 1.

[0129] Similarly, CPU data core 2 reads packets P13 through P16 from the lock-free circular queue 2 into the cache. Each packet occupies one cache line; for example, packet P13 occupies cache line 1, packet P14 occupies cache line 2, packet P15 occupies cache line 3, and packet P16 occupies cache line 4. Afterward, CPU data core 2 performs batch processing of packets P13 through P16 using the CPU pipeline, updating the corresponding flow record table 2 of CPU data core 2. Figure 8 and Figure 9 In the CPU data core 2, the execution processing logic for message P13 is performed in parallel, the scheduling processing logic for message P14 is performed, the decoding processing logic for message P15 is performed, and the instruction fetching processing logic for message P16 is performed, updating the flow record table 2 corresponding to CPU data core 2. During the batch processing of messages P13 to P16 in the CPU pipeline, CPU data core 2 reads messages P17 to P18 from the lock-free circular queue 2 into the cache, i.e., prefetches messages P17 to P18 into the cache, because at this time the lock-free circular queue 2 only contains messages P17 to P18. After the batch processing of messages P13 to P16 in the CPU pipeline is completed, CPU data core 2 performs batch processing of messages P17 to P18 in the CPU pipeline, updating the flow record table 2 corresponding to CPU data core 2.

[0130] The technical solution provided in this application employs batch processing and CPU pipelined methods to reduce the number of cache misses and improve the performance of a single CPU core in processing packets. Furthermore, the technical solution provided in this application is a completely lock-free solution; there is no contention or synchronization blocking between CPU cores, and the overall processing performance can be linearly scaled without loss according to the number of CPU cores.

[0131] After performance optimization using the technical solution provided in this application embodiment, with a sampling ratio of 1:1 and 2 million traffic flows, the processing performance of a single CPU data core can reach 0.94G, equivalent to 0.94T processing performance at a sampling ratio of 1000:1. Thus, three CPU data cores can meet the requirements of a single board with a traffic bandwidth of over 2T. In real network devices, the processing performance of a single CPU core in the technical solution provided in this application embodiment is improved by approximately (0.94-0.15) / 0.15 = 5.27 times.

[0132] The technical solution provided in this application improves the processing efficiency of the network device's CPU and increases the single-core traffic bandwidth supported by the network device's CPU. This effectively solves the problem of the IPFIX traffic monitoring function being unavailable on network devices when facing network traffic with extremely large bandwidth capacity.

[0133] Corresponding to the above message processing method, embodiments of this application provide a message processing apparatus, such as... Figure 10 As shown, the device is applied to each CPU data core of the network device, with each CPU data core corresponding to a different flow record table. The device includes:

[0134] Module 1001 is used to acquire multiple network packets;

[0135] Processing module 1002 is used to batch process multiple network packets using CPU pipeline and update the flow record corresponding to the service flow of each network packet in the flow record table corresponding to each CPU data core;

[0136] The sending module 1003 is used to send IPFIX messages to the collector. The IPFIX messages include flow records.

[0137] In the technical solution provided in this application embodiment, for each CPU data core of the network device, after the CPU data core acquires multiple network packets, it processes these multiple network packets in batches. Thus, after performing an operation on the first network packet among these multiple network packets, the data / instructions required for that operation are cached in the cache. When performing the same operation on other network packets among these multiple network packets, the data / instructions in the cache can be hit, and thus the data / instructions in the cache can be directly used to process other network packets. Compared to accessing main memory to obtain the required data / instructions, this greatly improves the processing efficiency of the network device CPU and increases the single-core traffic bandwidth supported by the network device CPU. Furthermore, each CPU data core processes multiple network packets using a CPU pipeline and updates its corresponding flow record table. Operations on the flow record table between different CPU data cores do not conflict, enabling parallel processing of these multiple network packets, accelerating the program execution process, further improving the processing efficiency of the network device CPU, and increasing the single-core traffic bandwidth supported by the network device CPU. This effectively solves the problem of the IPFIX traffic monitoring function being unavailable on network devices when facing extremely large bandwidth network traffic.

[0138] In some embodiments, the acquisition module 1001 is specifically used to read a first number of network packets from the cache queue corresponding to the CPU data core; cache the first number of network packets in the cache; and read the first number of network packets from the cache.

[0139] In some embodiments, the acquisition module 1001 is further configured to, during the process of batch processing multiple network packets using a CPU pipeline, further include: reading a second number of network packets after the first number of network packets from the cache queue; and caching the second number of network packets into the cache.

[0140] In some embodiments, the acquisition module 1001 is specifically used to read a preset number of network packets from the cache queue corresponding to the CPU data core if the number of network packets in the cache queue corresponding to the CPU data core is greater than or equal to a preset value; and to read all network packets in the cache queue corresponding to the CPU data core if the number of network packets in the cache queue corresponding to the CPU data core is less than the preset value.

[0141] In some embodiments, batch processing includes multiple processing stages, each processing stage includes multiple processing sub-stages, and each processing sub-stage consists of multiple processing logics included in the CPU pipeline; the processing module 1002 is specifically used to perform CPU pipeline processing on multiple network packets in the order of the multiple processing stages and the order of the multiple processing sub-stages in each processing stage.

[0142] In some embodiments, the network device includes multiple CPU data cores; the multiple CPU data cores satisfy at least one of the following conditions: each CPU data core corresponds to a different cache queue; each CPU data core corresponds to a lock-free circular queue; each CPU data core processes different service flows; each CPU data core corresponds to a different IPFIX thread.

[0143] In some embodiments, each CPU data core corresponds to a different flow record table and each CPU data core corresponds to a different IPFIX thread. The above-mentioned packet processing device further includes: a receiving module, used to receive an IPFIX service request, the IPFIX service request being used to update the flow record table; and a calling module, used to call the IPFIX thread corresponding to the CPU data core to update the flow record table.

[0144] This application also provides a network device, such as... Figure 11 As shown, the system includes a processor 1101, a communication interface 1102, a memory 1103, and a communication bus 1104. The processor 1101, communication interface 1102, and memory 1103 communicate with each other via the communication bus 1104. The memory 1103 stores computer programs. The processor 1101 includes multiple CPU cores, each of which executes a program stored in the memory 1103 to implement any of the aforementioned message processing methods.

[0145] The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0146] The communication interface is used for communication between the aforementioned network devices and other devices.

[0147] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0148] The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0149] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements any of the above-described message processing methods.

[0150] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the message processing methods described above.

[0151] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0152] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0153] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for apparatus, network devices, storage media, and program products are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0154] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A message processing method, characterized in that, The method, applied to each CPU data core of a network device, wherein each CPU data core corresponds to a different flow record table, includes: Retrieve multiple network packets; A CPU pipeline is used to process the multiple network packets in batches, and the flow record corresponding to the service flow of each network packet in the flow record table corresponding to each CPU data core is updated. Send an IPFIX message to the collector, the IPFIX message including the flow record.

2. The method according to claim 1, characterized in that, The step of acquiring multiple network packets includes: reading a first number of network packets from the cache queue corresponding to the CPU data core; caching the first number of network packets into a cache; and reading the first number of network packets from the cache.

3. The method according to claim 2, characterized in that, In the process of batch processing the multiple network packets using a CPU pipeline, the method further includes: reading a second number of network packets after the first number of network packets from the cache queue; and caching the second number of network packets into the cache.

4. The method according to claim 2 or 3, characterized in that, The step of reading a first number of network packets from the cache queue corresponding to the CPU data core includes: if the number of network packets in the cache queue corresponding to the CPU data core is greater than or equal to a preset value, then reading the preset number of network packets from the cache queue corresponding to the CPU data core; if the number of network packets in the cache queue corresponding to the CPU data core is less than the preset value, then reading all network packets in the cache queue corresponding to the CPU data core.

5. The method according to claim 1, characterized in that, The batch processing includes multiple processing stages, each processing stage includes multiple processing sub-stages, and each processing sub-stage consists of multiple processing logics included in the CPU pipeline; The process of using a CPU pipeline to batch process the multiple network packets includes: performing CPU pipeline processing on the multiple network packets according to the order of the multiple processing stages and the order of the multiple processing sub-stages in each processing stage.

6. The method according to claim 1, characterized in that, The network device includes multiple CPU data cores; the multiple CPU data cores satisfy at least one of the following conditions: Each CPU data core corresponds to a different cache queue; The cache queue corresponding to each CPU data core is a lock-free circular queue; Each CPU core processes different business flows; Each CPU data core corresponds to a different IPFIX thread.

7. The method according to claim 6, characterized in that, The method further includes: Receive an IPFIX service request, which is used to update the flow record table; The IPFIX thread corresponding to the CPU data core is invoked to update the stream record table.

8. A message processing apparatus, characterized in that, The device is applied to each CPU data core of a network device, each CPU data core corresponding to a different flow record table, and the device includes: The acquisition module is used to acquire multiple network packets; The processing module is used to batch process the multiple network packets using a CPU pipeline and update the flow record corresponding to the service flow of each network packet in the flow record table corresponding to each CPU data core. The sending module is used to send IPFIX messages to the collector, the IPFIX messages including the flow records.

9. The apparatus according to claim 8, characterized in that, The acquisition module is specifically used to read a first number of network packets from the cache queue corresponding to the CPU data core; cache the multiple network packets in a cache; and read the first number of network packets from the cache.

10. The apparatus according to claim 9, characterized in that, The acquisition module is further configured to, during the process of batch processing the multiple network packets using the CPU pipeline, read a second number of new network packets after the first number of network packets from the cache queue; and cache the second number of network packets in the cache.

11. The apparatus according to claim 9 or 10, characterized in that, The acquisition module is specifically configured to read the preset number of network packets from the cache queue corresponding to the CPU data core if the number of network packets in the cache queue corresponding to the CPU data core is greater than or equal to a preset value; and to read all network packets in the cache queue corresponding to the CPU data core if the number of network packets in the cache queue corresponding to the CPU data core is less than the preset value.

12. The apparatus according to claim 8, characterized in that, The batch processing includes multiple processing stages, each processing stage includes multiple processing sub-stages, and each processing sub-stage consists of multiple processing logics included in the CPU pipeline; the processing module is specifically used to perform CPU pipeline processing on the multiple network packets in the order of the multiple processing stages and the order of the multiple processing sub-stages in each processing stage.

13. The apparatus according to claim 8, characterized in that, The network device includes multiple CPU data cores; the multiple CPU data cores satisfy at least one of the following conditions: Each CPU data core corresponds to a different cache queue; The cache queue corresponding to each CPU data core is a lock-free circular queue; Each CPU core processes different business flows; Each CPU data core corresponds to a different IPFIX thread.

14. The apparatus according to claim 13, characterized in that, The device further includes: The receiving module is used to receive IPFIX service requests, which are used to update the flow record table. The calling module is used to call the IPFIX thread corresponding to the CPU data core to update the stream record table.

Images