A method, apparatus, electronic device, medium, and program product for identifying network traffic.

Abstract

This invention belongs to the field of data processing technology, specifically disclosing a network traffic identification method, device, electronic equipment, medium, and program product. The method is implemented through hardware and software collaboration. The hardware module first identifies fixed feature strings in link packets and writes them into a flow table, then filters packets with payloads according to a preset strategy and mirrors them to the software module. The software module parses the packets to extract the application layer payload, performs multi-feature string matching based on a dynamically updatable feature library, and generates dynamic service features. The results are then encapsulated into backwash frames in UDP and GRE formats. After receiving the backwash frames, the hardware module compares the software features with its own feature priorities, prioritizing the updating of high-priority software features to the flow table, thus completing the service backwash. This invention improves the refinement and accuracy of traffic identification, adapts to dynamic changes in services, reduces hardware resource consumption, and ensures real-time network processing.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and specifically relates to a network traffic identification method, device, electronic equipment, medium, and program product. Background Technology

[0002] With the continuous development of Internet technology and the continuous growth of smart terminal devices, including autonomous driving in cars and the emergence of various apps on mobile phones, more and more terminal devices are accessing the network, which puts forward higher requirements for network traffic identification capabilities. In particular, the current business volume has reached hundreds of thousands, and the finer segmentation of business identification is becoming more and more demanding. It is even more necessary to support the ability to identify multiple feature strings in combination. The previous single feature field identification can no longer meet the needs.

[0003] There are many methods for network traffic identification, including port-based identification methods, protocol layer and application layer-based identification methods, and user behavior-based identification methods. Port-based identification is a common network traffic identification technique. This method identifies services by the port number used during packet transmission. However, this method is relatively coarse because a large amount of traffic in network communication disguises itself as specific ports to evade control.

[0004] Protocol layer and application layer-based identification methods are relatively advanced traffic identification methods. This approach mainly involves parsing the content of data packets and extracting feature strings from the packets to identify the corresponding services. The advantages of this identification method are that it can accurately identify services and its identification speed is very fast. The disadvantages are that it can only identify single feature strings and cannot identify packets with multiple feature string combinations. In addition, this packet identification method mostly uses programmable devices (FPGAs) to implement packet parsing, which consumes a lot of chip logic resources. It is necessary to extract each packet, resulting in waste of chip resources. For software, identifying single and multiple feature strings is relatively easy and requires less resources. Therefore, the combination of hardware and software is the future development trend of packet feature identification.

[0005] User behavior recognition-based methods rely on big data analytics to analyze user web browsing behavior and traffic characteristics to identify abnormal traffic or potential cybersecurity threats. However, the accuracy of this detection method is not guaranteed, and it suffers from significant latency and cannot provide an immediate response. Summary of the Invention

[0006] To address this issue, the present invention provides a network traffic identification method, apparatus, electronic device, medium, and computer program product to solve the aforementioned technical problems.

[0007] This invention provides a network traffic identification method, comprising the following steps:

[0008] Step S1: The hardware module performs fixed feature string recognition on the network packets in the link and writes the recognized service features into the flow table.

[0009] Step S2: Based on the preset mirroring strategy, the hardware module filters out network packets with payloads when establishing flow tables, and mirrors the network packets to the software module through the mirroring interface.

[0010] Step S3: The software module parses the network packets and extracts application layer payload data. Based on a dynamically updatable feature library, it performs multi-feature string matching on the payload data to identify the corresponding service and generate dynamic service features.

[0011] Step S4: The software module encapsulates the service identification result into a backwash frame. The backwash frame is encapsulated using UDP and GRE message frame formats, and the payload is implemented by framing.

[0012] Step S5: The hardware module receives the backwash frame and parses out the service features identified by the software and their corresponding first priority. It then compares the first priority identified by the software module with the second priority identified by the hardware module in the flow table. If the first priority is higher than the second priority, the features identified by the hardware module in the flow table are updated to the features identified by the software, thus completing the service backwash. Otherwise, the service features identified by the software module are discarded.

[0013] In another aspect, this application also provides a network traffic identification device, comprising:

[0014] The flow table writing module is used by the hardware module to identify fixed feature strings of network packets in the link and write the identified service features into the flow table.

[0015] The network packet mirroring module is used to filter out network packets with payloads when establishing flow tables based on a preset mirroring strategy, and mirror the network packets to the software module through the mirroring interface.

[0016] The dynamic service feature generation module is used by the software module to parse the network packets and extract application layer payload data, perform multi-feature string matching on the payload data based on a dynamically updatable feature library, identify the corresponding service, and generate dynamic service features.

[0017] The back-coloring frame encapsulation module is used by the software module to encapsulate the service identification result into a back-coloring frame. The back-coloring frame is encapsulated using UDP and GRE message frame formats, and the payload is implemented by framing.

[0018] The service back-coloring module is used to receive the back-coloring frame from the hardware module and parse out the service features identified by the software and their corresponding first priority. The first priority identified by the software module is compared with the second priority identified by the hardware module in the flow table. If the first priority is higher than the second priority, the features identified by the hardware module in the flow table are updated to the features identified by the software, and the service back-coloring is completed. Otherwise, the service features identified by the software module are discarded.

[0019] This application also provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a network traffic identification method as described above.

[0020] In another aspect, this application provides a computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor to implement a network traffic identification method as described above.

[0021] Another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements a network traffic identification method as described above.

[0022] This invention achieves network traffic identification through a combination of hardware and software. The hardware module quickly completes fixed feature identification and payload-bearing packet mirroring, while the software module performs multi-feature matching based on a dynamic feature library and can automatically update the feature library using LightGBM and DBSCAN models. The back-coloring frame employs UDP+GRE encapsulation and multi-layer verification to ensure reliable transmission, and priority determination achieves accurate back-coloring. This solution addresses the problems of coarse traditional single-feature identification, wasted hardware resources, and difficulty in service expansion, improving the refinement and accuracy of service identification, reducing hardware costs, adapting to scenarios with dynamic traffic changes, and ensuring network real-time performance and security. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0025] Figure 1This is a business feature recognition process framework diagram provided for an embodiment of the present invention.

[0026] Figure 2 This is a flowchart illustrating a network traffic identification process provided in an embodiment of the present invention.

[0027] Figure 3 This is a flowchart of a network traffic identification method provided in an embodiment of the present invention.

[0028] Figure 4 This is a flowchart illustrating the multi-feature string matching of net load data by the software module provided in this embodiment of the invention.

[0029] Figure 5 This is a schematic diagram of the structure of a network traffic identification device provided in an embodiment of the present invention.

[0030] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] Currently, network traffic identification is accomplished using FPGA programmable chips, supporting interface bandwidth of 400G, such as Figure 1 The diagram illustrates the network traffic service identification process. Packet data is identified in two parts. The first part is fixed service identification, which involves parsing specific packets and using a fixed service identification algorithm to extract fixed fields from the packets for matching. Packets that match specific service rules are marked with a fixed feature number. The second part involves parsing the packet payload, extracting the packet data, and then caching it using a buffer (BUF). The data extraction algorithm module processes the data and sends the anchor points of each packet to the TCAM for feature lookup. The upper-layer software configures an external TCAM storage device via the CPU interface, writing the feature information to be identified into the TCAM. When the data extracted by the device matches the features stored in the TCAM, the current packet is marked with the corresponding feature number and output to the next level for processing.

[0033] After identifying the corresponding feature fields of a packet, it is written to the flow table. This means that the entire flow will be identified by a service feature, and its corresponding service is mapped through the feature number. This allows for service traffic statistics or traffic control. Simultaneously, the information corresponding to this flow is output in the form of flow records. The advantages of this method of identifying network traffic features through packets are low latency, fast packet identification response speed, and high accuracy in identifying specific packets.

[0034] However, when the messages in the link change, the corresponding service characteristics cannot be accurately identified, resulting in relatively poor flexibility. Furthermore, once there are new service requirements, only logic processing modules can be added to parse the new service characteristics, which requires a large amount of internal chip resources. The number of services supported is related to the size of hardware resources. If more services are to be supported, the FPGA chip needs to be replaced, which will result in higher costs and longer R&D and testing cycles.

[0035] Therefore, this application proposes a network traffic identification method that can utilize a combination of hardware and software to identify network traffic services, such as... Figure 2 As shown in the diagram, the present invention proposes a network traffic identification process framework. The hardware part (FPGA chip) performs specific service identification on the packets, that is, the identification of packets with fixed characteristics. At the same time, the packets are mirrored to the backend acquisition server according to the connection for fine-grained identification processing. Multiple features of the packets can be identified. Then, according to the priority of the service definition file, the identified features are back-colored into the flow table of the device to achieve accurate identification of network traffic features.

[0036] This invention improves the accuracy and refinement of network traffic service identification, taking into account chip resource utilization. It enables dynamic feature querying of services. Without requiring hardware upgrades, the software can extract multiple feature strings for feature querying by parsing packets in the link. When the service feature database changes, only the database information needs to be updated, avoiding limitations due to hardware storage resources. This achieves a dynamic, real-time identification mechanism, better suited to the constantly changing traffic environment of the Internet. The technical solution of this application will be described in detail below with reference to various embodiments.

[0037] like Figure 3 As shown, this embodiment of the invention discloses a network traffic identification method 100, which includes the following method steps:

[0038] Step S1: The hardware module performs fixed feature string recognition on the network packets in the link and writes the recognized service features into the flow table.

[0039] In some embodiments, the hardware module (FPGA chip) needs to complete basic configuration during the startup phase. For example, the basic configuration includes:

[0040] Fixed feature recognition logic loading: The preset fixed feature recognition algorithm (such as parallel comparison logic, feature field extraction logic) is loaded into the dedicated processing unit inside the FPGA. The fixed feature is a predefined string used to identify the basic business (such as "HTTP / 1.1" in the HTTP protocol, "USER" command in the FTP protocol), and these fixed features are stored in the hardware feature library integrated in the FPGA (each feature is associated with a unique "business feature number" and "second priority").

[0041] Link interface adaptation: Configure the physical link interface parameters of the FPGA, including interface speed, duplex mode, CRC check enable, etc., to ensure that it matches the transmission characteristics of the network link and avoid packet loss due to interface incompatibility.

[0042] Flow table storage area initialization: A dedicated flow table storage area is allocated in the on-chip RAM or external SRAM of the FPGA. For example, the flow table adopts a "hash index + linked list" storage structure. Each flow table entry includes "flow identifier field (five-tuple / IPv6 six-tuple), service feature number field, second priority field, flow status field (active / idle), and update timestamp field". It supports read and write operations of flow table entries within a single cycle to meet the real-time requirements of high-bandwidth scenarios.

[0043] Optionally, the FPGA receives raw packets from the network in real time through the physical link interface. During the reception process, preliminary filtering is performed synchronously to ensure that only valid packets enter subsequent processing. For example, this includes:

[0044] Link layer verification: Perform CRC check on the received Ethernet frame. If the check fails (indicating that the message was corrupted during transmission), it is discarded directly. If the check passes, strip the Ethernet frame preamble (7 bytes) and inter-frame gap (1 byte) and retain the complete Ethernet frame data (including Ethernet header, IP header, transport layer header, and payload).

[0045] Protocol type filtering: Parse the type field of the Ethernet header, retain only packets of IPv4 (0x0800) or IPv6 (0x86DD) protocols, and discard non-business protocol packets such as ARP and ICMP to reduce the occupation of hardware resources by invalid data.

[0046] Next, the FPGA performs protocol stack layer-by-layer parsing on the filtered valid packets, locating the regions containing fixed characteristic fields. Specifically, it disassembles the packet structure in the order of "Ethernet layer → Network layer → Transport layer" to extract key header information.

[0047] Ethernet layer parsing: Extract the source MAC address and destination MAC address from the Ethernet header to confirm the link layer affiliation of the packet. This information must be associated with the subsequent flow table to ensure the uniqueness of the flow identifier.

[0048] Network layer resolution: If it is an IPv4 packet, extract the source IP address (4 bytes), destination IP address (4 bytes), IP header length (IHL field, 4 bytes), and total length field (including IP header + transport layer header + payload), process IP options fields (such as routing options and fragmentation options), and retain only the core address information;

[0049] If it is an IPv6 packet, extract the source IPv6 address (16 bytes), destination IPv6 address (16 bytes), payload length (including transport layer header + payload), and next-hop header type. The IPv6 header is fixed at 40 bytes. Directly strip and process the extended header to ensure parsing efficiency.

[0050] Transport layer parsing: If it is a TCP packet, extract the source port number (2 bytes), destination port number (2 bytes), data offset field (4 bits, unit 4 bytes, indicating the length of the TCP header), TCP flags (pay special attention to the PSH flag, PSH=1 indicates that the packet carries application layer data), and strip the TCP header (including the options field).

[0051] If it is a UDP packet, extract the source port number (2 bytes) and destination port number (2 bytes). The UDP header is fixed at 8 bytes, and the subsequent data is retained after stripping it.

[0052] For the extraction of fixed feature fields, for example, based on the fixed feature location rules preset in the hardware feature library (such as "HTTP protocol fixed features are located within the application layer payload offset of 0-20 bytes", "FTP protocol fixed features are located within the TCP payload of bytes 5-15"), the FPGA feature extraction unit extracts fixed feature fields from the parsed message. For example, this specifically includes:

[0053] For fixed-length features (such as "HTTP / 1.1" being 6 bytes): directly extract the string of the corresponding length within the specified offset range as the feature to be matched;

[0054] For keyword association features (such as "USERxxx", where the keyword "USER" is followed by a dynamic username): first locate the keyword "USER", then extract a fixed-length string (such as 10 bytes) after the keyword as the feature to be matched;

[0055] If a message has no application layer payload (such as a TCP handshake message or an empty message containing only a header), it is marked as having no fixed feature fields, and subsequent feature matching steps are skipped, with only flow table creation operations performed.

[0056] Next, the FPGA calls the hardware feature library and uses a parallel comparison algorithm to quickly match the features to be matched with fixed features to determine the service feature number corresponding to the message. For example, this includes:

[0057] The FPGA sends the extracted features to be matched into its internal feature matching unit, which employs a hardware-implemented parallel comparison architecture (with lower processing latency compared to software serial comparison) and simultaneously compares them with all fixed features in the hardware feature library.

[0058] The hardware feature library is divided into storage partitions according to business type (such as web browsing business partition and file transfer business partition). The fixed features in each partition are sorted by the hash value of the feature string. The matching unit first calculates the hash value of the feature to be matched, locates the corresponding storage partition, and then performs a precise comparison to reduce the comparison range.

[0059] If the feature to be matched is exactly the same as a fixed feature in the hardware feature library (exact match), then the business feature number associated with that fixed feature (e.g., the business feature number "W001" corresponding to "HTTP / 1.1") and the "second priority" (e.g., the second priority of web browsing business is set to 80) are obtained.

[0060] If the feature to be matched does not match any of the fixed features, it is marked as a fixed feature not matched, the business feature number field is assigned "empty", and the second priority is assigned "0" (lowest priority).

[0061] After the matching is completed, the FPGA temporarily stores the "service feature number, second priority, flow identification information (MAC+IP+port+protocol), and message reception timestamp" in the internal "result cache unit".

[0062] Regarding writing the identified business features into the flow table, for example, the FPGA locates the flow table entry based on the flow identifier information, completes the establishment or update of the flow table, and accurately writes the matched business feature number into the flow table, so that one flow corresponds to one feature record.

[0063] Specifically, the FPGA calculates the hash value of the flow identifier based on the temporarily stored flow identifier information (the seven-tuple of "source MAC + destination MAC + source IP + destination IP + source port + destination port + TCP / UDP protocol" for IPv4 flow and "source MAC + destination MAC + source IPv6 + destination IPv6 + source port + destination port + TCP / UDP protocol" for IPv6 flow), and locates the corresponding flow table entry in the flow table storage area through the hash index.

[0064] At the same time, the flow type is determined. For example, if the flow table entry corresponding to the flow identifier hash value is empty, or the entry status is "idle" and the update timestamp exceeds the preset timeout threshold (such as 300 seconds), it is determined to be a "new flow" and a new entry needs to be created in the flow table; if the flow table entry corresponding to the flow identifier hash value exists and the status is "active", it is determined to be an "existing flow" and the entry is located directly.

[0065] Next, perform the corresponding flow table operations based on the flow type to ensure that the business feature number is accurately associated with the flow table. Specifically, this includes:

[0066] New flow table entry creation: Allocate new entry space in the flow table storage area and write the flow identifier information;

[0067] Write the temporary business feature number to the entry business feature number field, and the second priority to the second priority field; set the entry status to "active", and update the "update timestamp" to the current message reception timestamp;

[0068] Update existing flow table entries: If the service feature number matched by the current message is consistent with the service feature number already stored in the flow table entry, only the "update timestamp" will be updated, while other fields will remain unchanged;

[0069] If the service feature number matched by the current message is inconsistent with the service feature number already stored in the flow table entry (such as when the service type of subsequent messages in the same flow changes), the new service feature number will overwrite the old feature number, and the second priority field and the update timestamp will be updated synchronously.

[0070] Processing of flow tables that do not match fixed features: If the business feature number is "empty", then write "empty" in the business feature number field of the flow table entry, write "0" in the second priority field, and set the status to "to be supplemented" to facilitate quick identification when the software module re-colors the features.

[0071] Step S2: Based on the preset mirroring strategy, the hardware module filters out network packets with payloads when establishing flow tables, and mirrors the network packets to the software module through the mirroring interface.

[0072] Optionally, a preset mirroring strategy is implemented, specifically, the mirroring object is only a packet with a payload in a network flow. The payload length is calculated by parsing the total length field of the IP header, the length field of the IP header, and the length field of the transport layer header of the packet. When the payload length is greater than 0, it is determined to be a packet with a payload. In addition, the mirroring interface is physically isolated from the normal service forwarding port.

[0073] In some embodiments, the hardware module (FPGA chip) first loads a preset mirroring strategy through initialization configuration, which clarifies the mirroring object, filtering rules and transmission path to ensure that the mirroring operation is accurate and controllable.

[0074] Specifically, the core parameters for the mirroring strategy configuration include:

[0075] Mirror source definition: Specifies the source link / port of the mirrored packet. It supports single-port mirroring or multi-port aggregated mirroring. For example, single-port mirroring only collects packets from a specific 400G core service port, while multi-port aggregated mirroring integrates packets from multiple access layer ports through the FPGA's internal aggregation logic and then mirrors them uniformly to adapt to the identification needs of different link scales.

[0076] Mirroring target definition: Clearly define the recipient of the mirrored message (software module, i.e., message processing server), including the mirroring output interface (dedicated mirroring physical port on the FPGA, such as a 100G optical port) and the software module receiving address (the IP address of the software module and the preset mirroring receiving port, such as IP: 192.168.1.100, port: 50000), to ensure that the mirrored message can be accurately routed to the software module;

[0077] Filtering rule definition: Solidify the rules for mirroring only packets with payload when establishing flow tables, including the payload length calculation method (based on IP header and transport layer header fields), payload judgment threshold (payload length > 0), and configure supplementary rules for "forced mirroring of the first packet of a new flow" and "sampling mirroring of existing flows" (the first packet of a new flow refers to the packet with payload that is first established in the flow table, and existing flows are mirrored at a frequency of 1 sample per 10 packets with payload), balancing data integrity and interface performance;

[0078] Isolation strategy definition: Configure parameters to physically isolate the mirror interface from the normal service forwarding port, clarify that the mirror output interface is only used for mirror packet transmission and does not participate in any normal service data forwarding, and set the maximum bandwidth threshold of the mirror interface (e.g., 100G, not exceeding 80% of the interface's physical bandwidth) to avoid link congestion caused by excessive mirror traffic.

[0079] In one embodiment, the FPGA uses preset filtering rules to precisely filter out packets with payloads from packets in the established flow table, excluding empty packets without valid data, thus preventing invalid images from occupying interface resources. For example, the specific filtering process is as follows:

[0080] After receiving link packets and completing the flow table establishment in step S1, the FPGA compares and correlates the flow identifier information (five-tuple / IPv6 six-tuple) of each packet with the entries in the flow table:

[0081] If the flow identifier information of the message can be found in the flow table with a corresponding "active" entry (i.e. the flow table has been established), then proceed to the subsequent payload screening stage.

[0082] If the flow identification information of a packet does not match any flow table entry (such as a TCP handshake first packet without a flow table or an invalid fragment packet), the mirroring screening is skipped directly, and only normal business forwarding is performed. This is because such packets usually do not contain complete business data, and mirroring them cannot provide a valid basis for identification of software modules.

[0083] Next, the FPGA calculates the payload length by parsing the packet header fields to determine whether it is a payload-bearing packet. The calculation logic follows the TCP / IP protocol specification, and for example includes:

[0084] Header field extraction: For IPv4 packets, extract the total length field (2 bytes, indicating the total length of the entire IP packet, including the IP header + transport layer header + payload) and the IP header length field (4 bits, in units of 4 bytes, such as a field value of 5 indicating that the IP header length is 20 bytes) from the IP header; extract the TCP data offset field (4 bits, in units of 4 bytes, such as a value of 5 indicating that the TCP header length is 20 bytes) from the transport layer header (TCP / UDP) or directly take the fixed length of 8 bytes from the UDP header;

[0085] For IPv6 packets, extract the payload length field (2 bytes, directly indicating the length after the IP header, including the transport layer header + payload) from the IP header. The IPv6 header length is fixed at 40 bytes, so no additional calculation is required. Extract the TCP / UDP header length from the transport layer header (the calculation logic is the same as for IPv4).

[0086] Net load length calculation, for example:

[0087] IPv4 packet payload length = total IP length - IP header length - transport layer header length;

[0088] IPv6 packet payload length = IPv6 payload length - transport layer header length;

[0089] If the calculated payload length is greater than 0, it is determined to be a payload-bearing message and marked as pending mirroring; if the payload length is 0 (containing only the header without application layer data, such as TCP acknowledgment messages or empty UDP messages), it is marked as not requiring mirroring and excluded from the mirroring range.

[0090] Optionally, to further optimize the validity of mirrored data and interface performance, the FPGA performs differentiated filtering on packets identified as "carrying payload packets" according to the type of flow (new flow / existing flow), exemplarily including:

[0091] When a message belongs to a "new flow" (the flow whose flow table is established for the first time), its first message with payload is forcibly included in the mirroring scope. For example, the first packet of a new flow usually contains key identifiers for business initialization (such as HTTP request lines and application layer protocol version numbers), which are the basis for software modules to quickly locate the business attributes of the flow. Forced mirroring can ensure that software modules obtain the initial characteristics of the flow.

[0092] When the flow to which the packet belongs is an "existing flow" (a flow table has been established and is in an active state), the mirrored packets are selected according to a preset sampling frequency (e.g., 1 packet is selected for every 10 packets with payload) to avoid full mirroring causing congestion on the mirrored links.

[0093] Before mirroring the selected payload packets, the FPGA will add a sampling type identifier (such as 0x01 for the first packet of a new flow and 0x02 for a sampled packet of an existing flow) to the option field (IPv4) or extended header (IPv6) of the IP header. This will help the software module to distinguish the packet type and prioritize the processing of the first packet of a new flow to improve the speed of new flow identification.

[0094] Step S3: The software module parses the network packets and extracts application layer payload data. Based on a dynamically updatable feature library, it performs multi-feature string matching on the payload data to identify the corresponding service and generate dynamic service features.

[0095] In some embodiments, such as Figure 4 As shown, the software module performs multi-feature string matching on the payload data, specifically including the following steps:

[0096] Step S201: The software module first completes the reception and initial filtering of mirrored packets to ensure that only valid packets enter the subsequent parsing process. This specifically includes:

[0097] The software module receives mirrored packets through a dedicated network interface (such as a 100G Ethernet port) whose speed matches the mirrored interface speed of the hardware module. The receiving stage focuses on link adaptation and resource isolation, and for example, specifically includes:

[0098] Link parameter adaptation: Adapt to long-length mirrored packets sent by hardware modules (such as 1500-byte standard frames with complete payload) to avoid fragment loss due to excessive frame length; at the same time, configure network card interrupt affinity to bind the mirrored packet receiving queue to a dedicated CPU core, isolate resources from other services of the software module (such as log storage, backwash frame sending) to ensure real-time reception (single packet receiving delay ≤1ms).

[0099] Invalid message filtering: After receiving a message, perform multi-layer filtering to remove invalid data, exemplarily including:

[0100] Link layer verification: Perform CRC check on Ethernet frames, and discard corrupted packets that fail the check.

[0101] Mirror identifier verification: Parse the IP header options field (IPv4) or extended header (IPv6), retain only packets containing the hardware module's preset mirror identifier (such as 0x01), and exclude interference from non-mirror packets (such as normal service forwarding packets);

[0102] Protocol type filtering: Extract the protocol field of the IP header, keep only TCP (protocol number 6) or UDP (protocol number 17) protocol packets, and discard non-business protocol packets such as ICMP and ARP to reduce invalid parsing overhead.

[0103] Next, in step S202, preliminary structured parsing of the message: the software module performs preliminary parsing of the filtered valid messages in the order of "Ethernet layer → Network layer → Transport layer," stripping away non-core header data. For example, this specifically includes:

[0104] Ethernet layer parsing: Extract the source MAC address, destination MAC address, and type field (such as 0x0800 to identify IPv4, 0x86DD to identify IPv6) from the Ethernet header, strip the 14-byte Ethernet header, and send the remaining data to the network layer parsing.

[0105] Network layer parsing: For IPv4 packets, extract the source IP address, destination IP address, IP header length (IHL field, unit 4 bytes, such as a value of 5 indicates a header of 20 bytes), and total length field (including IP header + transport layer header + payload), process IP options (such as routing options, fragmentation options), and strip the IP header;

[0106] For IPv6 packets, extract the source IPv6 address, destination IPv6 address, payload length (directly identifying the data length after the IP header), and next-hop header type. The IPv6 header is fixed at 40 bytes. After stripping and processing the extended headers (such as the segmentation header), only the transport layer and above data are retained.

[0107] Transport layer parsing: For TCP packets, extract the source port, destination port, data offset field (4 bits, 4 bytes in total, indicating the length of the TCP header), and TCP flags—only retain packets with the PSH flag set to 1 (indicating that they carry application layer data), discard TCP handshake (SYN / SYN-ACK), handshake (FIN / FIN-ACK) and other packets without payload, and strip the TCP header (including the options field).

[0108] For UDP packets, extract the source port and destination port. The UDP header is fixed at 8 bytes, and the data is directly stripped away to retain the subsequent data (UDP packets usually contain payload, so there is no need to filter flags).

[0109] Step S203, extraction of application layer payload data: The software module extracts the complete and standardized application layer payload from the initially parsed message through protocol stack deep parsing and data standardization. For example, this includes:

[0110] The total header length (14 bytes for Ethernet header + length of IP header + length of transport layer header) is calculated based on the transport layer parsing results. The portion of the total data in the message that is "total length - total header length" is extracted; this is the application layer payload. For example, in an IPv4+TCP message with a total length of 1500 bytes, an IP header of 20 bytes, and a TCP header of 20 bytes, the total header length is 14 + 20 + 20 = 54 bytes, and the payload length is 1500 - 54 = 1446 bytes. This 1446 bytes is extracted as the payload data to be matched.

[0111] In one embodiment, to eliminate the impact of payload format differences on feature matching, the software module further performs standardization processing on the payload data, exemplarily including:

[0112] Remove invisible control characters (such as null bytes, redundant newline characters, tabs) and special symbols (such as meaningless garbled characters) from the payload, and retain recognizable business-related text data;

[0113] Unify the case of text-type payloads (such as HTTP request lines and API directives) to lowercase (e.g., "GET / index.htmlHTTP / 1.1" to "get / index.htmlhttp / 1.1"), and convert text in different encoding formats (such as UTF-8 and GBK) to UTF-8 encoding;

[0114] Encrypted payloads (such as TLS encrypted segments of HTTPS messages and encrypted data in VPN tunnels) are specially marked. Plaintext features cannot be extracted from such payloads, so they are skipped in subsequent matching processes. The business type is determined only by the header attributes (such as port and protocol).

[0115] Step S204: Based on multi-feature string matching using a dynamically updatable feature library, the software module calls the dynamically updatable feature library and, through a process of feature extraction, rule comparison, and priority determination, accurately identifies the corresponding business from the payload data. For example, this specifically includes:

[0116] In one embodiment, the feature library structure includes:

[0117] Business Classification Layer: Classified according to network business scenarios (such as "video streaming business", "e-commerce transaction business", "enterprise office business"), each category is assigned a unique business classification ID (such as video business ID=01, e-commerce business ID=02).

[0118] Multi-feature combination layer: Each business category contains multi-feature combination rules for subdivided businesses, which are composed of one or more single feature strings in an AND / OR logic (e.g., high-definition video business rule: [HLS protocol identifier: #extm3u] AND [bitrate field: bandwidth≥5000000] AND [resolution field: resolution=1920x1080]).

[0119] Feature attribute layer: Each single feature string is associated with attribute parameters, including the retrieval range (e.g., payload offset 0-200 bytes), matching mode (exact match / fuzzy match), and feature priority (0-255, the larger the value, the higher the priority).

[0120] Next, the software module extracts the single feature string to be matched from the standardized payload based on the feature library's feature attributes. For example, this specifically includes:

[0121] Fixed-position feature extraction: For features located at a fixed offset in the payload (such as HTTP request lines which are usually at the beginning of the payload), directly retrieve the feature string (such as "get / " or "post / ") within a specified range.

[0122] Keyword association feature extraction: For features bound to specific keywords (such as "bandwidth=" in video services), first locate the keyword, and then extract the data after the keyword (such as "bandwidth=5000000").

[0123] Dynamic length feature extraction: For features with variable length (such as API version number "v3.2.1"), the boundaries are determined by the separators (spaces, commas) to extract the feature string completely.

[0124] The extracted single-feature strings are stored in a temporary cache in association with "business category ID - feature name" for easy rule comparison later.

[0125] Next, the software module uses a logic based on business classification traversal, rule verification, and priority sorting to determine the business to which the message belongs. For example, this specifically includes:

[0126] Business category traversal: Traverse the feature library in order of business category ID, and verify the corresponding multi-feature combination rules under each category in turn to avoid repeated matching across categories; specifically, including: "AND" logic: all single features are successfully extracted and meet the rules (e.g., "#extm3u", "bandwidth≥5000000", "resolution=1920x1080" are satisfied at the same time); "OR" logic: any single feature is successfully extracted and meets the rules (e.g., "goods_search" or "order_submit" in e-commerce business); mixed logic: first verify the "AND" logic in parentheses, and then execute the "OR" logic (e.g., "(api=v4ANDdevice=mobile)OR(api=v3ANDdevice=pc)");

[0127] Priority determination: If a message matches multiple business rules at the same time (such as matching "high-definition video" and "normal video" at the same time), the business with the highest priority in the feature library is selected as the final recognition result to avoid business ambiguity.

[0128] Step S205: Generate dynamic business features. After the software module identifies valid business, it generates dynamic business features, which associates the business identification result with flow identification information through a standardized format. For example, the dynamic business features need to cover three dimensions: flow location, business identification, and priority, to ensure that the hardware module can accurately associate the flow table and determine the backwash priority. The core fields may include:

[0129] Flow Identifier Field: A unique flow identifier extracted from the packet header. For IPv4 flows, it is a 5-tuple of "source IP + destination IP + source port + destination port + transport layer protocol" and for IPv6 flows, it is a 6-tuple of "source IPv6 + destination IPv6 + source port + destination port + transport layer protocol".

[0130] Business Category ID: The business category ID corresponding to the recognition result (e.g., video business ID=01);

[0131] Business Feature Number: A unique identifier for a sub-business under a business category (e.g., High-definition video business feature number = V001).

[0132] Feature priority: The priority value of this service defined in the feature library (e.g., high-definition video service priority = 180).

[0133] Optionally, in one embodiment, candidate features are automatically extracted from traffic samples and the feature library is dynamically updated based on a machine learning model. For known business features, the LightGBM gradient boosting tree model is used for optimization and updating. For unknown business features, the DBSCAN density clustering model is used to identify new business types and extract their candidate features.

[0134] For example, this embodiment introduces the LightGBM gradient boosting tree model and the DBSCAN density clustering model, which can automatically extract candidate features from traffic samples and dynamically update the feature library, solving the problems of low efficiency and delayed identification of unknown services in traditional manual updates.

[0135] In one embodiment, for known business processes (such as video and e-commerce) already in the feature library, the LightGBM gradient boosting tree model is used to learn the feature patterns of samples, optimize existing feature combination rules, and extract more accurate candidate features. For example, the specific process includes:

[0136] Model training process: The input is the feature vector of the labeled sample (head attribute + payload text attribute), and the output is the "business feature number" corresponding to the sample (multi-classification task, the number of classifications is equal to the number of known business types);

[0137] Load the training set data. For example, use 5-fold cross-validation to optimize the hyperparameters (learning rate set to 0.1-0.3, tree depth set to 3-8, number of leaf nodes set to 31-127). Use the validation set classification accuracy ≥98% and F1-score ≥97% as the training termination condition to ensure a balance between model fitting effect and generalization ability.

[0138] After the model is trained, the optimal combination logic of core candidate features is mined through model decision tree path analysis. Feature attribute parameters are adjusted based on feature importance, such as narrowing the "search range" of high-contribution features to the net payload key area (such as offset 0-100 bytes) and setting the "matching mode" to "exact match" to improve the efficiency of subsequent feature matching. At the same time, feature priorities are updated to ensure that the optimized feature priorities are consistent with the business importance determined by the model (such as increasing the priority of high-definition video business from 150 to 180).

[0139] In one embodiment, for unknown traffic, the DBSCAN density clustering model is used to mine sample distribution patterns, identify potential new business types, and extract candidate features. For example, this specifically includes:

[0140] 1. The model clustering process specifically includes:

[0141] Model parameter determination: The input is the feature vector of unlabeled samples. The K-distance method is used to determine the ε-neighborhood radius (the median of the feature vector distance distribution is taken to ensure reasonable cluster density). The minimum number of samples MinPts is set according to the sample size (5 for 1000 samples and 10 for 5000 samples).

[0142] Clustering Execution and Effectiveness Evaluation: The model calculates the Euclidean distance between samples and groups samples with a distance < ε and a number of samples in their neighborhood ≥ MinPts into the same cluster (potential new business types). Isolated samples are marked as "noise samples" (such as occasional abnormal traffic). The model evaluates the clustering quality by using the silhouette coefficient. Only clusters with a silhouette coefficient ≥ 0.7 are retained (indicating high similarity of samples within the cluster and large differences between clusters). Clusters with a silhouette coefficient < 0.7 are re-clustered after parameter readjustment or merged into similar clusters.

[0143] New business cluster screening: Count the number of samples in each valid cluster, and retain only the clusters with ≥100 samples (to ensure stable business traffic and exclude temporary traffic interference), and mark them as potential new business clusters.

[0144] 2. New business candidate feature extraction, specifically including:

[0145] Cluster center calculation: For each potential new business cluster, the average value of all sample feature vectors is calculated (cluster center). The attribute values ​​of each dimension of the cluster center represent the typical characteristics of the new business. For example, in the payload text feature dimension of a certain cluster center, the weight of "live-stream" reaches 0.9, the average value of the "message length" dimension is 2000 bytes, the "transport layer protocol" dimension is TCP, and the "destination port" dimension is 1935 (a commonly used port for RTMP live streaming protocol).

[0146] Differentiated Feature Screening: Compare the new business cluster center with the known business cluster centers, and screen attributes with a difference of ≥80% as core candidate features for the new business. For example, if the weight of the "live-stream" text feature is <0.1 in the known business clusters, and the frequency of "destination port 1935" is <5% in the known businesses, then these attributes, together with "message length ≥1800 bytes", constitute the candidate feature combination for this new business.

[0147] In step S4, the software module encapsulates the service identification result into a backwash frame. The backwash frame is encapsulated using UDP and GRE message frame formats, and the payload is implemented in a frame assembly manner.

[0148] Preferably, the frame structure of the back-dyeing frame, from front to back, consists of a frame header identifier, a service data block, and a frame tail checksum. The frame header identifier is 8 bytes long and includes a 4-byte magic word, a 2-byte version number, and a 2-byte service quantity.

[0149] Each business data block is 32 bytes long and includes a 1-byte stream identifier type, a 4-byte business feature number, a 1-byte feature priority, a 24-byte stream identifier field, and a 2-byte data checksum.

[0150] The frame tail check length is 2 bytes, and the frame header identifier and the overall data of all business data blocks are checked based on the CRC16 algorithm.

[0151] Preferably, the UDP and GRE protocol encapsulation process of the backwash frame, from the inside out, is as follows: GRE header encapsulation, used to add a tunnel transmission identifier to the backwash frame; UDP header encapsulation, used to provide transport layer port mapping for the backwash frame; IP header, used to provide network layer routing for the backwash frame, respectively adapting to IPv4 and IPv6 protocols; and Ethernet header, used to provide a link layer transmission identifier for the backwash frame.

[0152] In one embodiment, the software module first performs structured framing on the filtered business identification results (dynamic business features). The payload adopts a three-segment structure of frame header identifier - business data block - frame tail check, ensuring that the hardware module can parse at a fixed length, thus improving data extraction efficiency. The implementation details of each part are as follows:

[0153] Regarding the frame header identifier (8 bytes) implementation, for example, the frame header identifier is located at the very beginning of the payload and is used for frame validity verification, version compatibility, and service quantity statistics. Specifically, it includes a 4-byte Magic Word. During frame assembly, the software module directly writes this fixed value into the first 4 bytes of the payload.

[0154] A 2-byte version number (Version) is used to identify the current payload framing format version. During framing, the software module reads the current version number from the local configuration file.

[0155] The 2-byte Service Count records the total number of "service data blocks" contained in the current re-rendering frame (i.e., the number of services in a single re-rendering). When framing, the software module counts the number of services currently to be re-rendered.

[0156] Regarding the framing implementation of business data blocks, the business data block is the core content of the payload. Each data block corresponds to the service identification result of a network flow and includes flow location information, service characteristic information, and data verification information. This ensures that the hardware module can locate the flow table and verify data integrity. For example, the single block field definition and framing logic are as follows:

[0157] The 1-byte FlowIDType identifies the parsing format of subsequent flow identifier fields. It has two fixed values: 0x01 represents IPv4 flow (corresponding to a 5-tuple identifier) ​​and 0x02 represents IPv6 flow (corresponding to a 6-tuple identifier). During framing, the software module assigns a value based on the "IP version" field in the service identification result.

[0158] The 4-byte Service ID is a unique service identifier that the storage software module can identify. During framing, the software module extracts the Service ID from the dynamic service characteristics.

[0159] The 1-byte feature priority stores the priority value of the service feature (0-255, the larger the value, the higher the priority, consistent with the priority in the feature library). It is used by the hardware module to determine the priority of the back-dyeing process. When framing, the software module directly extracts the priority value from the dynamic service feature and writes it into this field.

[0160] The 24-byte FlowIDField stores a unique flow identifier used to locate hardware flow table entries. During framing, the software module extracts the flow identifier information from dynamic service characteristics in the above format and writes it into this field sequentially.

[0161] The 2-byte data checksum uses the CRC16 check algorithm to perform check calculations on 29 bytes of data, including "flow identifier type + service feature number + feature priority + flow identifier field", generate a check value and write it into the field. During framing, the software module performs CRC16 calculations in real time.

[0162] Regarding the framing implementation of the end-of-frame checksum (2 bytes), the end-of-frame checksum is located at the very end of the payload and is used to verify the integrity of the entire payload data, preventing overall payload damage due to transmission interference. During framing, the software module uses the CRC16 checksum algorithm to perform checksum calculations on the entire data of "frame header identifier (8 bytes) + all service data blocks (32 bytes × number of services)," generating a 2-byte checksum value and writing it to the end of the payload. After receiving the echo frame, the hardware module must first recalculate the overall CRC16 value of the payload and compare it with the end-of-frame checksum value. Only when the two match will subsequent parsing be performed, ensuring that the payload data is not tampered with or lost.

[0163] In one embodiment, after the software module completes payload framing, it performs protocol encapsulation in the order of "GRE header → UDP header → IP header → Ethernet header" from the inside out. This ensures that the echo frame can be stably transmitted to the hardware module through the mirror link, while also being compatible with IPv4 / IPv6 dual-protocol scenarios. For example, the encapsulation details of each layer are as follows:

[0164] The inner layer is a GRE header encapsulation (4 bytes, tunnel transmission identifier). The GRE (Generic Routing Encapsulation) protocol is used to add tunnel attribute identifiers to the reverted frame, ensuring that the hardware module can quickly distinguish the reverted frame from ordinary service packets. The encapsulation field definitions and logic are as follows:

[0165] The 2-byte GRE flags are fixed at 0x0000, indicating that GRE extended functions (such as checksum, key, and serial number) are not enabled, and only the basic package is retained. The "C bit" (checksum present bit), "K bit" (key present bit), and "S bit" (serial number present bit) are all set to 0, which simplifies the hardware parsing logic and reduces FPGA resource consumption.

[0166] The 2-byte GRE protocol type (ProtocolType) identifies the network layer protocol type corresponding to the GRE payload, and is consistent with the IP version carried by the re-encapsulation frame: if the re-encapsulation frame is used for IPv4 service re-encapsulation, this field is assigned a value of 0x0800 (corresponding to the IPv4 protocol); if it is used for IPv6 service re-encapsulation, it is assigned a value of 0x86DD (corresponding to the IPv6 protocol). During encapsulation, the software module dynamically assigns this value based on the IP version in the service identification result. After receiving the data, the hardware module can directly call the corresponding IP version parsing logic through this field, improving processing efficiency.

[0167] The middle layer encapsulates the UDP header (8 bytes, transport layer port mapping). The UDP protocol provides transport layer port location for the reflow frame, used for reflow data interaction between software and hardware modules. Simultaneously, the length field controls the frame size to avoid IP fragmentation (fragmentation easily leads to reflow data loss). For example, the encapsulation field definitions and logic are as follows:

[0168] The 2-byte source port is dynamically allocated by the software module, with a value range of 0x0400-0xFFFF (corresponding to 1024-65535, to avoid conflicts with well-known ports). It is randomly generated each time a frame is assembled. The hardware module does not need to record the source port, but only filters back-stained frames by the destination port.

[0169] The 2-byte Destination Port is a fixed, dedicated port for receiving echo frames (e.g., 0xC351, corresponding to decimal 50001) pre-defined by the hardware module. This port is configured via the CPU interface during hardware module initialization and is used solely for receiving echo frames; it does not participate in normal service forwarding. During encapsulation, the software module directly writes this fixed port number to ensure that the hardware module can quickly filter echo frames through port filtering, avoiding confusion with other service packets.

[0170] The 2-byte UDP length (UDPLength) field value is calculated as follows: UDP header length (8 bytes) + GRE header length (4 bytes) + payload length (8 bytes for frame header identifier + 32 bytes for service data block × number of services + 2 bytes for frame tail check). The software module calculates this value in real time and writes it during encapsulation. After receiving the data, the hardware module can verify the integrity of the UDP packet through this field. If the actual received length does not match the field value, the frame is considered corrupted and discarded.

[0171] 2-byte UDP checksum (UDPChecksum), optional (enabled by default), uses a UDP pseudo-header check mechanism: the software module calculates the checksum based on the overall data of "IP header source IP / destination IP, protocol type (47, corresponding to GRE), UDP header, GRE header, payload" and writes it into this field.

[0172] The outer layer is an IP header encapsulation (IPv4 20 bytes / IPv6 40 bytes, network layer routing). The IP header provides network layer routing capabilities for the raster frame, adapting to IPv4 and IPv6 protocols respectively, ensuring that the raster frame can be routed to the hardware module through the mirror link. For example, the different definitions of the encapsulation fields are shown in Table 1 below.

[0173] Table 1 Differentiated Definitions of IP Header Encapsulation Fields

[0174]

[0175] The outermost layer is the Ethernet header encapsulation (14 bytes, data link layer transmission). The Ethernet header provides a data link layer transmission identifier for the reverted frame, ensuring that the reverted frame can be correctly transmitted on the physical link (such as an optical port or an electrical port). For example, the encapsulation fields are defined as follows:

[0176] The 6-byte destination MAC address is the MAC address of the hardware module's "reflection receiving dedicated port". The software module dynamically obtains this address through the ARP protocol (IPv4) or NDP protocol (IPv6) to avoid resource waste caused by broadcast transmission and ensure that the reflection frame is accurately delivered to the designated port of the hardware module.

[0177] The 6-byte source MAC address is the MAC address of the software module's "return transmission port," which helps the hardware module locate the transmitting end through the source MAC when receiving abnormalities, thus assisting in troubleshooting.

[0178] This 2-byte field corresponds to the IP header version: if it is an IPv4 refurbished frame, it is assigned the value 0x0800; if it is an IPv6 refurbished frame, it is assigned the value 0x86DD. Link layer devices (such as switches) can use this field to quickly forward the data to the corresponding IP protocol processing module, thereby improving link transmission efficiency.

[0179] Step S5: The hardware module receives the backwash frame and parses out the service features identified by the software and their corresponding first priority. It then compares the first priority identified by the software module with the second priority identified by the hardware module in the flow table. If the first priority is higher than the second priority, the features identified by the hardware module in the flow table are updated to the features identified by the software, thus completing the service backwash. Otherwise, the service features identified by the software module are discarded.

[0180] In some embodiments, the hardware module (FPGA) listens to the reverted frames in real time through a dedicated "reverted receive port" (physically isolated from the normal service forwarding port), and parses them layer by layer from the outside in, in the order of "Ethernet layer → IP layer → UDP layer → GRE layer → payload", to extract the service characteristics and priorities identified by the software. The valid service data (flow identifier, software service feature number, first priority) is sorted according to the "flow identifier hash value" and temporarily stored in the "reverted data buffer" inside the FPGA, awaiting priority determination.

[0181] In one embodiment, a decision is made on whether to perform a flow table update based on a numerical comparison between the current feature priority (second priority) and the software feature priority (first priority) of the flow table. For example, this includes:

[0182] Target flow table entry location: Based on flow identifier information (5-tuple for IPv4, 6-tuple for IPv6), calculate the hash value of the flow identifier, and quickly locate the corresponding flow table entry in the flow table storage area through the hash index module inside the FPGA;

[0183] If the identified flow table entry is in an "idle" state (the update timestamp exceeds the timeout threshold, such as 300 seconds), or the flow identifier does not match the entry at all (caused by a hash collision), the flow table entry is deemed invalid, the software feature is discarded, and no further operations are performed; if the entry is in an "active" state and the flow identifier matches, it is deemed a "valid entry" and enters the priority comparison stage.

[0184] For valid entries, read the hardware feature number from the valid flow table entries, which is the fixed feature identifier identified by the hardware module in step S1 (empty if no match is found); and the second priority: the priority value corresponding to the hardware feature (preset in the hardware feature library, such as basic video service priority 100; if the hardware feature number is empty, the second priority is assigned a value of 0, i.e., the lowest priority); and at the same time read the temporarily stored software feature "first priority" (such as high-definition video service priority 180).

[0185] For numerical comparison logic, for example, if the first priority is greater than the second priority (e.g., 180 > 100): it is determined that the flow table needs to be updated, that is, the software features are more refined and need to cover the hardware features to ensure that the flow table records the most accurate business information.

[0186] If the first priority is less than or equal to the second priority (e.g., 90 ≤ 120): it is determined that no update is needed - hardware features have higher priority and hardware features are retained.

[0187] Optionally, in one embodiment, before step S4, the software module loads the service list pre-subscribed by the hardware module, matches the generated dynamic service features with the service list, and filters out the services required by the hardware module. Only the services required by the hardware module are included in the backwashing process, avoiding invalid data from occupying backwashing link bandwidth and hardware resources.

[0188] Specifically, during the initialization phase, the hardware module (FPGA chip) sends service subscription requests to the software module, specifying the scope of services that need to be reflected back. The software module receives and stores the subscription list, completing the synchronization of the subscription lists of the hardware and software modules.

[0189] For example, subscription service requirements may include device ID, subscription service category / feature number, subscription granularity (five-tuple / IP), effective / valid period, and specify the scope of services that need to be re-evaluated;

[0190] The subscription list is encapsulated into a configuration frame with CRC16 checksum via the CPU interface and synchronized to the software module. The software module verifies the validity of the checksum and fields (such as whether the business ID exists). If it passes the verification, it is stored in the device subscription database, indexed by the device ID.

[0191] For matching dynamic service features with subscription lists, for example, the effective subscription list of the device is called from the source hardware device ID associated with the dynamic service feature (generated in step S3); first, the service category is matched: the service category ID of the feature must be in the subscription category list; then, the feature number is matched (if specified): the feature number must be in the subscription feature number list; finally, the granularity is matched: according to the subscription 5-tuple / IP granularity, the flow identifier / IP of the feature is verified to be within the range.

[0192] Figure 5 A network traffic identification device 500 is shown. This device embodiment is similar to... Figure 3 Corresponding to the illustrated method embodiments, this device can be specifically applied to various electronic devices. Specifically, it includes:

[0193] The flow table writing module 501 is used by the hardware module to identify fixed feature strings of network packets in the link and write the identified service features into the flow table.

[0194] The network packet mirroring module 502 is used to filter out network packets with payloads when establishing flow tables based on a preset mirroring strategy, and mirror the network packets to the software module through the mirroring interface.

[0195] The dynamic service feature generation module 503 is used by the software module to parse the network packets and extract application layer payload data, perform multi-feature string matching on the payload data based on a dynamically updatable feature library, identify the corresponding service, and generate dynamic service features.

[0196] The back-coloring frame encapsulation module 504 is used by the software module to encapsulate the service identification result into a back-coloring frame. The back-coloring frame is encapsulated using UDP and GRE message frame formats, and the payload is implemented by framing.

[0197] The service back-coloring module 505 is used to receive the back-coloring frame from the hardware module and parse out the service features identified by the software and their corresponding first priority. The first priority identified by the software module is compared with the second priority identified by the hardware module in the flow table. If the first priority is higher than the second priority, the features identified by the hardware module in the flow table are updated to the features identified by the software, and the service back-coloring is completed. Otherwise, the service features identified by the software module are discarded.

[0198] Based on the same inventive concept, this application also provides an electronic device. The method corresponding to the electronic device can be the method in the foregoing embodiments, and its problem-solving principle is similar to that method. The electronic device provided in this application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the foregoing embodiments of this application.

[0199] Figure 6 The diagram illustrates the structure of an apparatus suitable for implementing the methods and / or technical solutions in the embodiments of this application. The apparatus 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The RAM 603 also stores various programs and data required for system operation. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0200] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, touchscreen, microphone, infrared sensor, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), LED display, OLED display, etc., and speakers, etc.; a storage section 608 including one or more computer-readable media such as hard disk, optical disk, magnetic disk, semiconductor memory, etc.; and a communication section 609 including a network interface card such as a LAN (local area network) card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet.

[0201] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 601, it performs the functions defined in the methods of this application.

[0202] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application described above.

[0203] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application.

[0204] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A method for identifying network traffic, characterized in that, Includes the following steps: Step S1: The hardware module performs fixed feature string recognition on the network packets in the link and writes the recognized service features into the flow table. Step S2: Based on the preset mirroring strategy, the hardware module filters out network packets with payloads when establishing flow tables, and mirrors the network packets to the software module through the mirroring interface. Step S3: The software module parses the network packets and extracts application layer payload data. Based on a dynamically updatable feature library, it performs multi-feature string matching on the payload data to identify the corresponding service and generate dynamic service features. Step S4: The software module encapsulates the service identification result into a backwash frame. The backwash frame is encapsulated using UDP and GRE message frame formats, and the payload is implemented by framing. Step S5: The hardware module receives the backwash frame and parses out the service features identified by the software and their corresponding first priority. It then compares the first priority identified by the software module with the second priority identified by the hardware module in the flow table. If the first priority is higher than the second priority, the features identified by the hardware module in the flow table are updated to the features identified by the software, thus completing the service backwash. Otherwise, the service features identified by the software module are discarded.

2. The network traffic identification method according to claim 1, characterized in that, The preset mirroring strategy is as follows: the mirroring object is only a packet with a payload in a network flow. The payload length is calculated by parsing the total length field of the IP header, the length field of the IP header, and the length field of the transport layer header of the packet. When the payload length is greater than 0, it is determined to be a packet with a payload. In addition, the mirroring interface is physically isolated from the normal service forwarding port.

3. The network traffic identification method according to claim 2, characterized in that, It also includes, Before step S4, the software module loads the service list pre-subscribed by the hardware module, matches the generated dynamic service features with the service list, and filters out the services required by the hardware module.

4. The network traffic identification method according to claim 1, characterized in that, include: Based on machine learning models, candidate features are automatically extracted from traffic samples and the feature library is dynamically updated. For known business features, the LightGBM gradient boosting tree model is used for optimization and updating. For unknown business features, the DBSCAN density clustering model is used to identify new business types and extract their candidate features.

5. The network traffic identification method according to claim 1, characterized in that, include: The frame structure of the back-dyeing frame, from front to back, consists of frame header identifier, service data block, and frame tail check. The frame header identifier is 8 bytes long and includes a 4-byte magic word, a 2-byte version number, and a 2-byte service quantity. Each business data block is 32 bytes long and includes a 1-byte stream identifier type, a 4-byte business feature number, a 1-byte feature priority, a 24-byte stream identifier field, and a 2-byte data checksum. The frame tail check length is 2 bytes, and the frame header identifier and the overall data of all business data blocks are checked based on the CRC16 algorithm.

6. The network traffic identification method according to claim 1, characterized in that, The encapsulation process of the UDP and GRE protocols in the backwash frame, from the inside out, is as follows: GRE header encapsulation, used to add a tunnel transmission identifier to the backwash frame; UDP header encapsulation, used to provide transport layer port mapping for the backwash frame; IP header, used to provide network layer routing for the backwash frame, adapting to IPv4 and IPv6 protocols respectively; and Ethernet header, used to provide a link layer transmission identifier for the backwash frame.

7. A network traffic identification device, characterized in that, include: The flow table writing module is used by the hardware module to identify fixed feature strings in network packets in the link and write the identified service features into the flow table. The network packet mirroring module is used to filter out network packets with payloads when establishing flow tables based on a preset mirroring strategy, and mirror the network packets to the software module through the mirroring interface. The dynamic service feature generation module is used by the software module to parse the network packets and extract application layer payload data, perform multi-feature string matching on the payload data based on a dynamically updatable feature library, identify the corresponding service, and generate dynamic service features. The back-coloring frame encapsulation module is used by the software module to encapsulate the service identification result into a back-coloring frame. The back-coloring frame is encapsulated using UDP and GRE message frame formats, and the payload is implemented by framing. The service back-coloring module is used to receive the back-coloring frame from the hardware module and parse out the service features identified by the software and their corresponding first priority. The first priority identified by the software module is compared with the second priority identified by the hardware module in the flow table. If the first priority is higher than the second priority, the features identified by the hardware module in the flow table are updated to the features identified by the software, and the service back-coloring is completed. Otherwise, the service features identified by the software module are discarded.

8. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the processor; wherein, The memory stores instructions that can be executed by the processor to enable the processor to perform the method of any one of claims 1-6.

9. A computer-readable medium having computer program instructions stored thereon, characterized in that, The computer program instructions can be executed by a processor to implement the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images