Application protocol identification control method based on AI learning and FPGA rapid matching
By employing a rapid matching method based on AI learning and FPGA, the whitelist requirement and processing performance bottleneck of traditional network security devices in application protocol identification and control are resolved. This achieves efficient application protocol whitelist control and high-speed processing, enhancing network isolation security.
Patent Information
- Application Number
- CN202511758366.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-27
- Publication Date
- 2026-02-06
AI Technical Summary
Traditional network security devices suffer from problems such as unmet whitelist requirements, processing performance bottlenecks, and low efficiency in protocol feature extraction in application protocol identification and control, especially under high-speed network traffic, making it difficult to achieve efficient application protocol whitelist control.
By employing an AI-based learning and FPGA-based fast matching approach, a protocol feature library is constructed and described using regular expressions through packet splitting, application layer data reassembly, FPGA protocol feature identification, and whitelist control decisions, thereby achieving high-speed protocol identification and filtering.
It improves data processing performance, enhances network isolation security, can detect known and unknown attacks, supports user-defined and dynamically loaded feature rules, and is compatible with existing stateful firewalls.
Smart Images
Figure CN121486052A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of digital information transmission technology, specifically relating to protocol identification and control technology in network security isolation devices, and in particular to an application protocol whitelist identification and control method based on AI for protocol feature learning and FPGA fast feature rule matching. Background Technology
[0002] With the continuous improvement of network bandwidth and the diversification of network applications, traditional network security devices face three major challenges in application protocol identification and control. Currently, commonly used application protocol identification and filtering technologies have two problems: First, their application scenarios are usually limited to protocol blocking, essentially a blacklist mechanism for application protocols. The limitation of the blacklist mechanism is that in normal network data flow, it can only identify known malicious or unauthorized protocols, such as WeChat, QQ, and email, which are listed as blacklisted network applications, and discard corresponding packets according to security policies to block the corresponding sessions. This cannot meet the whitelist security requirements of isolation devices such as network gateways, which require "only allowing specific protocols to pass." The whitelist requirement of network gateways and other network isolation devices is to only allow specific application protocols to pass, and all other protocol packets not in the whitelist will be blocked. Second, there is a performance bottleneck: after application protocol identification functions are set up on typical network security devices, the protocol identification function suffers from insufficient feature rule matching processing capacity when handling high-speed network traffic due to the large amount of computation required for feature matching, failing to meet the needs of high-performance network devices. Currently, with the continuous increase in network bandwidth, the processing needs of network security devices at the 10 Gigabit level and above are commonplace. Therefore, there is an urgent need for a solution that can achieve both efficient application protocol whitelist control and high-speed processing capabilities. Thirdly, there is the issue of the efficiency of application protocol feature extraction. Relying on manual analysis and extraction by network security experts is insufficient to meet the ever-increasing application scenarios and protocol requirements. Summary of the Invention
[0003] To address the aforementioned issues, the purpose of this invention is to overcome the shortcomings of existing technologies and provide a network gateway application protocol whitelist identification and control method based on AI for protocol feature learning and FPGA (Field-Programmable Gate Array) for rapid feature matching. This method achieves high-performance application protocol identification and filtering through hardware acceleration, thereby improving the security and processing efficiency of the network gateway device.
[0004] The technical solution of the present invention is as follows: An application protocol identification and control method based on AI learning and FPGA fast matching includes the following steps: S1: Packet splitting: Performs preliminary splitting of incoming data packets based on the destination port to distinguish data streams of different application protocols; S2: Application layer data reassembly: Reassemble and aggregate data packets from the same data stream to restore the application layer data content; S3: FPGA Protocol Feature Recognition: The reconstructed data is sent to the FPGA module, which has a built-in library of multiple protocol features. The application protocol is identified and its compliance is judged by regular expression matching. S4: Whitelist control decision: If the protocol identification is successful and conforms to the whitelist policy, the data stream enters the subsequent state detection and packet filtering module; if the identification fails, all packets in the data stream are directly discarded. S5: State Inspection and Forwarding: Fine-grained access control is performed through the state inspection and packet filtering module, and legitimate data packets are forwarded out.
[0005] The protocol feature library is constructed based on AI learning algorithms to extract application protocol feature information, using regular expressions for description, and supports user-defined and dynamically loaded features. The basic process of constructing the feature library based on AI learning is as follows.
[0006] First, a large amount of communication data from different application protocols is fed into the AI-specific learning model, and the AI model is informed of the application protocol feature extraction methods and rule description methods, such as the range of bytes in each message within a communication session.
[0007] Then, based on the application protocol feature extraction method and rule description method, the AI model extracts protocol features from a sufficient number of application protocol messages based on its learning, and describes them using regular expression rules.
[0008] Next, the AI generates the corresponding application protocol feature rule file according to the feature library file format requirements.
[0009] Finally, the application protocol feature file is imported into the feature database, and the FPGA fast matching engine reads it for application identification.
[0010] Application protocol feature rules are described using regular expressions. To improve matching efficiency, symbols such as "*", "+", and "|" are used sparingly or not at all in the matching patterns. Preferably, feature information is matched in the first few packets of the communication session whenever possible.
[0011] Compared with the prior art, the present invention has the following advantages: 1. This invention utilizes the parallel processing capabilities of FPGA to achieve high-speed protocol identification and filtering. The identification and control method adopted avoids the delay caused by traditional application agents having to pass through the kernel protocol stack to reach the application layer, and then the application layer analyzing and reassembling the data before returning through all protocol stacks. Therefore, it greatly improves data processing performance. 2. The regular expression description of the protocol feature library of the present invention supports user customization and dynamic loading, and can adapt to a variety of application scenarios; 3. This invention employs a whitelist mechanism, allowing only authorized protocols to pass through, which significantly enhances network isolation security and can detect both known and unknown attacks; 4. It can be seamlessly integrated with existing stateful inspection firewalls and other modules, with strong compatibility; 5. The method of the present invention no longer uses traditional, simple string matching, but intelligently decodes various commonly used protocols and can simultaneously check their application layer data. Attached Figure Description
[0012] Figure 1 This is a schematic diagram of the FPGA-based network gateway application protocol whitelist identification and control method of the present invention. Detailed Implementation
[0013] The invention will now be described in further detail with reference to the accompanying drawings.
[0014] This invention proposes an FPGA-based application protocol whitelist identification and control method for network gateways. Current Linux kernels already implement packet state detection and filtering functions. Building upon this, this invention proposes a dedicated application protocol detection engine that integrates application layer protocol parsing technology into the state detection mechanism, enabling the network gateway to more effectively detect and control application layer protocols. The implementation process includes application protocol traffic splitting, application layer data reassembly, and protocol identification and detection based on application protocol characteristics. The application protocol detection function based on protocol characteristics is implemented by an FPGA hardware module to achieve high-speed processing capabilities.
[0015] like Figure 1 As shown, when data packets enter the network gateway system, they are first split according to the destination port of the application protocol. Then, data packets from the same data stream undergo application layer data reassembly and aggregation. The reassembled data is sent to the FPGA feature recognition module, which uses a pre-set protocol feature library to identify and detect the application protocol, and analyzes its compliance and security. If the protocol detection fails, the data stream is cut off, and all data packets are discarded. If the protocol detection passes, the data packets from the data stream are sent to the stateful packet filtering module for further fine-grained detection based on user-configured access control rules. Once the stateful detection also passes, the data packets are forwarded normally.
[0016] The application protocol feature extraction in the protocol feature library involves extracting protocol features and writing standardized language based on the RFC specifications, implementation standards, and related communication requirements of each protocol. RFCs (Request For Comments) contain almost all important textual information about the Internet. Users can write different protocol features according to specific applications to achieve identification and control of different application protocols. Protocol features are described using regular expressions, with each protocol having its own separate file, usually named after the protocol. For example, the detection file for the HTTP protocol is http.swt, and its contents are as follows.
[0017] # HTTP - HyperText Transfer Protocol - RFC 2616 # Usually runs on port 80 # This pattern has been tested and is believed to work well. # http # Status-Line = HTTP-Version SP Status-Code SP Reason-Phrase CRLF (rfc 2616) http / (0\.9|1\.0|1\.1) [1-5][0-9][0-9] [\x09-\x0d -~] *(connection:|content-type:|content-length:|date:)|post [\x09-\x0d - ~]* http /
[01] \.
[019] In the code above, the first line without comments, "http", is the name of the protocol, which must be the same as the protocol name specified in the protocol detection command, and must also match the file name "http.swt"; the second line is a regular expression used to match the content of the application layer data stream payload.
[0018] The feature file database stores detection feature files for various application protocols. The application protocol detection engine supports loading multiple detection rule files for different protocols simultaneously to flexibly meet users' actual application needs.
[0019] The network gateway system provides default feature rule files for commonly used application protocols, including HTTP, FTP, and SSH. Additionally, the system offers a dedicated interface for modifying application protocol detection templates. Users can use this interface to modify protocol detection features according to specific needs, achieving precise detection and control of application protocols.
[0020] To improve matching efficiency, it is important to write appropriate regular expressions, which mainly includes the following points.
[0021] First, the matching pattern should be neither too specific nor too broad.
[0022] Example 1: Using "bear" to match "Bearshare" is too broad, which will result in matching many non-"Bearshare" links, such as "http: / / bear.com".
[0023] Example 2: Using "220 .*ftp.*(\[.*\]|\(.*\))" to match FTP is too specific, because not all servers will send "()" or "[]" after sending "220".
[0024] Second, the matching process should be as simple as possible.
[0025] The use of "*", "+" and "|" in the matching pattern should be minimized as much as possible, because matching these characters can be very time-consuming.
[0026] Third, matching should be completed in the first few packets that appear, if possible.
[0027] For example, for the FTP protocol, the matching pattern can be written as "^220[\x09-\x0d -~]*\x0d\x0aUSER[\x09-\x0d -~]*\x0d\x0a331", which will not complete the match until the third packet appears. However, if "^220[\x09-\x0d -~]*ftp" is used, the match will be completed when the first packet appears.
[0028] The method of this invention no longer employs traditional, simple string matching, but intelligently performs full decoding of various commonly used protocols while simultaneously inspecting their application layer data. It offers enhanced security and is capable of detecting both known and unknown attacks.
[0029] In addition, the identification and control method adopted in this invention avoids the delay caused by traditional application agents having to pass through the kernel protocol stack to reach the application layer, and then the application layer analyzing and reassembling the data before returning through all the protocol stacks, thus greatly improving data processing performance.
[0030] The above description, in conjunction with specific preferred technical solutions, provides a further detailed explanation of the present invention and should not be construed as limiting the specific implementation of the invention to these descriptions. It should be noted that those skilled in the art can make various modifications without departing from the principles of the invention, and these modifications should also be considered to fall within the scope of protection of the present invention.
Claims
1. An application protocol identification control method based on AI learning and FPGA fast matching, characterized in that, The method comprises the following steps: S1: data packet shunting: shunting the data packets entering the gateway according to the destination port, and distinguishing the data streams of different application protocols; S2: application layer data recombination: recombining and converging the data packets of the same data stream, and recovering the application layer data content; S3: FPGA protocol feature recognition: sending the recombined data into the FPGA module, which is internally provided with a plurality of protocol feature libraries, and identifying and judging the compliance of the application protocol through the regular expression matching mode; S4: white list control decision: if the protocol identification passes and meets the white list strategy, the data stream enters the subsequent state detection and packet filtering module; if the identification does not pass, all the messages of the data stream are directly discarded; S5: state detection and forwarding: performing fine-grained access control through the state detection and packet filtering module, and finally forwarding the legal data packets.
2. The application protocol recognition control method according to claim 1, characterized by, The protocol feature library is constructed based on AI learning algorithm for application protocol feature information extraction, and the application protocol feature rules are described by regular expressions and support user customization and dynamic loading.
3. The application protocol recognition control method according to claim 2, wherein The application protocol feature extraction and construction based on AI learning algorithm comprises the following processes: S31: feeding a large number of different application protocol communication data into an AI special learning model, and informing the AI model of the application protocol feature extraction method and rule description method; S32: the AI model extracts the protocol features from a sufficient number of application protocol messages based on the learning and the application protocol feature extraction method and rule description method, and describes the protocol features by regular expressions; S33: the AI generates the corresponding application protocol feature rule file according to the feature library file format requirements; S34: importing the application protocol feature file into the feature database, and reading and identifying the application by the FPGA fast matching engine.
4. The application protocol recognition control method of claim 2, wherein The regular expression does not use or uses few "*", "+" and "|" symbols in the matching mode when written.
5. The application protocol recognition control method of claim 2, wherein The feature information is matched in the first few packets of the communication session.