A regular matching method and device for Shenwei platform

By using the assembly instruction program in the regular feature library on the Shenwei platform for regular matching, directly mapped to memory space and using DFA algorithm, the problem of low regular expression matching performance in the existing technology is solved, and efficient network security is achieved Detection performance and easy-to-scaling feature library management.

CN118819547BActive Publication Date: 2024-11-19CLP KESHENTAI INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411312941.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-20
Publication Date
2024-11-19
Estimated Expiration
2044-09-20

AI Technical Summary

Technical Problem

The matching performance of regular expressions on the Shenwei platform is low, especially when the real-time requirements for specific regular expressions in network security detection are high, there is room for performance improvement.

Method used

By using the assembly instruction program in the regular feature library on the Shenwei platform for regular matching, it is directly mapped to the memory space for execution, and using the DFA algorithm for matching, avoiding the intermediate layer of high-level languages ​​and interpreters, and improving the matching speed.

Benefits of technology

It significantly improves the regular expression matching speed of Shenwei platform in network security detection, improves processing efficiency, and simplifies the process of adding new regular expressions, making it efficient and easy to use.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118819547B_ABST
    Figure CN118819547B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of information communication and network security technology, and particularly relates to a regular matching method and device of the Shenwei platform. It includes: obtaining a regular expression and message data to be retrieved; identifying and parsing the input regular expression according to the current regular feature library; wherein the regular feature library includes a regular matching type and a corresponding assembly instruction program, and the regular matching assembly instruction program is written using the Shenwei instruction set; mapping the corresponding assembly instruction program to the memory space according to the matching type of the regular expression; and using the regular matching algorithm in the assembly instruction program to perform regular matching on the message data. This method does not require the compilation of other software, which improves the matching speed of the Shenwei platform in processing specific regular expressions and improves the performance of Shenwei in network security detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of information communication and network security technology, and in particular relates to a regular matching method and device for a Shenwei platform. Background Art

[0002] Currently, there are two main types of libraries commonly used for regular matching. One is a branch library under a large library, such as boost regex and glibc regex. This library depends on boost and glibc libraries, and the program is complex and the compilation time is long. The other is a regular expression engine specially designed, such as hyperscan, RE2 and pcre. Although hyperscan library is a library for high-performance text search, it mainly runs on the x86 platform and relies on modern instruction sets such as SSE4.2 and AVX2. Therefore, the matching regular expression performance on the Shenwei platform is greatly limited, which is not as good as other regular matching libraries. RE2 library is a C++-based regular library provided by Google. Compared with assembly language, the machine code generated by high-level languages ​​like C++ is more complex and has lower execution efficiency. In addition, the machine code generated by high-level languages ​​requires the support of intermediate layers such as interpreters or virtual machines, which consumes more resources, resulting in lower execution efficiency of regular matching. The JIT technology of the pcre library requires that the assembly instruction program be dynamically generated using the current regular expression and C code program before each regular match is executed. This process will affect the performance of regular matching on the Shenwei platform. In addition, PCRE JIT uses the NFA regular algorithm, which is less efficient. Therefore, the existing technology on the Shenwei platform can still have room for improvement for applications that match specific regular expressions and have high real-time requirements.

[0003] When conducting network security testing, communication messages are mainly searched for specific regular expressions such as IP addresses, account passwords, and special strings. Therefore, in order to improve the performance of the Shenwei platform in network security testing, a method and device that can improve the matching speed of specific regular expressions is needed. Summary of the invention

[0004] The purpose of the present invention is to provide a regular matching method and device for the Shenwei platform. When the Shenwei platform performs text character search and network security detection, the method does not need to be compiled by other software, thereby improving the matching speed of the Shenwei platform in processing specific regular expressions and improving the performance of Shenwei in network security detection.

[0005] In order to solve the above technical problems, the present invention provides a regular matching method of the Shenwei platform, comprising the following steps:

[0006] Step 1: Obtain the regular expression and the message data to be retrieved;

[0007] Step 2: Identify and parse the input regular expression according to the current regular feature library;

[0008] The regular feature library includes regular matching types and corresponding assembly instruction programs, and the regular matching assembly instruction program is written using the Shenwei instruction set;

[0009] Step 3: Map the corresponding assembly instruction program into the memory space according to the matching type of the regular expression;

[0010] Step 4: Use the regular matching algorithm in the assembly instruction program to perform regular matching on the message data.

[0011] Preferably, the regular expression includes the IP address, HTTP URL, user name and password stored in the feature library; for each regular expression in the feature library, the corresponding assembly instruction program is written based on the Shenwei instruction set, and the regular matching type is marked and stored in the feature library.

[0012] Preferably, in the method, each regular expression and the assembly instruction program are bound via a matching type assigned in a feature library, and the matching type is represented by a uniquely identifiable ID number.

[0013] Preferably, the parsing and matching algorithm of the regular expression does not need to go through C language and other high-level languages, but is directly written in the assembly language composed of the Shenwei instruction set. The assembly instruction program can directly control and operate the Shenwei CPU.

[0014] Preferably, if a new regular expression needs to be matched, it is only necessary to add the Shenwei assembly instruction program corresponding to the new regular expression to the cache and add the corresponding matching type ID number.

[0015] The present invention also provides a regular matching device for the Shenwei platform, which adopts a regular matching method for the Shenwei platform as described above, including:

[0016] An acquisition module is used to obtain a regular expression and message data to be matched;

[0017] Recognition module, which is a parser module for regular feature expressions;

[0018] A mapping module is used to map the corresponding assembly instruction program into the Shenwei shared memory space;

[0019] The matching module uses the regular matching algorithm in the assembly instruction program to perform regular matching on the message data.

[0020] Preferably, the parser module specifically includes the following logic:

[0021] When a regular expression is passed in, the parser module will first parse the syntax of the regular expression; if the regular expression meets the grammatical rule requirements, the parser module will use the current regular feature library to identify the regular expression. If the identification is successful, the regular matching type ID number corresponding to the regular expression in the feature library will be returned, and the regular expression will be bound to the corresponding assembly instruction program in the feature library according to the ID number; if the regular expression does not meet the grammatical rule requirements, the parser module will return a syntax error prompt.

[0022] Preferably, the mapping module specifically includes the following logic:

[0023] When the specified regular expression is passed in, the parser module maps the corresponding assembly instruction program to the Shenwei shared memory space and creates a pointer callback function of the Shenwei shared memory space, so that the assembly instruction program can be directly executed in the Shenwei shared memory space.

[0024] Preferably, the matching module specifically includes the following logic:

[0025] After mapping the assembly instruction program to the Shenwei shared memory space, a pointer to the Shenwei shared memory space is returned. A callback function is created through the pointer, and the message pointer and message length to be matched are passed in the callback function and output to the assembly instruction program; the assembly instruction program compares the string with the regular symbols one by one; if the match is successful, the position of the character match in the current string is recorded, and the match continues with the next character in the regular expression; if the match fails, it backtracks to the first character of the regular expression. After the assembly program is executed, the regular matching result of the string is returned.

[0026] Preferably, the regular matching process in the assembly instruction program adopts a regular matching DFA algorithm.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. High efficiency: The present invention completes the regular matching process of a specific regular expression by loading the Shenwei assembly instruction program in the regular feature library, adopts a more efficient DFA regular algorithm, and directly maps the assembly program to the Shenwei shared memory address space for execution. This not only saves the dynamic generation time of the Shenwei assembly instruction program, but also avoids the parsing and loading process of traditional codes. The matching speed of a specific regular expression can be significantly improved, thereby making the Shenwei platform more efficient in processing network security detection.

[0029] 2. Usability and expansibility: If a new regular expression match needs to be added in the method of the present invention, it is only necessary to add the Shenwei assembly instruction program corresponding to the new regular expression and the corresponding matching type number in the cache. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of a regular matching method of the Shenwei platform provided by the present invention.

[0031] Figure 2 It is a binding relationship diagram of the feature library provided by the present invention.

[0032] Figure 3 It is a structural block diagram of a regular matching device of the Shenwei platform provided by the present invention.

[0033] Figure 4 It is a logic flow chart of the parser module provided by the present invention.

[0034] Figure 5 It is a logic flow chart of the matching module provided by the present invention. DETAILED DESCRIPTION

[0035] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description. It should be noted that the accompanying drawings are in very simplified form and in non-precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0036] like Figure 1 As shown, an embodiment of the present invention provides a regular matching method of the Shenwei platform, comprising the following steps:

[0037] Step 1: Obtain the regular expression and the message data to be retrieved;

[0038] Step 2: Identify and parse the input regular expression according to the current regular feature library;

[0039] The regular feature library includes regular matching types and corresponding assembly instruction programs, and the regular matching assembly instruction program is written using the Shenwei instruction set;

[0040] Step 3: Map the corresponding assembly instruction program into the memory space according to the matching type of the regular expression;

[0041] Step 4: Use the regular matching algorithm in the assembly instruction program to perform regular matching on the message data.

[0042] The method implements the DFA algorithm of regular matching in the Shenwei assembly instruction program, and directly maps the instruction program to the memory address space for execution. The assembly instruction program is written and saved in advance, and these specific regular expressions are saved in the regular feature library.

[0043] This method uses the generated assembly program to perform regular matching in the Shenwei memory space. It does not rely on complex program structure and long compilation time, and saves the time of dynamic generation of assembly instruction program, which improves the speed and efficiency of regular matching. Specific regular expressions include commonly used regular expressions such as IP address, HTTP URL, user name and password, which have been saved in the feature library. For each regular expression in the feature library, the corresponding assembly instruction program is written based on the Shenwei instruction set, and the regular matching type is marked and stored in the feature library. In this method, each regular expression and assembly instruction program are bound through the matching type assigned in the feature library, and the matching type is represented by a uniquely identifiable ID number, such as Figure 2 shown.

[0044] The parsing and matching algorithm of regular expressions does not need to go through C language and other high-level languages, but is directly written in the assembly language composed of the Shenwei instruction set. The assembler can directly control and operate the Shenwei CPU, and has higher execution efficiency, so the matching speed of time-consuming regular expressions can be significantly improved. This method has good ease of use and scalability. If you need to add a new regular expression match, you only need to add the Shenwei assembly instruction program corresponding to the new regular expression in the cache, and add the corresponding matching type ID number.

[0045] like Figure 3 As shown, an embodiment of the present invention provides a regular matching device for the Shenwei platform, which includes four functional modules, namely, an acquisition module, an identification module, a mapping module and a matching module.

[0046] Acquisition module: used to obtain regular expressions and message data to be matched.

[0047] The recognition module can also be called the parser module of regular features. Figure 4 As shown, the recognition module includes the following logical flow: when a regular expression is passed in, the parser will first parse the syntax of the regular expression. For regular expressions that do not meet the grammatical rule requirements, the parser returns an error prompt. If the regular expression meets the grammatical rule requirements, the parser will use the current regular feature library to recognize the regular expression. If the recognition is successful, the corresponding regular type (ID number) of the regular expression in the feature library is returned, and the regular expression is bound to the corresponding assembly instruction program in the feature library according to the ID number.

[0048] Mapping module: When the specified regular expression is passed in, the parsing module maps the corresponding assembly instruction program to the Shenwei shared memory space, and creates a pointer callback function of the Shenwei shared memory space, so that the assembly instruction program can be directly executed in the Shenwei shared memory space.

[0049] like Figure 5 As shown, the matching module includes the following logical flow: after mapping the assembly instruction program to the Shenwei shared memory space, a pointer to the Shenwei shared memory space is returned, a callback function is created through the pointer, the message pointer and message length to be matched are passed in the pointer callback function, and output to the assembly instruction program. The regular matching process in the assembly instruction program adopts the regular matching DFA algorithm. The DFA algorithm is more efficient than the NFA algorithm. The program compares the string with the regular symbol one by one. If the match is successful, the position of the character match in the current string is recorded, and the match with the next character in the regular expression is continued. If the match fails, it backtracks to the first character of the regular expression. After the assembly instruction program is executed, the regular matching result of the string is returned.

[0050] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.

Claims

1. A regular matching method for the Shenwei platform, It is characterized in that The steps include: Step 1: Obtain the regular expression and the message data to be retrieved; Step 2: Identify and parse the input regular expression according to the current regular feature library; The regular feature library includes regular matching types and corresponding assembly instruction programs, and the regular matching assembly instruction program is written using the Shenwei instruction set; Step 3: Map the corresponding assembly instruction program into the memory space according to the matching type of the regular expression; Step 4: Perform regular matching on the message data using the regular matching algorithm in the assembly instruction program, wherein the regular matching algorithm in the assembly instruction program adopts the DFA regular algorithm; Also includes: The mapping module specifically includes the following logic: When the specified regular expression is passed in, the parser module maps the corresponding assembly instruction program to the Shenwei shared memory space and creates a pointer callback function of the Shenwei shared memory space, so that the assembly instruction program is directly executed in the Shenwei shared memory space; The matching module specifically includes the following logic: After mapping the assembly instruction program to the Shenwei shared memory space, a pointer to the Shenwei shared memory space is returned. A callback function is created through the pointer, and the message pointer and message length to be matched are passed in the callback function and output to the assembly instruction program; the assembly instruction program compares the string with the regular symbols one by one; if the match is successful, the position of the character match in the current string is recorded, and the match continues with the next character in the regular expression; if the match fails, it backtracks to the first character of the regular expression. After the assembly program is executed, the regular matching result of the string is returned.

2. A regular matching method for the Shenwei platform as claimed in claim 1, It is characterized in that The regular expression includes the IP address, HTTP URL, user name and password stored in the feature library; for each regular expression in the feature library, the corresponding assembly instruction program is written based on the Shenwei instruction set, and the regular matching type is marked and stored in the feature library.

3. A regular matching method for the Shenwei platform as claimed in claim 2, It is characterized in that In this method, each regular expression and assembly instruction program are bound through a matching type assigned in a feature library, and the matching type is represented by a uniquely identifiable ID number.

4. A regular matching method for the Shenwei platform as claimed in claim 1, It is characterized in that The parsing and matching algorithm of the regular expression does not need to go through C language and other high-level languages, and can be directly written in the assembly language composed of the Shenwei instruction set. The assembly instruction program can directly control and operate the Shenwei CPU.

5. A regular matching method for the Shenwei platform as claimed in claim 1, It is characterized in that If you need to add a new regular expression match, you only need to add the Shenwei assembly instruction program corresponding to the new regular expression in the cache and add the corresponding matching type ID number.

6. A canonical matching device for a Shenwei platform, using a canonical matching method for a Shenwei platform as described in any one of claims 1 to 5, It is characterized in that include: An acquisition module is used to obtain a regular expression and message data to be matched; Recognition module, which is a parser module for regular feature expressions; A mapping module is used to map the corresponding assembly instruction program into the Shenwei shared memory space; The matching module uses the regular matching algorithm in the assembly instruction program to perform regular matching on the message data; The mapping module specifically includes the following logic: When the specified regular expression is passed in, the parser module maps the corresponding assembly instruction program to the Shenwei shared memory space and creates a pointer callback function of the Shenwei shared memory space, so that the assembly instruction program is directly executed in the Shenwei shared memory space; The matching module specifically includes the following logic: After mapping the assembly instruction program to the Shenwei shared memory space, a pointer to the Shenwei shared memory space is returned, and a callback function is created through the pointer. The message pointer and message length to be matched are passed into the callback function and output to the assembly instruction program; the assembly instruction program compares the string with the regular symbols one by one; if the match is successful, the position of the character match in the current string is recorded, and the match with the next character in the regular expression continues; if the match fails, it backtracks to the first character of the regular expression, and the regular matching result of the string is returned after the assembly program is executed; The regular matching algorithm in the assembly instruction program adopts the DFA regular algorithm.

7. A regular matching device for the Shenwei platform as claimed in claim 6, It is characterized in that The parser module specifically includes the following logic: When a regular expression is passed in, the parser module will first parse the syntax of the regular expression; if the regular expression meets the grammatical rule requirements, the parser module will use the current regular feature library to identify the regular expression. If the identification is successful, the regular matching type ID number corresponding to the regular expression in the feature library will be returned, and the regular expression will be bound to the corresponding assembly instruction program in the feature library according to the ID number; If the regular expression does not meet the grammatical rule requirements, the parser module returns a syntax error.