A method and system for detecting webshell files

Abstract

This application discloses a method and system for detecting webshell files, relating to the field of network security technology. The method includes the following steps: converting the code in the script file to be detected into a first abstract syntax tree (AST), and replacing special identifiers in the AST with general tags; restoring the replaced AST to a first script file, and obtaining the file features of the first script file and the metadata features of the script file to be detected; inputting the file features and metadata features into a trained classification model, and determining whether the script file to be detected is a webshell file based on the detection results of the classification model. This application relies on a constructed classification model to analyze and detect webshell files, reducing the demand on system resources and effectively identifying highly concealed and unknown webshell files.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular to a method and system for detecting webshell files. Background Technology

[0002] Webshell is a code execution environment that exists in the form of web scripts such as ASP, ASPX, JSP, and PHP. Network administrators can use it to manage servers and website backends, while attackers can use webshells to execute system commands, steal data, implant viruses, extort core data, and perform malicious operations such as SEO hacking on web servers, which are extremely harmful. In order to evade security detection, attackers often use strategies such as code obfuscation and encryption to disguise webshells.

[0003] Existing machine learning detection solutions cannot restore the actual content of encoded, encrypted, or obfuscated webshell files through traditional preprocessing and feature extraction processes, thus affecting the accuracy of detection and leading to false positives and false negatives. Summary of the Invention

[0004] This application provides a method for detecting webshell files, which aims to solve the problem that existing webshell file detection methods cannot detect highly concealed or previously unseen webshell files.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] This application discloses a method for detecting webshell files, comprising the following steps:

[0007] The code in the script file to be detected is converted into a first abstract syntax tree, and the special identifiers in the first abstract syntax tree are replaced with general tags;

[0008] The replaced first abstract syntax tree is restored to the first script file, and the file characteristics of the first script file and the metadata characteristics of the script file to be detected are obtained;

[0009] The file features and metadata features are input into the trained classification model, and the detection results of the classification model are used to determine whether the script file to be detected is a webshell file.

[0010] Preferably, replacing the special identifiers in the first abstract syntax tree with general tags includes:

[0011] Extract the special identifiers from the first abstract syntax tree. The special identifiers include variable names, function names, class names, and method names. Replace the variable names, function names, class names, and method names with general tags.

[0012] Preferably, the process includes the following steps before converting the code in the script file to be tested into a first abstract syntax tree:

[0013] Clean the non-script code characters in the script file to be detected, which include comments and blank lines.

[0014] Preferably, the file features include risk function, file content information entropy, file compression ratio, file content overlap index, longest word in the file, information entropy of the longest word in the file, function nesting depth, array nesting depth, loop nesting depth, and judgment nesting depth.

[0015] The metadata features include file size, modification date, creation date, and last access date.

[0016] Preferably, the methods for obtaining the function nesting depth, array nesting depth, loop nesting depth, and the method for determining the nesting depth include:

[0017] The first abstract syntax tree after recursive traversal is modified according to the type of the new structure block when entering a new structure block, until the traversal is completed, and each maximum depth is used as the final value of the corresponding nesting depth.

[0018] Preferably, the training process of the classification model includes:

[0019] Obtain the original sample file and convert the code in the original sample file into a second abstract syntax tree;

[0020] Replace the special identifiers in the second abstract syntax tree with general tags, and restore the second abstract syntax tree after replacement to the first sample file;

[0021] The file features of the first sample file and the metadata features of the original sample file are obtained, and the file features of the first sample file and the metadata features of the original sample file are optimized using a recursive feature elimination method.

[0022] The optimized feature data is input into the xgboost model for training to obtain a classification model for webshell files.

[0023] Preferably, the method further includes:

[0024] The detection results are manually judged to determine whether they are correct. If the result is a false alarm, the type of the script file to be detected is re-labeled, and the classification model is incrementally trained based on the script file to be detected.

[0025] A webshell file detection system includes:

[0026] The conversion module is used to convert the code in the script file to be detected into a first abstract syntax tree and replace the special identifiers in the first abstract syntax tree with general tags.

[0027] The extraction module is used to restore the replaced first abstract syntax tree to the first script file, and to obtain the file features of the first script file and the metadata features of the script file to be detected.

[0028] The classification module is used to input the file features and metadata features into the trained classification model, and determine whether the script file to be detected is a webshell file based on the detection results of the classification model.

[0029] An electronic device includes a memory and a processor, the memory being used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement a webshell file detection method as described in any one of the preceding descriptions.

[0030] A computer-readable storage medium storing a computer program that, when executed by a computer, implements a method for detecting webshell files as described in any one of the above descriptions.

[0031] The present invention has the following beneficial effects:

[0032] This application relies on a constructed classification model to analyze and detect webshell files, reducing the demand on system resources and effectively identifying highly concealed and unknown webshell files. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart of a webshell file detection method provided in an embodiment of this application;

[0035] Figure 2 This is a schematic diagram of the structure of a webshell file detection system provided in an embodiment of this application;

[0036] Figure 3 This is a schematic diagram of an electronic device that implements a method for detecting webshell files, as provided in an embodiment of this application. Detailed Implementation

[0037] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] The terms “first,” “second,” etc., used in the claims and description of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate. This is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0039] Example

[0040] like Figure 1 As shown, this application provides a method for detecting webshell files, including the following steps:

[0041] S110. Convert the code in the script file to be detected into a first abstract syntax tree, and replace the special identifiers in the first abstract syntax tree with general tags.

[0042] After obtaining the script file to be detected, it is preprocessed. In this embodiment, the preprocessing includes two parts: cleaning and code abstraction.

[0043] Cleaning refers to removing non-script code characters from the script file to be tested. Preferably, non-script code characters include comments and blank lines. This cleaning can reduce data noise and ensure that subsequent analysis targets the core content of the script file.

[0044] Next, code abstraction is performed. Specifically, php-parser is used to convert the code in the cleaned script files to be tested into a first abstract syntax tree. php-parser is a PHP library designed to convert PHP code into an abstract syntax tree (AST) and extract special identifiers such as variable names, function names, class names, and method names from the first AST. These special characters are then replaced with generic characters, such as renaming variables to var1, var2, and var3. This eliminates personalized information in the code, allowing similar code structures to be recognized as having the same or similar results even when using different special identifiers.

[0045] S120. Restore the replaced first abstract syntax tree to the first script file, and obtain the file characteristics of the first script file and the metadata characteristics of the script file to be detected.

[0046] The preprocessed files will be used for file feature extraction. Specifically, the file features include risk function, file content information entropy, file compression ratio, file content overlap index, longest word in the file, information entropy of the longest word in the file, function nesting depth, array nesting depth, loop nesting depth, and judgment nesting depth.

[0047] In this embodiment, the replaced first abstract syntax tree is converted into a script file, namely the first script file. The file features such as the risk function, file content information entropy, file compression ratio, file content overlap index, longest word in the file, and information entropy of the longest word in the file are all determined based on the first script file. This is prior art and will not be elaborated here.

[0048] Nesting depth features, such as function nesting depth, array nesting depth, loop nesting depth, and judgment nesting depth, are obtained based on the first abstract syntax tree after identifier replacement. Specifically, the syntax tree is recursively traversed. Each time a new structural block, such as an if statement, while loop, for loop, function call, or array, is entered, the nesting depth type can be determined by adding one based on the type of the structural block. During the traversal, the maximum depth encountered is recorded. This maximum value is the final value of the corresponding nesting depth. Nesting depth features can intuitively represent the complexity of the code. Highly complex code may often have malicious intent.

[0049] At the same time, the metadata features of the script file to be detected are extracted. These metadata features include file size, modification date, creation date, and last access date, which can help identify abnormal behavior. For example, frequent modifications to a file in a short period of time may indicate that the file has been maliciously modified.

[0050] S130. Input the file features and metadata features into the trained classification model, and determine whether the script file to be detected is a webshell file based on the detection result of the classification model.

[0051] The file features extracted from the first script file and the metadata features extracted from the script file to be detected are input into the trained classification model. The classification model analyzes these features and outputs the detection results. Based on the detection results, it can be determined whether the script file to be detected is a webshell file. Then, a human is used to determine whether the detection result is correct. If it is a false alarm, the human judgment result is fed back to the system for review by security experts to ensure that the false alarm label is accurate. Based on the feedback result, the system re-labels the type of the script file to be detected as a normal file or a webshell file, and incrementally trains the classification model accordingly.

[0052] The training process of the classification model specifically includes the following steps:

[0053] Obtain the original sample file and convert the code in the original sample file into a second abstract syntax tree;

[0054] Replace the special identifiers in the second abstract syntax tree with general tags, and restore the second abstract syntax tree after replacement to the first sample file;

[0055] The file features of the first sample file and the metadata features of the original sample file are obtained, and the file features of the first sample file and the metadata features of the original sample file are optimized using a recursive feature elimination method.

[0056] The optimized feature data is input into the xgboost model for training to obtain a classification model for webshell files.

[0057] The processing method for the original sample files is the same as that for the script files to be detected. First, preprocessing is performed to remove non-script code characters from the original sample files. Then, the code in the cleaned original sample files is converted into a second abstract syntax tree (RBT), replacing variable names, function names, class names, and method names with generic tags. The replaced RBT is then converted into a first sample file, and the file features of the first sample file and the metadata features of the original sample files are extracted. Next, the Recursive Feature Elimination (RFE) method is used to optimize and remove all extracted features. Preferably, the RFE class from the scikit-learn library is used, with a target feature count of 10 and an iteration step size of 1, meaning that the least important feature is removed in each step. The sample files are divided into training and validation sets, and the RFE is fitted on the training data. After RFE training is completed, the xgboost model is trained using the features selected by RFE and the model is saved. Then, the performance of the model after feature selection is verified using an independent test set. The performance indicators of the model, such as accuracy and recall, are recorded. Then, the number of features in RFE is reduced by 1. The above steps are repeated until the performance indicators of the model on the test set no longer improve. The model with the best performance indicators is used as the final model, which is the classification model for webshell files. If the performance of the model is not good, it can be optimized in two ways: (1) Adjust the xgboost parameters learning rate, tree depth and number of trees, (2) Increase training data and retrain. More training data can help the model learn more data patterns and features, reduce over-reliance on individual samples, and thus improve the model's generalization ability.

[0058] This application can effectively detect novel webshell files that are highly concealed or have never appeared before, with high accuracy and low system resource consumption.

[0059] like Figure 2 As shown, this application also provides a webshell file detection system, including a conversion module, an extraction module, and a classification module, wherein:

[0060] The conversion module is used to convert the code in the script file to be detected into a first abstract syntax tree, and replace the special identifiers in the first abstract syntax tree with general tags;

[0061] The extraction module is used to restore the replaced first abstract syntax tree to the first script file, and to obtain the file features of the first script file and the metadata features of the script file to be detected.

[0062] The classification module is used to input the file features and metadata features into the trained classification model, and determine whether the script file to be detected is a webshell file based on the detection results of the classification model.

[0063] like Figure 3 As shown, this application also provides an electronic device, including a memory 301 and a processor 302. The memory 301 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 302 to implement the above-described method for detecting webshell files.

[0064] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the electronic device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0065] This application also provides a computer-readable storage medium storing a computer program that, when executed by a computer, enables a webshell file detection method as described above.

[0066] For example, a computer program can be divided into one or more modules / units. One or more modules / units are stored in memory 301 and executed by processor 302. Data I / O interface transmission is completed by input interface 305 and output interface 306 to complete the present invention. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions. The instruction segments are used to describe the execution process of the computer program in the computer device.

[0067] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device may include, but is not limited to, a memory 301 and a processor 302. Those skilled in the art will understand that this embodiment is merely an example of a computer device and does not constitute a limitation on the computer device. It may include more or fewer components, or a combination of certain components, or different components. For example, the computer device may also include an input device 307, a network access device, a bus, etc.

[0068] The processor 302 can be a central processing unit (CPU), or other general-purpose processor 302, digital signal processor 302 (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor 302 can be a microprocessor 302, or any conventional processor 302, etc.

[0069] The memory 301 can be an internal storage unit of a computer device, such as a hard disk or RAM. The memory 301 can also be an external storage device of a computer device, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 301 can include both internal and external storage units. The memory 301 is used to store computer programs and other programs and data required by the computer device. The memory 301 can also be used for temporary storage in the output device 308. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM) 303, random access memory (RAM) 304, discs, or optical discs.

[0070] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting webshell files, characterized in that, Includes the following steps: The code in the script file to be detected is converted into a first abstract syntax tree, and the special identifiers in the first abstract syntax tree are replaced with general tags; The replaced first abstract syntax tree is restored to the first script file, and the file characteristics of the first script file and the metadata characteristics of the script file to be detected are obtained; The file features and metadata features are input into the trained classification model, and the detection results of the classification model are used to determine whether the script file to be detected is a webshell file. The file features include risk function, file content information entropy, file compression ratio, file content overlap index, longest word in the file, information entropy of the longest word in the file, function nesting depth, array nesting depth, loop nesting depth, and judgment nesting depth. The metadata features include file size, modification date, creation date, and last access date.

2. The method for detecting webshell files according to claim 1, characterized in that, The step of replacing special identifiers in the first abstract syntax tree with general tags includes: Extract the special identifiers from the first abstract syntax tree. The special identifiers include variable names, function names, class names, and method names. Replace the variable names, function names, class names, and method names with general tags.

3. The method for detecting webshell files according to claim 1, characterized in that, Before converting the code in the script file to be tested into a first abstract syntax tree, the following steps are also included: Clean the non-script code characters in the script file to be detected, which include comments and blank lines.

4. The method for detecting webshell files according to claim 1, characterized in that, The methods for obtaining the function nesting depth, array nesting depth, loop nesting depth, and methods for determining the nesting depth include: The first abstract syntax tree after recursive traversal is modified according to the type of the new structure block when entering a new structure block, until the traversal is completed, and each maximum depth is used as the final value of the corresponding nesting depth.

5. The method for detecting webshell files according to claim 1, characterized in that, The training process of the classification model includes: Obtain the original sample file and convert the code in the original sample file into a second abstract syntax tree; Replace the special identifiers in the second abstract syntax tree with general tags, and restore the second abstract syntax tree after replacement to the first sample file; The file features of the first sample file and the metadata features of the original sample file are obtained, and the file features of the first sample file and the metadata features of the original sample file are optimized using a recursive feature elimination method. The optimized feature data is input into the xgboost model for training to obtain a classification model for webshell files.

6. The method for detecting webshell files according to claim 1, characterized in that, The method further includes: The detection results are manually judged to determine whether they are correct. If the result is a false alarm, the type of the script file to be detected is re-labeled, and the classification model is incrementally trained based on the script file to be detected.

7. A webshell file detection system, characterized in that, include: The conversion module is used to convert the code in the script file to be detected into a first abstract syntax tree and replace the special identifiers in the first abstract syntax tree with general tags. The extraction module is used to restore the replaced first abstract syntax tree to the first script file, and to obtain the file features of the first script file and the metadata features of the script file to be detected. The classification module is used to input the file features and metadata features into the trained classification model, and determine whether the script file to be detected is a webshell file based on the detection results of the classification model. The file features include risk function, file content information entropy, file compression ratio, file content overlap index, longest word in the file, information entropy of the longest word in the file, function nesting depth, array nesting depth, loop nesting depth, and judgment nesting depth. The metadata features include file size, modification date, creation date, and last access date.

8. An electronic device, characterized in that, The system includes a memory and a processor, the memory being used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement a webshell file detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, The computer program enables the computer to implement a webshell file detection method as described in any one of claims 1 to 6 when executed.

Images