Web mining attack detection method and system based on deep learning and multi-dimensional feature recognition
By combining deep learning and multi-dimensional feature recognition methods with VGG-16 and BiLSTM models, end-to-end detection of web mining attacks was achieved, solving the problem of insufficient detection capabilities in existing technologies and improving the accuracy and robustness of detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- THE THIRD RES INST OF MIN OF PUBLIC SECURITY
- Filing Date
- 2023-09-25
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies are insufficient to effectively detect web mining attacks, especially in real-world scenarios where they suffer from insufficient recall, low precision, inability to handle feature extraction biases caused by code obfuscation and human factors, and poor generalization ability.
By employing a deep learning-based and multi-dimensional feature recognition approach, and training a VGG-16-based web mining traffic detection model and a BiLSTM-based web mining JavaScript script detection model, combined with multi-dimensional feature fusion, end-to-end web mining attack detection is achieved. The raw traffic and scripts are directly used as input, eliminating the need for manual feature design.
It improves the generalization ability and robustness of the detection method, reduces false positives and false negatives caused by JavaScript code obfuscation, and improves the accuracy and recall rate of web mining attack detection.
Smart Images
Figure CN119696803B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a technology in the field of information security, specifically a method and system for detecting web mining attacks based on deep learning and multi-dimensional feature recognition. Background Technology
[0002] Originally, cryptocurrency mining was designed to provide legitimate website owners or non-profit organizations with alternative income streams, replacing traditional web advertising. Due to its lightweight and highly covert nature, it was quickly exploited by attackers and gradually replaced traditional software / Trojan mining. Attackers exploit website vulnerabilities to inject mining scripts, hijacking users' computing resources for personal profit. However, because cryptocurrency mining uses relatively low thresholds for host computing resources, and because communication between cryptocurrency mining and mining pools is mixed with normal browser traffic, traditional software / Trojan mining detection techniques are ineffective in detecting it. To prevent cryptocurrency mining attacks, researchers have proposed various detection methods, including those based on static and dynamic website features. While these methods perform well on known datasets, they still have some problems in real-world scenarios, such as insufficient recall, low precision, inability to handle code obfuscation and hiding techniques, feature extraction bias caused by human factors, and poor generalization ability. Summary of the Invention
[0003] To address the aforementioned shortcomings of existing technologies, this invention proposes a web-based cryptocurrency mining attack detection method and system based on deep learning and multi-dimensional feature recognition. This method detects mining behavior from two aspects: mining network traffic and mining JavaScript scripts. By directly using raw traffic data and JavaScript scripts as input, end-to-end web-based cryptocurrency mining attack detection is achieved without the need for manual feature design, thereby improving the generalization ability and robustness of the detection method.
[0004] This invention is achieved through the following technical solution:
[0005] This invention relates to a web mining attack detection method based on deep learning and multi-dimensional feature recognition. In the offline stage, the collected traffic is preprocessed and grayscale images are generated, which are then used to train two web mining traffic detection models based on VGG-16. The extracted JavaScript scripts are deobfuscated and character-replaced, and then used to train two web mining JavaScript script detection models based on BiLSTM. In the online stage, the trained web mining traffic detection models and web mining JavaScript script detection models are used for classification, and the classification results are fused using multi-dimensional features to achieve web mining attack detection.
[0006] The BiLSTM-based JavaScript detection model for web mining includes a download unit, a deobfuscation unit, a character replacement processing unit, a first detection model unit, and a second detection model unit. Specifically: the download unit downloads all JavaScript files from the webpage using an asynchronous downloader and stores the extracted code in the corresponding JS files. The deobfuscation unit uses the Esprima tool to parse the obfuscated JavaScript script into an Abstract Syntax Tree (AST). It then traverses the nodes of the AST using a reconstruction algorithm, retaining nodes such as variables, arrays, and functions to restore the JavaScript script. The character replacement processing unit replaces URLs with "url", numbers with "0", and string function parameters with "param" in the JavaScript script. It also performs special character deletion and word segmentation, obtaining the word vector matrix of the JavaScript script using a Word2Vec model. The first detection model unit uses training sets of normal JavaScript scripts, non-mining malicious JavaScript scripts, and mining malicious JavaScript scripts to distinguish between normal and malicious scripts using a BiLSTM model. The second detection model unit uses training sets of non-mining malicious JavaScript scripts and mining malicious JavaScript scripts to distinguish between non-mining malicious scripts and mining malicious scripts through a BiLSTM model.
[0007] This invention relates to a web mining attack detection system that implements the above-described method, comprising: a simulated access unit, a data preprocessing unit, a grayscale image generation unit, and a traffic detection unit. The simulated access unit uses the Selenium tool to simulate a Chrome browser accessing the webpage link to be detected in headless mode, based on user input, and captures the webpage's communication traffic in a closed Docker environment, saving it as a pcap file. The data preprocessing unit extracts session data from the collected traffic, randomizes MAC and IP addresses, removes invalid data packets, and generates a grayscale image as input. The grayscale image generation unit of a first web mining traffic detection model M1 based on VGG-16 performs binary classification on the generated grayscale image to distinguish between malicious and non-malicious traffic. The traffic detection unit of a second web mining traffic detection model M2 based on VGG-16 performs binary classification on malicious traffic, further distinguishing between mining malicious traffic and non-mining malicious traffic, thereby obtaining a three-classification result for the traffic.
[0008] Technical effect
[0009] Compared with existing technologies, this invention eliminates the need for manual feature design, enhancing the generalization ability and robustness of the web mining traffic detection model; it reduces false positives and false negatives caused by JavaScript code obfuscation, and enhances the detection capability of mining JavaScript scripts; the web mining attack detection system can detect more mining websites while maintaining a low false positive and false negative rate, exhibiting high precision and recall. Attached Figure Description
[0010] Figure 1 This is a system view of the present invention;
[0011] Figure 2 A schematic diagram of the architecture of a web mining traffic detection model based on VGG-16;
[0012] Figure 3 This is a schematic diagram of the architecture of a JavaScript script detection model for web mining based on BiLSTM. Detailed Implementation
[0013] like Figure 1 As shown in the figure, this embodiment relates to a web mining attack detection method based on deep learning and multi-dimensional feature recognition, including:
[0014] Step 1) Receive the webpage link to be tested input by the user. In a closed Docker environment with limited CPU usage, use the Selenium tool to simulate the Chrome browser accessing the URL in headless mode. Stay on the webpage for 45 seconds, use tshark to capture the webpage's communication traffic, save the communication traffic as a pcap file, and at the same time use an asynchronous downloader to download all the js files carried in the webpage, and save the code extracted from the script tags of the html file into the js files respectively.
[0015] Step 2) Preprocess the collected traffic and generate a grayscale image, and deobfuscate and replace the extracted JavaScript script.
[0016] The preprocessing refers to: extracting traffic data from the pcap packets according to the session; for TCP sessions, the start and end of the session are identified by the SYN and FIN fields respectively; for UDP sessions, a timeout mechanism is used; storing each session separately into its own pcap packet; then, to prevent host characteristics from being used as a basis for judging malicious traffic, randomizing the MAC address and IP address; removing invalid data packets such as ARP packets, DNS packets, and retransmission packets from the session; finally, after removing the header information, truncating the first 1024 bytes of each session, padding any bytes shorter than 1024 bytes with 0s; converting each byte of the sliced session data into an 8-bit grayscale pixel; converting the sliced session data into a 32×32 grayscale image to generate a dataset.
[0017] The aforementioned deobfuscation refers to: using the Esprima tool to parse the obfuscated JavaScript script into an abstract syntax tree (AST), and according to the code obfuscation method, using the abstract syntax tree reconstruction algorithm described in (Ma Hongliang, Wang Wei, Han Zhen. Research on detection and deobfuscation methods of obfuscated malicious JavaScript code [J]. Chinese Journal of Computers, 2017, 40(7): 1699-1713.), traversing all nodes of the abstract syntax tree, and if the node is a variable, array, function, etc., then the node is retained, and the JavaScript script is restored by traversing the tree.
[0018] The character replacement processing refers to: replacing all URLs in the JavaScript script with "url", replacing all numbers in the JavaScript script with "0", replacing all strings used as function parameters in the JavaScript script with "param", deleting tabs and special characters such as control characters, using the word segmentation tool NLTK to segment the generalized JavaScript script data, and using the Word2Vec model provided by the gensim library to calculate the vector representation of each word in the JavaScript script to obtain the word vector matrix of the JavaScript script.
[0019] Step 3) Input the generated grayscale image into the first VGG-16-based web mining traffic detection model M1 after training for binary classification. Then, input the data classified as malicious traffic into the second VGG-16-based web mining traffic detection model M2 after training for binary classification to obtain the three-class classification result of the traffic. Input the preprocessed JavaScript script into the first BiLSTM-based web mining JavaScript script detection model M3 for binary classification. Then, input the data classified as malicious JavaScript script into the second BiLSTM-based web mining JavaScript script detection model M4 for binary classification to obtain the three-class classification result of the JavaScript script.
[0020] The first VGG-16-based webpage mining traffic detection model refers to a model trained using training sets of three types of traffic data: normal traffic, non-mining malicious traffic, and mining malicious traffic. Specifically, during training, both non-mining malicious traffic and mining malicious traffic are considered malicious traffic. The specific parameter settings are as follows: the optimization method uses SGD, the learning rate is 0.0001, the batch size is 32, the loss function of the classification result is calculated in each training epoch, and the network parameters are updated. Training stops after 50 training epochs.
[0021] The second VGG-16-based webpage mining traffic detection model refers to a model trained using training sets of three types of traffic data: normal traffic, non-mining malicious traffic, and mining malicious traffic. This model distinguishes between mining malicious traffic and non-mining malicious traffic. Specifically, the optimization method used is SGD with a learning rate of 0.0001 and a batch size of 32. The loss function of the classification result is calculated in each training epoch, and the network parameters are updated. Training stops after 50 training epochs.
[0022] The training sets for the three types of traffic data—normal traffic, non-mining malicious traffic, and mining malicious traffic—are obtained as follows: traffic data for normal traffic and non-mining malicious traffic are obtained from the public dataset USTC-TFC2016, and traffic data for mining malicious traffic is obtained from the public dataset SmaSheD. A dataset containing the three types of traffic data—normal traffic, non-mining malicious traffic, and mining malicious traffic—is generated and stored in pcap format.
[0023] The first BiLSTM-based JavaScript detection model for web mining refers to a model trained using a training set of three types of JavaScript scripts: normal JavaScript scripts, non-mining malicious JavaScript scripts, and mining malicious JavaScript scripts. Specifically, both non-mining malicious JavaScript scripts and mining malicious JavaScript scripts are considered malicious. The specific parameter settings are as follows: the optimization method uses Adam, the learning rate is 0.001, the batch size is 128, the loss function of the classification result is calculated in each training epoch, and the network parameters are updated. Training stops after 50 training epochs.
[0024] The second BiLSTM-based JavaScript detection model for web mining refers to a model trained using training sets of both non-mining malicious JavaScript scripts and mining malicious JavaScript scripts to distinguish between them. Specifically, the optimization method used is Adam, with a learning rate of 0.001 and a batch size of 128. The loss function of the classification result is calculated in each training epoch, and the network parameters are updated. Training stops after 50 training epochs.
[0025] The training set containing three types of JavaScript scripts—normal JavaScript scripts, non-mining malicious JavaScript scripts, and mining malicious JavaScript scripts—was obtained as follows: non-mining malicious JavaScript scripts were obtained from the public dataset JavaScript Malware Collection on GitHub; the top 200 websites recorded by Alexa were considered normal websites, and the JavaScript scripts in the source code of these web pages were obtained using a web crawler as normal JavaScript scripts; and mining malicious JavaScript scripts were obtained from web pages in Coinblocklist's recent blacklist. This generated a dataset containing the three types of JavaScript scripts: normal JavaScript scripts, non-mining malicious JavaScript scripts, and mining malicious JavaScript scripts.
[0026] Step 4) The three-class classification results of traffic and the three-class classification results of JavaScript scripts are fused using multi-dimensional features to obtain the detection results of web page links. Specifically, this includes: when the web page mining traffic detection model determines it to be malicious mining traffic, the website is determined to be an ongoing web page mining attack; when the web page mining script detection model determines it to be a malicious mining script, but the web page mining traffic detection model does not determine it to be malicious mining traffic, the website is determined to be a potential web page mining attack with high confidence; when both the web page mining traffic and script detection models determine it to be normal, the website is determined to be a normal website; in other cases, the website is determined to be a potential web page mining attack with low confidence.
[0027] Based on specific practical experiments, under the specific environment settings of Ubuntu 18.04 64-bit system, using Python 3.9 and PyTorch 1.13.1, the effects of the web mining traffic detection model, web mining script detection model, and web mining attack detection system involved in this embodiment are as follows:
[0028] The dataset used by the webpage mining traffic detection model contains 313,454 data points, including 138,431 normal traffic points and 175,023 malicious traffic points. Among the malicious traffic, 83,459 are mining-related malicious traffic points and 91,564 are non-mining malicious traffic points. The classification accuracy of models M1 and M2, as well as the precision, recall, and F1-score for each type of traffic, are shown in the table below:
[0029] The dataset used by the basic web mining detection model contains 6540 data entries, including 3651 normal scripts and 2889 malicious scripts. Among the malicious scripts, 769 are mining malicious scripts and 2120 are non-mining malicious scripts. The classification accuracy of models M3 and M4, as well as the precision, recall, and F1 score of each script class, are shown in the table below:
[0030] The dataset used for the performance evaluation of the web mining attack detection system contains 60 websites, including 20 normal websites, 20 websites containing unobfuscated mining scripts that run normally, 10 websites containing obfuscated mining scripts that run normally, and 10 websites containing mining scripts that do not run. The precision of the web mining attack detection system is 100%, the recall is 98.91%, and the F1 score is 99.45%.
[0031] Compared with existing technologies, the web mining traffic detection model of this invention directly uses raw traffic as data input to achieve end-to-end web mining traffic detection. It eliminates the need for manual feature design, enhancing the model's generalization ability and robustness. The web mining script detection model, after deobfuscating the obfuscated JavaScript scripts, uses the Doc2Vec model to extract the script's code feature vectors and leverages BiLSTM to classify the JavaScript code, identifying malicious mining JavaScript scripts. This enhances the detection capability of malicious web mining scripts and reduces false positives and false negatives caused by JavaScript code obfuscation. The web mining attack detection system combines the advantages of both the web mining traffic detection model and the web mining script detection model, maintaining low false positives and false negatives while detecting more web mining attacks, demonstrating high precision and recall, and is suitable for practical application scenarios.
[0032] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.
Claims
1. A method for detecting web mining attacks based on deep learning and multi-dimensional feature recognition, characterized in that, In the offline phase, the collected traffic was preprocessed and grayscale images were generated, which were then used to train two VGG-16-based web mining traffic detection models. The extracted JavaScript scripts were deobfuscated and character-replaced, and then used to train two BiLSTM-based web mining JavaScript script detection models. In the online phase, the trained web mining traffic detection models and web mining JavaScript script detection models were used for classification, and the classification results were fused with multi-dimensional features to achieve web mining attack detection. The BiLSTM-based JavaScript detection model for web mining includes a download unit, a deobfuscation unit, a character replacement processing unit, a first detection model unit, and a second detection model unit. Specifically: the download unit downloads all JavaScript files from the webpage using an asynchronous downloader and stores the extracted code in the corresponding JS file; the deobfuscation unit uses the Esprima tool to parse the obfuscated JavaScript into an abstract syntax tree (AST), traversing the nodes of the AST through a reconstruction algorithm, retaining variable, array, and function nodes to restore the JavaScript; the character replacement processing unit performs character replacement, deletion of special characters, and word segmentation in the JavaScript, obtaining the word vector matrix of the JavaScript using the Word2Vec model; the first detection model unit uses training sets of normal JavaScript, non-mining malicious JavaScript, and mining malicious JavaScript to distinguish between normal and malicious scripts using a BiLSTM model; the second detection model unit uses training sets of non-mining malicious JavaScript and mining malicious JavaScript to distinguish between non-mining malicious scripts and mining malicious scripts using a BiLSTM model.
2. The web mining attack detection method based on deep learning and multi-dimensional feature recognition according to claim 1, characterized in that, The webpage mining traffic detection model includes: a simulated access unit, a data preprocessing unit, a grayscale image generation unit, and a traffic detection unit. Specifically: the simulated access unit uses the Selenium tool to simulate a Chrome browser accessing the webpage link in headless mode based on user input, and captures the webpage's communication traffic in a closed Docker environment, saving it as a pcap file; the data preprocessing unit extracts session data from the collected traffic, randomizes MAC and IP addresses, removes invalid data packets, and generates a grayscale image as input; the grayscale image generation unit of the first webpage mining traffic detection model M1 based on VGG-16 performs binary classification on the generated grayscale image to distinguish between malicious and non-malicious traffic; the traffic detection unit of the second webpage mining traffic detection model M2 based on VGG-16 performs binary classification on malicious traffic, further distinguishing between mining malicious traffic and non-mining malicious traffic, thus obtaining a three-classification result for the traffic.