A Malicious URL Detection Method Based on Hybrid Binary Neural Tree

Abstract

This invention relates to a malicious URL detection method based on a hybrid binary neural tree, belonging to the field of information security technology. The method includes: acquiring URL data and performing preprocessing and annotation; constructing a URL dataset; constructing a malicious URL detection network based on a hybrid binary neural tree, introducing the interpretable branching decision mechanism of decision trees and the automatic spatial feature extraction characteristics of neural networks, and fitting the extraction, enhancement, and detection of local URL features into a complete binary tree from top to bottom; training the network using a URL training set and a validation set, and inputting the URLs to be detected from the test set into the trained network to detect whether the URLs possess malicious attributes. This invention, based on the character-level features of URLs, flexibly integrates various functional neural network models into each node of the decision tree model, greatly improving the detection accuracy of malicious URLs.

Description

Technical Field

[0001] This invention belongs to the field of information security technology, and relates to the fields of natural language processing and text mining. Specifically, it relates to a malicious URL detection method based on a hybrid binary neural tree. Background Technology

[0002] As internet companies race to release more new features and functions, more new vulnerabilities are emerging. Attackers exploit these vulnerabilities for profit, such as through malicious emails, malicious advertising, financial fraud, and malware. While the economic or commercial motives behind these incidents vary widely, the basic approach involves users visiting malicious websites, searching online, or clicking on related links. These approaches share a common thread: they require user clicks and searches, all of which correspond to Uniform Resource Locators (URLs). Therefore, detecting malicious URLs has become a top priority.

[0003] Currently, traditional malicious URL detection is accomplished using blacklist matching. However, this method has a fatal flaw: malicious URLs typically only exist for a few hours or even minutes, and their number grows exponentially, making it difficult to match valid malicious URLs in real time. As an extension of blacklisting, heuristic methods can identify common network attacks and assign a label to each type of attack based on expert experience. Specifically, this method looks for labels of suspicious behavior by analyzing the execution dynamics of web pages, such as abnormal process creation and repeated redirects. However, heuristic methods still require execution in a controlled environment (e.g., a virtual machine) and are very resource-intensive. Since machine learning models offer the ability to predict new and unknown URLs, numerous methods using machine learning models to detect malicious URLs have emerged. Malicious URL detection using machine learning first extracts appropriate lexical statistical features from the URL and then uses these features to train a machine learning prediction model. However, research on machine learning-based malicious URL detection requires a complex subjective feature selection process, and the resulting feature representation of the URL is not particularly convincing.

[0004] In recent years, deep learning methods have become increasingly popular for automatic feature extraction, and these features are then used to train neural network prediction models. Character-level CNNs and RNNs, the minimal variants of neural networks, have achieved good results in the timely detection of malicious URLs. Furthermore, many malicious website classification methods based on natural language processing exist, such as those using two state-of-the-art pre-trained language models, BERT and ELECTRA, where fine-tuned pre-trained language models outperform models trained using custom vocabularies based on URLs. However, the development of these advanced models requires significant computational resources and hardware support, and is extremely time-consuming. For unstructured data like URLs, many of these neural network models can only partially capture the local spatial and sequence features of the URL, failing to fully demonstrate the interpretability of neural network models. Therefore, how to efficiently mine URL information and effectively integrate multiple feature vectors has become a pressing problem. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a malicious URL detection method based on a hybrid binary neural tree, which solves the problem of low detection accuracy caused by insufficient URL character-level local feature representation information in existing deep learning malicious URL detection methods. This method efficiently mines URL information and effectively integrates multiple feature vector information to achieve real-time detection of malicious URLs, thereby improving the accuracy of malicious URL detection.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A malicious URL detection method based on a hybrid binary neural tree includes the following steps:

[0008] S1: Obtain URLs, label malicious URLs and normal URLs, and perform preprocessing on the labeled URLs, including at least filtering of garbled characters and URL length consistency specification.

[0009] S2: Construct a URL dataset and ensure that the proportion of malicious URLs and normal URLs in the training set, validation set, and test set are basically the same;

[0010] S3: Construct a malicious URL detection network based on a hybrid binary neural tree, that is, to integrate decision trees and neural networks to form a model with interpretable character features. This network combines the interpretable branch decision mechanism of decision trees and the automatic spatial feature extraction mechanism of neural networks to fit the extraction, enhancement and detection of local URL feature information into a complete binary tree; wherein, the decision tree is composed of complete binary trees, and the neural network is composed of encoders from CNN and Transformer.

[0011] In the mixed binocular nerve tree In, N h ={n h1 ,n h2 ,...,n hi} is a tree Each of the internal nodes consists of a branch decision module composed of a multi-scale CNN. The detection module D, composed of a single-scale CNN, k (·) constitutes; E h ={e h1 ,e h2 ,...,e hj} represents the connecting edges between internal nodes. Each edge is encoded by the position encoding and multi-head self-attention joint module in the encoder of the Transformer. Composition; Mixed Digeminal Neural Tree Use a complete binary tree structure, i = 1, 2, ..., 2 h-1 j = 1, 2, ..., 2 h k = 1, 2, ..., 2 H-1 h = 1, 2, ..., H, meaning h is a tree of height H. The h-th level is counted starting from the root node;

[0012] S4: Train the malicious URL detection network based on the hybrid binary neural tree using the URL training set, and use the URL validation set to guide the parameter optimization process of the malicious URL detection network based on the hybrid binary neural tree until training is complete; wherein, an improved joint objective function is used to optimize the parameters of the malicious URL detection network based on the hybrid binary neural tree; input the URLs to be detected from the test set into the converged malicious URL detection network based on the hybrid binary neural tree, and output whether the URL has malicious attributes;

[0013] S5: Returns detected malicious URLs to the malicious website blocking and alert plugin, thus warning users that they are visiting potentially malicious websites.

[0014] Furthermore, in step S3, the URL character embedding input of the malicious URL detection network based on hybrid binary neural tree is... A single character can be used as a token, such as "A", "a", "1", "@", "?", etc. This applies to URL character embedding. d L For a URL with a fixed character length, d k For character feature dimensions.

[0015] Furthermore, in step S3, the main network structure of the branch decision module of the malicious URL detection network based on hybrid binary neural tree is a multi-scale CNN, and the operation of obtaining different local features of the URL includes the following steps:

[0016] S301: Embedding URL characters Input to the i-th branch decision module at level h Multi-scale local features are calculated using different convolutional kernels within the same convolutional layer, with kernel size f×d. k ; where d L For a URL with a fixed character length, d k Character feature dimension;

[0017] S302: ReLU activation function and max pooling are used to map and transform the features at each scale, and the multi-scale local features are concatenated to generate fine-grained local features. Where, d L ' represents the concatenation length of local feature characters;

[0018] S303: Generates direction decision probabilities through L2 regularization and fully connected layers with a Sigmoid activation function. Therefore, it can be determined that the URL characters are embedded. In which direction should the flow go to the next feature enhancement module? Character embedding The probability of being sent to the left branch is The probability of being sent to the right branch is then...

[0019] Furthermore, in step S3, the main network structure of the feature enhancement module of the malicious URL detection network based on hybrid binary neural tree is the encoder in Transformer. The operation of enhancing different local features of the URL includes the following steps:

[0020] S311: URL character embedding input to the feature enhancement module The important local feature characters that have been filtered by the branch decision module are embedded, and the remaining characters are automatically zeroed out.

[0021] Through the Position encoding is performed to obtain Obtain the order and position of URL characters, and embed the URL characters. and position encoding Concatenated as a URL union representation

[0022] Important local tokens are used as inputs to multi-head self-attention and mapped to three matrices: Q (Query), K (Key), and V (Value). Q, K, and V represent query, key, and value, respectively. Q and K are obtained by vector dot product and normalization to form a mask matrix with all values ​​from 0 to 1, which is the self-attention matrix. V retains the features of the input text.

[0023] S312: Q and K are obtained by vector dot product and normalization to a mask matrix with all values ​​from 0 to 1, which is the self-attention matrix. V retains the features of the input text.

[0024] S313: Decompose the three matrices into S subspaces, each subspace corresponding to an attention head, and each attention head corresponding to a sequence of length D. Multiple attention heads are connected in the S subspaces via Q. S and K S The dot product is used to calculate the similarity weights of multiple local features; where Q S K S Let represent the submatrices decomposed into one of S subspaces by the Q and K mappings, respectively;

[0025] S314: Compare the local feature similarity of the entire text space with the corresponding input feature information V S We can get u by combining the weighted sums and concatenations. A Feature enhancement representation.

[0026] Furthermore, in step S311, URL characters are embedded. The location code is represented as:

[0027]

[0028]

[0029] in, and The relative positions of characters concatenated using sine and cosine encoding are represented by local features; pos represents the position of each character in the URL, and x represents the position of the character vector.

[0030] Furthermore, in step S311, the URL union vector The enhanced features are represented as:

[0031]

[0032]

[0033] Q S =QW S Q ,K S=KW S K V S =VW S V

[0034] Q = u A "W Q K = u A "W K V=u A "W V

[0035] in, Indicate u A "Enhanced features, head" S Let W represent the S-th attention head, and D represent the length of the corresponding attention head; O W Q W K W V W S Q W S K W S V To optimize gradient descent parameters when training a malicious URL detection network based on a hybrid binary neural tree.

[0036] Furthermore, in step S3, the main network structure of the detection module of the malicious URL detection network based on hybrid binary neural tree is a single-scale CNN. The operation of detecting whether a URL has malicious attributes includes the following steps:

[0037] S321: Enhanced text semantic representation Input to detection module D k (·);

[0038] S322: Using a convolution kernel of size 5×d k The convolutional layers are then followed by fully connected layers with ReLU activation, max pooling, L2 regularization, and softmax activation.

[0039] S323: Calculated The local feature category label is equal to that from the tree The final detection probability is the sum of the local category feature labels of each leaf node, which is the cumulative probability of branch decision from the root to the leaf node and the probability of sub-labels.

[0040] Furthermore, in step S323, the detection probability Φ of the URL attribute is expressed as:

[0041]

[0042] Among them, P k (u D ) is D k (u D ) for u D The probability of local feature category detection. For branch decision module The probability of the resulting decision.

[0043] Furthermore, in step S4, during network training, URL character embedding... As a tree The input is processed sequentially from top to bottom through the tree. The branch decision module acquires character space features from different local areas and obtains the decision result based on these features. This determines the direction of flow to the next feature enhancement module, the tree. The leaf nodes correspond to the enhancement features of different local parts of the URL and are input into the detection module. The final URL detection category depends on the sum Φ of all local enhancement features.

[0044] An improved joint objective function is used to optimize the parameters of a malicious URL detection network based on a hybrid binary neural tree. The objective function consists of three parts: the first part is the final label prediction loss, the second part is the leaf node classification loss due to branch decisions, and the third part is a regularization term that controls the network complexity. Improved Joint Objective Function Represented as:

[0045]

[0046] L(Φ,y)=-(y·log(Φ)+(1-y)·log(1-Φ))

[0047]

[0048]

[0049] Where L(Φ,y) is the tree The final predicted result Φ and the actual class y are related by the binary cross-entropy loss, L(P). k (·),y) is a tree The label discrimination result P of the detection function of the k-th leaf node k (·) Binary cross-entropy loss between the actual class y; Regularization terms for controlling the network complexity of malicious URL detection based on hybrid binary neural trees, including tree... The number of nodes N, the tree The sum of the height H and the embedded network parameters ω of each node, where α, γ, and λ are regularization coefficients.

[0050] During real-time detection, the URL to be tested is input into the hybrid binary neural tree. This will determine whether the test URL has malicious attributes.

[0051] Furthermore, step S5 specifically includes: returning the detected malicious URL to the malicious website blocking plugin and storing it in the blacklist control list of the URL database, detecting potential malicious URLs in real time, and alerting the user that they are visiting a potential malicious website.

[0052] The beneficial effects of this invention are as follows:

[0053] (1) In view of the problem that the detection accuracy is low due to insufficient local feature representation information of URL characters in deep learning malicious URL detection methods, this invention designs a malicious URL detection method based on hybrid binary neural tree. This method integrates the interpretable mechanism of decision tree and the ability of neural network to automatically capture semantic features, so that important local information of URL string can be captured and enhanced, thereby improving the accuracy of malicious URL detection.

[0054] (2) This invention considers the randomness of URL character arrangement and the weak representation of semantic logic. It uses a multi-scale CNN with spatial feature capture capability to obtain URL local character composition features of different granularities in the child nodes of the binary tree. Then, it calculates the flow path direction of the URL vector from top to bottom based on these representative features. The Transformer encoder is used on the connecting edges of the nodes in the binary tree to realize multi-granularity local feature enhancement of the URL character vector, which is used to represent the prominence and difference of each local discrete feature. A single-scale CNN is used in the leaf nodes of the binary tree to maintain the homogeneity of the URL feature representation. The final URL detection attribute is equivalent to the sum of the discrimination results of the local enhanced features in all leaf nodes.

[0055] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0056] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0057] Figure 1 This is a flowchart of the malicious URL detection method based on a hybrid binary neural tree according to the present invention;

[0058] Figure 2 This is a schematic diagram of the malicious URL detection network framework based on a hybrid binary neural tree used in this invention.

[0059] Figure 3 This is a schematic diagram of the branch decision-making module used in this invention;

[0060] Figure 4 This is a schematic diagram of the feature enhancement module used in this invention;

[0061] Figure 5 This is a schematic diagram of the detection module used in this invention;

[0062] Figure 6 This is a schematic diagram of the malicious URL interception plugin device used in this invention. Detailed Implementation

[0063] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0064] Please see Figures 1-6 This invention proposes a malicious URL detection method based on a hybrid binary neural tree, such as... Figure 1 As shown, the method specifically includes the following steps:

[0065] S1: Obtain the URL. Since this invention focuses on detecting malicious URLs, malicious URLs are labeled as positive class 1 and normal URLs are labeled as negative class 0. The labeled URLs are preprocessed, including at least garbled character filtering and URL length consistency specification.

[0066] S2: Construct a URL dataset, ensuring that the proportion of malicious URLs and normal URLs in the training, validation, and test sets is approximately the same. The training and validation sets are used for training and optimizing the hybrid binary neural network model, while the test set is used for performance evaluation of the hybrid binary neural network and to output whether unknown URLs have malicious attributes.

[0067] S3: Construct a malicious URL detection network based on a hybrid binary neural tree. Train the hybrid binary neural tree using the Adam gradient descent algorithm with a fixed learning rate; that is, the input to the hybrid binary neural tree during training is a non-generic URL. Use a URL validation set to guide the parameter optimization process of the hybrid binary neural tree, and use backpropagation to optimize and update the loss generated during training. Set a fixed period for training and validation. When the number of data iterations exceeds a threshold and the validation set metrics do not show significant improvement, stop training and use the trained model for testing.

[0068] S4: Input the URLs to be detected from the test set into the converged hybrid binary neural tree and output whether the URLs have malicious attributes;

[0069] S5: Returns the detected malicious URLs to the network access control list of the malicious website blocking plugin, providing real-time alerts to users who are accessing potentially malicious websites, ultimately allowing users to decide whether to proceed to the next step.

[0070] This invention details the above method in three aspects: data preprocessing, detection network construction, and browser detection plugin extension, and includes the following steps:

[0071] (I) Data Preprocessing

[0072] Malicious and normal URLs are obtained from public websites using web crawling. The preprocessing in this embodiment includes, but is not limited to, data labeling, data cleaning, and exploratory data analysis. Malicious URLs are labeled as positive class 1, and normal URLs are labeled as negative class 0. The labeled URLs undergo preprocessing that includes at least regularization filtering of garbled characters. By visualizing the statistical distribution of URL lengths, the 3σ principle is used to perform consistent padding and truncation of URL lengths, maximizing the preservation of valid URL information.

[0073] A URL dataset is constructed, comprising a training set, a validation set, and a test set, ensuring that the ratio of malicious URLs to normal URLs is approximately equal across all three sets. This URL dataset forms the basis for training the malicious URL detection network. To ensure the experimental results in this embodiment closely resemble real-world network environments, the ratio of malicious URLs to normal URLs is approximately 1:3. To better train and validate the network's detection performance, the training and validation sets are used to train and optimize the hybrid binary neural tree malicious URL detection network, enabling it to adapt to the semantic feature distribution of malicious and normal URLs. The URL test set is used to test the performance of the trained network, with the ratio of malicious URLs to normal URLs also approximately 1:3.

[0074] (II) Detection Network Construction

[0075] A malicious URL detection network based on a hybrid binary neural tree is constructed and trained using a URL training set. For each training task, malicious and normal URLs are obtained from the training set at a ratio of approximately 1:3. During training, malicious and normal URLs are randomly batch-sampled and input into the malicious URL detection network. A validation set is used to guide and optimize the malicious URL detection process until training is complete. During testing, the network outputs whether a URL possesses malicious attributes.

[0076] Figure 2 This invention presents the overall architecture of a malicious URL detection network based on a hybrid binary neural tree. The overall architecture integrates decision trees and neural networks to form a character feature interpretable model. The decision trees are composed of complete binary trees, and the neural networks include multi-scale convolutional neural networks and encoders in Transformers.

[0077] In the mixed binocular nerve tree In, N h ={n h1 ,n h2 ,...,n hi} is a tree The internal nodes are each replaced by a neural network, serving as both the branch decision function and the prediction function; that is, the branch decision module is composed of a multi-scale convolutional neural network. The detection module D, composed of a single-scale convolutional neural network. k (·);E h ={e h1 ,e h2 ,...,e hj} represents the edges connecting internal nodes. Each edge is replaced by a neural network and used as a feature enhancement function, i.e., the joint module of position encoding and multi-head self-attention in the encoder of the Transformer. Mixed Digeminal Neural Tree Use a complete binary tree structure, i = 1, 2, ..., 2 h-1 j = 1, 2, ..., 2 h k = 1, 2, ..., 2 H-1 h = 1, 2, ..., H, meaning h is a tree of height H. The h-th layer is counted starting from the root node.

[0078] Mixed Digeminal Neural Tree URL character embedding input A single character can be used as a token, such as "A", "a", "1", "@", "?", etc. This applies to URL character embedding. d L For a URL with a fixed character length, d k For character features. URL character embedding. Passing through the trees from top to bottom The branch decision module acquires character space features from different local areas and obtains the decision result based on these features. This determines the direction of flow to the next feature enhancement module, the tree. The leaf nodes correspond to the enhancement features of different local parts of the URL and are input into the detection module. The final URL detection category depends on the sum Φ of all local enhancement features.

[0079] An improved joint objective function is used to train the hybrid binary neural tree network. The objective function consists of three parts: the first part is the final label prediction loss; the second part is the leaf node classification loss due to branch decisions; and the third part is a regularization term that controls the network complexity. The improved joint objective function is expressed as:

[0080]

[0081] L(Φ,y)=-(y·log(Φ)+(1-y)·log(1-Φ))

[0082]

[0083]

[0084] Where L(Φ,y) is the tree The final predicted result Φ and the actual class y are related by the binary cross-entropy loss, L(P). k (·),y) is a tree The label discrimination result P of the detection function of the k-th leaf node k (·) Binary cross-entropy loss between the actual class y; For regularization terms used to control the complexity of hybrid binary neural tree networks, including trees The number of nodes N, the tree The sum of the height H and the embedded network parameters ω of each node, where α, γ, and λ are regularization coefficients.

[0085] During the detection process, the URL to be tested is input into the hybrid binomial neural tree. This will determine whether the test URL has malicious attributes.

[0086] Figure 3This is a schematic diagram of the branch decision module in the malicious URL detection network based on a hybrid binary neural tree used in this invention. The main network structure of the branch decision module is a multi-scale CNN. Because malicious URLs, compared to normal URLs, have weaker semantic coherence and logic in their character embeddings, some important features often appear in discrete local special characters. To extract important discrete local features from the URL and use these features as the criteria for branch decisions, this embodiment uses a multi-scale CNN to obtain URL local character composition features of different granularities, and then obtains the results based on these representative features. The probability of the flow path direction.

[0087] Embed URL characters Input to the i-th branch decision module at level h Multi-scale local features are calculated using different convolutional kernels within the same convolutional layer, with kernel size f×d. k Where f = 3, 4, 5, 6, 7. Then, the ReLU activation function and Max-Pooling are used to map and transform the features at each scale, concatenating the multi-scale features to generate fine-grained local features. Directional decision probabilities are generated using L2 regularization and fully connected layers with a Sigmoid activation function. Therefore, it can be determined that the URL characters are embedded. In which direction should the flow go to the next feature enhancement module? Character embedding The probability of being sent to the left branch is The probability of being sent to the right branch is then...

[0088] Figure 4 This is a schematic diagram of the feature enhancement module in the malicious URL detection network based on a hybrid binary neural tree used in this invention. The main network structure of the feature enhancement module is the encoder in a Transformer. The key module of the encoder, multi-head self-attention, helps the neural network model capture richer features. In this embodiment, it helps improve the detection model's ability to learn local features of discrete characters in the URL, and is used to represent the salience and difference of each local discrete feature. The input is fed into the j-th self-attention module A of the h-th layer. h j (·), with the tree As the height increases and the branching decisions of internal nodes change, the breadth and depth of the URL's local feature enhancement will also change accordingly.

[0089] At this point, the URL character embedding in this embodiment... The important local feature characters, filtered by the branch decision module, are embedded, while the remaining characters are automatically zeroed out. This is achieved through... Position encoding is performed to obtain Retrieves the order and position of characters in a URL. URL character embedding. The location code is represented as:

[0090]

[0091]

[0092] in, pos represents the position of each character in the URL, and x represents the position of the character vector.

[0093] Embed URL characters and position encoding Concatenated as a URL union representation Important local tokens are used as input to multi-head self-attention and mapped to three matrices: Q (Query), K (Key), and V (Value). Q and K are multiplied by vectors and normalized to obtain a mask matrix containing all values ​​from 0 to 1, which is the self-attention matrix. V preserves the features of the input text. The three matrices are decomposed into S subspaces, each corresponding to an attention head, and each attention head corresponds to a sequence of length D. Multiple attention heads interact within the S subspaces through Q... S and K S The dot product is used to calculate the similarity weights of multiple local features. Finally, the text local feature similarities across the entire space are compared with the corresponding input feature information V. S The weighted sums and concatenations are used to obtain the URL joint vector. The enhanced features are represented as:

[0094]

[0095] Q S =QW S Q ,K S =KW S K V S =VW S V

[0096] Q = u A "W Q K = u A "W K V=u A "W V

[0097] Among them, W O W Q W K W V W S Q W S K W S V To optimize gradient descent parameters when training a malicious URL detection network based on a hybrid binary neural tree.

[0098] Figure 5 This is a schematic diagram of the leaf node detection module in the malicious URL detection network based on a hybrid binary neural tree used in this invention. The main network structure of the detection module is a single-scale CNN. The single-scale CNN acquires the local alignment features of the URL space after being enhanced by a multi-head self-attention module, and uses these features to identify the malicious attributes of local features in each branch node.

[0099] First, enhance the semantic representation of the text. Input to detection module D k (·). A convolution kernel of size 5×d is used. k The convolutional layer then passes through a fully connected layer with ReLU activation, Max-Pooling, L2 regularization, and a softmax activation. Then, the calculation is... The local feature category label is equal to that from the tree The cumulative branch decision probability from the root to the leaf node and the sub-label probability, the final detection probability is the sum of the local category feature labels of each leaf node. That is, the detection probability of the URL attribute is expressed as:

[0100]

[0101] Among them, P k (u D ) is D k (u D ) for u D The probability of local feature category detection. For branch decision module The probability of the resulting decision.

[0102] (III) Browser detection plugin extension

[0103] This invention provides a malicious URL detection browser plugin device based on a hybrid binary neural tree, such as... Figure 6 As shown, it specifically includes:

[0104] The web interface interaction module allows users to enter the URL link they want to access in the web interface address window.

[0105] The storage module uses a MongoDB database on the server side to store the URLs entered by the user. A network access control table is constructed, including a URL field and an attribute field. The URL field stores the URL link, and the attribute field stores the tag for that URL, with 1 for malicious and 0 for legitimate. The attribute field is initialized to null before passing the hybrid binary neural tree detection.

[0106] The algorithm script module encapsulates a hybrid binary neural tree network detection algorithm, calls the API to detect malicious attributes of the currently input URL, and returns the detection results to the network access control list on the server.

[0107] If the web access control module detects a malicious URL, it will pop up a window to warn the user that the website may pose a potential danger, and the user can decide for themselves whether to continue accessing the website.

[0108] In a preferred embodiment, when a new URL link appears, the new URL is cleaned, preprocessed, and classified and labeled. Malicious URL character embeddings and normal URL character embeddings are randomly selected and input into the pre-trained network. The model parameters are iteratively optimized so that the model can adapt to the detection of new malicious URLs.

[0109] This invention enables rapid malicious URL detection by selecting different types and quantities of datasets at different stages. Furthermore, this invention implements a malicious URL detection method based on a hybrid binary neural tree, capable of detecting and identifying malicious URLs in web interface URL input windows.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1.A method for detecting malicious URLs based on hybrid binary neural tree, characterized in that, The method comprises the following steps: S1: obtaining a URL, marking malicious URLs and normal URLs, and pre-processing the marked URLs; S2: constructing a URL data set, and ensuring that the proportions of malicious URLs and normal URLs in the training set, the validation set and the test set are the same; S3: constructing a malicious URL detection network based on a mixed binary neural tree, that is, fusing a decision tree and a neural network to form a character feature interpretable model, the network combines a branch decision mechanism of the decision tree and an automatic spatial feature extraction mechanism of the neural network, and extracts, enhances and detects local feature information of the URL in a complete binary tree; wherein the decision tree is composed of a complete binary tree, and the neural network is composed of a CNN and an encoder in a Transformer; In the mixed binary neural tree , N h ={n h1 ,n h2 ,...,n hi} are internal nodes of the tree , each of which is composed of a branch decision module composed of multi-scale CNNs and a detection module D k (·) composed of single-scale CNNs; E h ={e h1 ,e h2 ,...,e hj} are edges connecting between internal nodes, each of which is composed of a position encoding in the encoder of the Transformer and a multi-head self-attention joint module ; the mixed binary neural tree adopts a full binary tree structure, i.e., i=1,2,...,2 h-1 , j=1,2,...,2 h , k=1,2,...,2 H-1 , h=1,2,...,H, that is, h is the hth layer counted from the root node of the tree with height H; S4: training the malicious URL detection network based on the mixed binary neural tree using a URL training set, and guiding a parameter optimization process of the malicious URL detection network based on the mixed binary neural tree using a URL validation set until the training is completed; wherein an improved joint objective function is used to optimize the parameters of the malicious URL detection network based on the mixed binary neural tree; the to-be-detected URL of the test set is input into the converged malicious URL detection network based on the mixed binary neural tree, and whether the URL has a malicious attribute is output; S5: returning the detected malicious URL to a malicious website interception and reminder plug-in to realize the warning effect on the user accessing a potential malicious website. 2.The hybrid binary neural tree based malicious URL detection method of claim 1, wherein, In step S3, the network structure of the branch decision module of the malicious URL detection network based on the mixed binary neural tree is a multi-scale CNN, and the operation of obtaining different local features of the URL comprises the following steps: S301: embedding URL characters input into the hth layer and ith branch decision module The multi-scale local features are calculated by different convolution kernels of the same convolution layer, and the size of the convolution kernel is fxd k ; wherein d L is the fixed character length of the URL, and d k is the character feature dimension; S302: Map and transform each scale feature using ReLU activation function and maximum pooling, and concatenate the multi-scale local features to generate fine-grained local features where d L is the concatenation length of local feature characters S303: Generate direction decision probability by L2 regularization and full connection layer with Sigmoid activation function Thus, judge the URL character embedding Should flow into the next feature enhancement module in which direction; wherein, Character embedding The probability of being sent into the left branch is The probability of being sent into the right branch is 3.The hybrid binary neural tree based malicious URL detection method of claim 2, wherein, In step S3, the network structure of the feature enhancement module of the malicious URL detection network based on the mixed binary neural tree is an encoder in a Transformer, and the operation of enhancing different local features of the URL comprises the following steps: S311: URL character embedding input to the feature enhancement module For important local feature character embeddings screened by the branch decision module, the rest of the characters are automatically zeroed out; By position encoding ​​​​ The important local token is taken as the input of the multi-head self-attention and is mapped into three matrices of Q, K and V; wherein, Q, K and V represent query, key and value respectively, Q and K obtain a mask matrix with all values of 0-1, i.e. a self-attention matrix, through vector dot multiplication and normalization, and V retains the features of the input text; In step S312, Q and K obtain a mask matrix with all values between 0 and 1 through vector dot product and normalization, that is, a self-attention matrix, and V retains the features of the input text; S313: decompose the three matrices into S subspaces, each corresponding to one attention head and each attention head corresponding to a sequence of length D, and the plurality of attention heads are in the S subspaces through Q S and the point multiplication of K S to obtain the similarity weight of the plurality of local features; wherein Q S , K S respectively represent the sub-matrix decomposed from Q, K mapping into one of the S subspaces. S314: The text local feature similarity of the whole space is spliced with the corresponding input feature information V S to obtain the weighted sum of u A feature enhanced representation of " ". 4.The method of claim 3, wherein, In step S311, the URL character embedding is represented as: wherein, and The relative position of the local feature spliced character is represented by the cosine encoding; pos is the position of each character in the URL, and x represents the position of the character vector. 5.The hybrid binary neural tree based malicious URL detection method of claim 3, wherein, In step S311, the URL joint vector The enhanced feature representation is expressed as: Q S = QW S Q K S = KW S K V S = VW S V Q = u A "W Q K = u A "W K V = u A "W V wherein, denotes u A ” to enhance the feature, head S denotes the S-th attention head, D denotes the length corresponding to the attention head; W O , W Q , W K , W V , W S Q , W S K , W S V is the optimization parameter of gradient descent when training the malicious URL detection network based on the hybrid binary neural tree. 6.The hybrid binary neural tree based malicious URL detection method of claim 3, wherein, In step S3, the network structure of the detection module of the malicious URL detection network based on the mixed binary neural tree is a single-scale CNN, and the operation of detecting whether the URL has a malicious attribute comprises the following steps: S321: the enhanced textual semantic representation is input to the detection module D input to the detection module D k (·); S322: by a convolution kernel size of 5 x d k of ReLU activation function, max-pooling, L2 regularization and full connection layer carrying softmax activation function; S323: Calculate The local feature class label of the tree The final detection probability is the sum of the local class feature labels of each leaf node. 7.The hybrid binary neural tree based malicious URL detection method of claim 6, wherein, In step S323, the detection probability of the URL attribute is represented as: Among them, P k (u D ) is D k (u D ) for u D The probability of local feature category detection. For branch decision module The probability of the resulting decision. 8.The hybrid binary neural tree based malicious URL detection method of claim 1, wherein, In step S4, the improved joint objective function includes three parts: the final label prediction loss, the leaf node classification loss due to branch decision, and the regularization term for controlling network complexity; then the improved joint objective function is represented as: L(Φ,y)=-(y·log(Φ)+(1-y)·log(1-Φ)) Where L(Φ,y) is the tree The final predicted result Φ and the actual class y are related by the binary cross-entropy loss, L(P). k (·),y) is a tree The label discrimination result P of the detection function of the k-th leaf node k (·) Binary cross-entropy loss between the actual class y; For regularization terms used to control the complexity of hybrid binary neural tree networks, including trees The number of nodes N, the tree The sum of the height H and the embedded network parameters ω of each node, where α, γ, and λ are regularization coefficients.

Images