A malicious URL detection method integrating a malicious dictionary and a BERT pre-trained model

Abstract

This invention relates to a malicious URL detection method integrating a malicious dictionary and a BERT pre-trained model, belonging to the field of information security technology. The method includes: collecting malicious vocabulary to construct a malicious dictionary, obtaining URL page text information, preprocessing it to construct a URL page text content dataset; constructing a text content detection network integrating the malicious dictionary and the BERT pre-trained model, using the BERT pre-trained model to extract features from the text and generate text vectors containing contextual information; obtaining malicious weight vectors of the text sequence through a malicious value calculation module, and then obtaining a malicious weighted vector of the text vectors; passing the malicious weighted vector through a multi-scale convolution module to extract features at different scales and generate multi-scale feature vectors, and calculating the malicious probability, thereby detecting malicious URLs. This invention significantly enhances the accuracy and stability of malicious URL detection, providing stronger protection for information security.

Description

Technical Field

[0001] This invention belongs to the field of information security technology and relates to a malicious URL detection method that integrates a malicious dictionary and a BERT pre-trained model. Background Technology

[0002] Malicious URLs are commonly used for spreading viruses, phishing attacks, and information theft, causing significant losses to users and businesses. Although various malicious URL detection technologies exist, most methods still have limitations and fail to fully capture the complexity of URL text and the relationship between it and malicious words.

[0003] Most existing detection technologies rely on static features or simple rule matching, such as blacklists and keyword matching. However, these methods often cannot cope with new variants and disguises of malicious URLs. At the same time, traditional machine learning methods often rely on manual selection for feature extraction, resulting in insufficient generalization ability of the models and difficulty in adapting to rapidly changing malicious attack patterns.

[0004] In recent years, the application of deep learning technology has brought new opportunities for malicious URL detection. In particular, the remarkable achievements of BERT pre-trained models in the field of natural language processing demonstrate their powerful ability to understand contextual information.

[0005] However, detection methods that rely solely on the BERT model or a malicious dictionary still have certain limitations and cannot fully consider the multidimensional features of the text. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a malicious URL detection method that integrates a malicious dictionary and a BERT pre-trained model. This method constructs a malicious dictionary by collecting sensitive words, pornographic words, violent and terrorist words, fraudulent words, and reactionary words, and obtains and preprocesses URL page text data to construct a text content dataset. Based on this, a text content detection network integrating the malicious dictionary and the BERT pre-trained model is designed. The BERT pre-trained model is used to extract features from the text, generating text vectors containing contextual information. These vectors are then multiplied by malicious value weights to obtain a malicious weighted vector. This weighted vector is then passed through a multi-scale convolution module to extract features at different scales, and finally, the predicted malicious classification result is output through Softmax. This ultimately achieves accurate classification of malicious URLs.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A malicious URL detection method integrating a malicious dictionary and a BERT pre-trained model, comprising the following steps:

[0009] S1: Collect malicious words to construct a malicious dictionary W, and obtain URL page text information, preprocess it, and construct a URL page text content dataset;

[0010] S2: Construct a text content detection network that integrates a malicious dictionary and a BERT pre-trained model. The text content detection network includes an input layer, a BERT pre-trained model, a malicious value calculation module, a multi-scale convolution module, and a Softmax classifier.

[0011] S3: Use the BERT pre-trained model to extract features from the text and generate a text vector H containing contextual information. L ;

[0012] S4: Obtain the malicious weight vector of the text sequence through the malicious value calculation module, and then combine it with the text vector H. L The malicious weighted vector H′ is obtained through calculation. L ;

[0013] S5: Transfer the malicious weighted vector H′ L After passing through the multi-scale convolution module, features at different scales are extracted and multi-scale feature vectors M are generated. The malicious probability is then calculated using Softmax.

[0014] S6: Use the constructed dataset to train the text content detection network, save the parameter configuration file after training, and encapsulate it into an interface for external calls.

[0015] Furthermore, in step S1, for the malicious dictionary W, the collected malicious words are first processed to form the malicious dictionary:

[0016] W = {W1, W2, ..., W} j , ..., W M}

[0017] Among them W j Let M represent the j-th word, and M be the total number of words in the malicious dictionary;

[0018] For the URL page text content dataset, malicious and normal URLs are obtained from public websites. The obtained URL text content is then preprocessed, including: removing irrelevant words, removing stop words, right truncation, and annotation. Right truncation involves padding statements shorter than the maximum length L with zeros and truncating words after L for statements longer than L. Annotation assigns malicious URL page text to the positive class 1 and normal URL page text to the negative class 0.

[0019] The preprocessed text content is divided into training and validation sets according to the preset ratio of malicious and normal URL page text and the preset ratio of training and validation set data volume.

[0020] Furthermore, in step S2, a malicious URL text content detection network integrating a malicious dictionary and a BERT pre-trained model is established. The text content detection network includes an input layer, a BERT pre-trained model, a malicious value calculation module, a multi-scale convolution module, and a Softmax classifier. The input layer takes text content as input and uses the BERT pre-trained model to extract features from the text. The malicious value calculation module is used to calculate the malicious weight vector. The multi-scale convolution module extracts features at different scales and outputs the final predicted malicious classification result through Softmax.

[0021] Furthermore, in step S3, the BERT pre-trained model extracts features from the input text content according to the following steps:

[0022] S31. Segment the preprocessed text into words, and mark the beginning of each sentence with the symbol [cls] and the end with [sep] to obtain the text sequence T:

[0023] T = {x cls x1, x2, ..., x i , ..., x L x sep}

[0024] The length of the text sequence T is L+2, x i This represents the i-th character in the text;

[0025] S32. The text sequence T is encoded using word vectors, sentence encoding, and positional encoding to obtain word vectors:

[0026] E wi ={e wcls e w1 e w2 , ..., e wi , ..., e wL e wsep}

[0027] Sentence vector:

[0028] E si ={e scls e s1 e s2 , ..., e si , ..., e sL e ssep}

[0029] Position vector:

[0030] E pi ={e pcls e p1 e p2 , ..., e pi , ..., e pL e psep}

[0031] The final vector representation of the input to the BERT pre-trained model is: E bi =E wi +E si +E pi ;

[0032] S33, Input vector E bi The text is passed sequentially to multiple encoding layers. Each encoding layer performs multi-head self-attention calculation, layer normalization, residual connection, and fully connected layer processing, finally generating a text vector H containing contextual information. L .

[0033] Furthermore, in step S4, the malicious value calculation module calculates the malicious weight vector according to the following steps:

[0034] S41. The malicious value calculation module first performs Jieba segmentation on the preprocessed text data to obtain the set of words to be matched V and the set of word counts to be matched V′:

[0035] V = {V1, V2, ..., V} i , ..., V N}

[0036] V′={V′1, V′2,...,V′ i ,...,V′ N}

[0037] Where V i V′ represents the i-th word. i This represents the number of characters in the i-th word;

[0038] S42. Map the words to be matched and malicious words in the malicious dictionary into word vectors using the Word2Vec model. Calculate the similarity between the word to be matched and the malicious word, and establish a mapping relationship between the malicious word and the word to be matched; the formula for calculating the similarity between the malicious word and the word to be matched is:

[0039]

[0040] Among them, V i W represents the i-th word. j sim(V) represents the j-th word. i Wj ) represents V i With W j Word similarity between them; find malicious entries in the dictionary that are similar to V through similarity calculation. i The word with the highest matching degree, W j ;

[0041] S43, The malicious value of the defined term is:

[0042]

[0043] Among them, WMS(V i ) represents V i The score in the malicious dictionary, if max_sim(V i When ) is greater than or equal to the threshold σ, then WMS(V) i ) is 2V′ i Conversely, it is V′. i ;where V′ i This represents the number of characters in the i-th word;

[0044] The malicious value of each word is evenly distributed among each character in that word, and the malicious value of a character is represented as follows:

[0045]

[0046] Among them, P k This represents the malicious value of the k-th character in the sentence, which belongs to the word V. i The malicious value of each word in the sentence and the identifiers [cls] and [sep] is calculated. The malicious value of the identifiers [cls] and [sep] is 1 by default.

[0047] S44. Construct a malicious value vector from the malicious values ​​of each word calculated above:

[0048] P = {P} cls P1, P2, ..., P k , ..., P L P sep}

[0049] The obtained malicious value vector P is normalized to obtain the malicious weight vector P′. For a sentence of length L, one term P′ in its weight vector P′ is... k for:

[0050]

[0051] Finally, the text vector H output by the Bert pre-trained model is... L Multiplying the malicious weight vector P′ by the malicious weight vector yields the malicious weight vector H′. L for:

[0052] H′ L =P′H L

[0053] Where P′ is the malicious weight vector, H L The text vector output by the Bert pre-trained model.

[0054] Furthermore, in step S5, the processing procedure of the multi-scale convolution module includes:

[0055] S51. Introduce channel attention ECA to obtain the channel-weighted text vector matrix H″. L ;

[0056] S52. The channel-weighted text vector matrix H″ is processed using convolution kernels of different row dimensions. L Convolution is performed, and the convolution kernel filters out parts of the words in the input sentence in a sliding order and generates features. This leads to the eigenvector A. c ;

[0057] S53. Preserve the contextual relationships in the feature vectors through multi-head self-attention pooling;

[0058] S54. Calculate the score z of the i-th category by passing the pooled feature vector M through a fully connected layer. i This constructs a score vector Z, and finally, the Softmax function is used to determine the malicious URL based on the score vector Z.

[0059] Furthermore, in step S51, the kernel size k of the one-dimensional convolution is first adaptively calculated based on the number of channels. The formula for calculating the kernel size k is:

[0060]

[0061] Where C represents the number of channels in the input feature, γ and β are hyperparameters, |·| odd This indicates that k is an odd number;

[0062] The input text vector matrix is ​​of size C×L×H; global features are extracted using global average pooling to obtain a 1×1×C vector; a one-dimensional convolution operation with a kernel size of k is performed to capture cross-channel interaction information and obtain the channel weight vector ω; the weights are normalized using the Sigmoid function; each generated channel weight is multiplied by the corresponding channel of the input word vector matrix to obtain the weighted text vector matrix H″. L .

[0063] Furthermore, in step S52, the final feature vector A c Represented as:

[0064]

[0065] feature The calculation formula is:

[0066]

[0067] in, Let i represent the i-th feature generated by a convolution kernel of size c, where i = 1, 2, ..., L-c+1; Rows i to i+c-1 of the matrix are sliding windows representing text vectors at different positions; W is the weight in the network, and b is the bias.

[0068] Furthermore, in step S53, the multi-head self-attention is calculated as follows:

[0069]

[0070] A = Concat(A) 2 A 3 A 4 )

[0071]

[0072] Among them, Q h K h V h This represents a vector of queries, keys, and values ​​used for the g-th attention header; To initialize the weight matrix; T denotes the transpose of the vector; A 2 A 3 A 4 d represents the feature vectors obtained with convolution kernel sizes of 2, 3, and 4, respectively, and A is the matrix obtained by concatenating the feature vectors; k It is the dimension of query, key, and value;

[0073] Multi-head pooling is calculated as follows:

[0074] P h =head h ×u g

[0075] a g =Softmax(P h )

[0076] head h =a g T ×head h

[0077] M=Concat(head′1, head′2,..., head′ h)

[0078] Among them, u g To initialize the parameter vector, set the head h with u g Perform matrix multiplication to obtain the joint vector P. h The probability distribution vector a between each row vector is calculated using the Softmax function. g ;a g T Let vector a g Transpose of a, then use a g T With matrix head h Multiplying them together yields the vector head′. h The information obtained from each space is concatenated to obtain the pooled vector M.

[0079] Furthermore, in step S54, the score z of the i-th category of feature vector M is... i for:

[0080] z i =W i M+b i

[0081] Where z i W represents the score for the i-th category. i and b i The weights and biases corresponding to category i;

[0082] The score vector Z is then represented as:

[0083] Z = [z1, z2, ..., z] i , ..., z α ]

[0084] Where α represents the number of categories, which is determined by the category distribution of the data annotations;

[0085] The score vector Z is passed through the Softmax function to calculate the probability of each category:

[0086]

[0087] Where q i This represents the probability of the i-th category;

[0088] We obtain a probability vector Q of length α:

[0089] Q = [q1, q2, ..., q] i , ..., q α ]

[0090] Calculate the class with the highest probability value in the probability vector Q:

[0091] i max =argmax(Q)

[0092] Where i max This represents the index corresponding to the maximum probability value; i max The detection results are obtained by matching the categories labeled in the S2 data.

[0093] The beneficial effects of this invention are as follows:

[0094] (1) In view of the fact that the page text content in the deep learning malicious URL detection method cannot make full use of the malicious word information in the text, the present invention designs a malicious URL detection method that integrates a malicious dictionary and a BERT pre-trained model. The text is extracted for context features, and a malicious dictionary is introduced for weighted processing, which makes the malicious features more prominent; the accuracy of malicious URL detection is improved, and URLs containing sensitive or malicious content can be identified more accurately.

[0095] (2) This invention performs word similarity matching calculations for words that do not appear in the malicious dictionary. Even if the word to be matched is not a complete match of the malicious word, the potential association can still be judged by the similarity, thereby enhancing the coverage and flexibility of malicious detection. When the similarity exceeds the set threshold, the word to be matched is judged as a malicious matching word, which improves the accuracy and robustness of detection and helps to identify malicious content with deformation, synonyms or similar expressions.

[0096] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0097] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0098] Figure 1 This is a flowchart of the malicious URL detection method that integrates a malicious dictionary and a BERT pre-trained model according to the present invention;

[0099] Figure 2 This is a schematic diagram of the text content detection network framework that integrates a malicious dictionary and a BERT pre-trained model used in this invention.

[0100] Figure 3 This is a schematic diagram of the Bert pre-trained model structure used in this invention;

[0101] Figure 4 This is a schematic diagram of the malicious value calculation module used in this invention;

[0102] Figure 5 This is a schematic diagram of the multi-scale convolution module used in this invention. Detailed Implementation

[0103] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0104] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0105] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0106] Please see Figures 1-5 This invention proposes a malicious URL detection method that integrates a malicious dictionary and a BERT pre-trained model, such as... Figure 1 As shown, the method specifically includes the following steps:

[0107] S1: Collect malicious words to construct a malicious dictionary W, and obtain URL page text information, preprocess it, and construct a URL page text content dataset;

[0108] S2: Construct a text content detection network that integrates a malicious dictionary and a BERT pre-trained model. The model structure consists of a BERT pre-trained model, a malicious value calculation module, a multi-scale convolution module, and a Softmax classifier.

[0109] S3: Use the BERT pre-trained model to extract features from the text and generate a text vector H containing contextual information. L ;

[0110] S4: The text vector H output by the above Bert pre-trained model... L Multiplying the weights obtained from the malicious value calculation module yields the malicious weighted vector H′. L ;

[0111] S5: Calculate the above malicious weighted vector H′ L After passing through the multi-scale convolution module, features at different scales are extracted and multi-scale feature vectors M are generated. The malicious probability is then calculated using Softmax.

[0112] S6: Train the text content detection network using the constructed dataset, save the best trained model in the parameter configuration file, and encapsulate it into an interface for external calls.

[0113] Example

[0114] This embodiment details the above method in two aspects, mainly S1 (data preprocessing) and S2-S6 (detection network construction and training), which includes the following steps:

[0115] (I) Data Preprocessing

[0116] In step S1 of this embodiment, the construction of the malicious dictionary first involves collecting malicious words, including sensitive words, pornographic words, violent and terrorist words, fraudulent words, reactionary words, etc.; then the malicious words are processed to form the malicious dictionary.

[0117] For the URL page text content dataset, malicious and normal URLs were obtained from public websites. Normal URLs were directly crawled from the APPPC site; malicious URLs were crawled from sites such as Cisco TalosIntelligenceGroup, Norton Safe Web, Clean-MX, and openphish. The crawled URLs were then used to simulate browser HTTP requests to obtain HTML files. A Python script was then used to parse the HTML, extracting... <title>、< / title> , Extract the text content under tags such as "etc".

[0118] Considering the presence of special characters, duplicate field content, and disorganized formatting in the crawled website data, preprocessing is necessary. Preprocessing operations include:

[0119] 1) Remove irrelevant words: Considering that malicious website pages may contain some irrelevant information, and most pages are similar to this kind of meaningless information, the text is simplified to remove this useless information;

[0120] 2) Stop word removal: Use a stop word list to remove punctuation, special characters, interjections, etc.

[0121] 3) Right truncation: Set the maximum sentence length to L. For sentences whose length does not meet L, pad with 0. For sentences whose length exceeds L, truncate to the right.

[0122] 4) Labeling: Malicious URL page text is classified as positive class 1, and normal URL page text is classified as negative class 0.

[0123] Next, a dataset of URL page text content is constructed, including a training set and a validation set. To ensure that the experimental results in this embodiment closely resemble the real network environment, the ratio of malicious to normal URL page text in the training and validation sets is maintained at 1:3. Specifically, the ratio of the training set to the validation set is 8:2. To better train and validate the network's detection performance, the training and validation sets are used to train and optimize the malicious URL detection network that integrates a malicious dictionary with a BERT pre-trained model, enabling the network to adapt to the distribution of malicious and normal URL page text.

[0124] (II) Detection Network Construction

[0125] In step S2 of this embodiment, a malicious URL detection network that integrates a malicious dictionary and a BERT pre-trained model is constructed. Figure 2 This invention presents the overall architecture of a malicious URL detection network that integrates a malicious dictionary and a BERT pre-trained model. The text content detection network includes an input layer, a BERT pre-trained model, a malicious value calculation module, a multi-scale convolution module, and a Softmax classifier. It uses the BERT pre-trained model to extract features from the text, generating a text vector containing contextual information, and multiplies it with the malicious value weights to obtain a malicious weighted vector. The weighted vector passes through the multi-scale convolution module to extract features at different scales, and then outputs the final predicted malicious classification result through Softmax.

[0126] In step S3 of this embodiment, as Figure 3 The diagram shown illustrates the structure of the BERT pre-trained model in the malicious URL detection network that integrates a malicious dictionary and a BERT pre-trained model, as used in this invention. The BERT pre-trained model extracts features from the input text content, specifically including:

[0127] S31. Segment the preprocessed text into words, and mark the beginning of each sentence with the symbol [cls] and the end with [sep], resulting in the text sequence T = {x}. cls x1, x2, ..., x i , ..., x L x sep }; where the length of the text sequence is L+2, x i This represents the i-th character in the text;

[0128] S32. The text sequence T is processed by word vector encoding, sentence encoding, and position encoding to obtain the word vector Ew. i ={e wcls e w1 e w2 , ..., e wi ,..,,e wL e wsep };Sentence vector E si ={e scls e s1 e s2 , ..., e si , ..., e sL e ssep };Position vector E pi ={e pcls e p1 e p2 , ..., e pi , ..., e pL e psep }; The final vector representation of the input to the BERT pre-trained model is: E bi =E wi +E si +E pi ;

[0129] S33, Input vector E bi The data is sequentially passed through 12 encoding layers. Each encoding layer undergoes a multi-head self-attention computation, layer normalization, residual connection, and a fully connected layer. After passing through 12 encoding layers, a text vector H containing contextual information is generated. L .

[0130] In step S4 of this embodiment, as Figure 4 The diagram shown illustrates the malicious value calculation module in the malicious URL detection network that integrates a malicious dictionary and a BERT pre-trained model, as used in this invention. The malicious value calculation module calculates the malicious weight vector for each word in the text sequence based on the input text sequence and the pre-built malicious dictionary. The specific process includes:

[0131] S41. The malicious value calculation module first performs Jieba segmentation on the preprocessed text data to obtain the set of words to be matched, V = {V1, V2, ..., V...}. i , ..., V N }, where V i This represents the i-th word; the set of word characters to be matched is V′={V′1,V′2,...,V′ i ,...,V′ N }, where V′ i This represents the number of characters in the i-th word. The malicious word set constructed based on S1 is W = {W1, W2, ..., W...}. j , ..., W M }, where W j This represents the j-th word;

[0132] S42. Map the words to be matched and malicious words into word vectors using the Word2Vec model. Calculate the similarity between the word to be matched and the malicious word, and establish a mapping relationship between the malicious word and the word to be matched; the formula for calculating the similarity between the malicious word and the word to be matched is:

[0133]

[0134] Among them, V i W represents the i-th word. j sim(V) represents the j-th word. i W j ) represents V i With W j Word similarity between them; find malicious entries in the dictionary that are similar to V through similarity calculation. i The word with the highest matching degree, W j ;

[0135] S43. Define the Word Malicious Score (WMS) as follows:

[0136]

[0137] Among them, WMS(V i ) represents V i The score in the malicious dictionary, if max_sim(V i When ) is greater than or equal to the threshold σ, then WMS(V) i ) is 2V′ i Conversely, it is V′. i ;where V′ i This represents the number of characters in the i-th word;

[0138] Based on the above calculations, the malicious value of each word is evenly distributed among each character in that word, and the malicious value of a character is represented as follows:

[0139]

[0140] Among them, P k This represents the malicious value of the k-th character in the sentence, which belongs to the word V. i The malicious value of each word in the sentence and the identifiers [cls] and [sep] is calculated; if V i max_sim(V i If V is less than the threshold σ, then the malicious value of each character corresponding to that word is 1; otherwise, if V is greater than the threshold σ, then the malicious value of each character is 1. i max_sim(V i If the value of a word is greater than or equal to the threshold σ, then the malicious value of each character corresponding to that word is 2; the malicious value of the identifiers [cls] and [sep] is 1 by default.

[0141] S44. Construct a malicious value vector P = {P} for each word whose malicious value was calculated above. cls P1, P2, ..., P k , ..., P L P sep };

[0142] The obtained malicious value vector P is normalized to obtain the malicious weight vector P′. For a sentence of length L, one term P′ in its weight vector P′ is... k for:

[0143]

[0144] Malicious weighted vector H′ L for:

[0145] H′ L =P′H L

[0146] Where P′ is the malicious weight vector, H L The text vector output by the Bert pre-trained model.

[0147] In step S5 of this embodiment, as Figure 5 The diagram shown illustrates the multi-scale convolutional module in the malicious URL detection network that integrates a malicious dictionary and a BERT pre-trained model, as used in this invention. The resulting text malicious weighted vector H′ is then processed. L As input, features at different scales are extracted, which specifically includes the following steps:

[0148] S51. First, ECA (channel attention) is introduced to obtain the channel-weighted text vector matrix H″. L This allows the model to learn the importance of each channel feature, thus enabling the network to focus on advantageous features.

[0149] The kernel size k of the one-dimensional convolution is adaptively calculated based on the number of channels. The formula for calculating the kernel size k is:

[0150]

[0151] Where C represents the number of channels of the input feature, γ and β are hyperparameters, and odd indicates that k takes an odd number, which is beneficial for central symmetry;

[0152] The input text vector matrix is ​​of size C×L×H; global features are extracted using global average pooling to obtain a 1×1×C vector; a one-dimensional convolution operation with a kernel size of k is performed to capture cross-channel interaction information and obtain the channel weight vector ω; the weights are normalized using the Sigmoid function; each generated channel weight is multiplied by the corresponding channel of the input word vector matrix to obtain the weighted text vector matrix H″. L ;

[0153] S52. Convolution is performed using convolution kernels with row dimensions of 2, 3, and 4. The convolution kernels filter parts of the input sentence and generate features in a sliding order.

[0154]

[0155] in, This represents the i-th feature generated by a convolution kernel of size c; i = 1, 2, ..., L-c+1; Rows i to i+c-1 of the matrix are the sliding window, representing the text vector at different positions; W is the weight in the network, and b is the bias.

[0156] Obtain the eigenvector A c :

[0157]

[0158] S53. Next, multi-head self-attention pooling is used so that the integrated vector can retain the context relationship.

[0159] Multi-head self-attention is calculated as follows:

[0160]

[0161] A = Concat(A) 2 A 3 A 4 )

[0162]

[0163] Among them, Q h K h V h This represents a vector of queries, keys, and values ​​used for the g-th attention header; To initialize the weight matrix; T denotes the transpose of the vector; A 2 A 3 A 4 d represents the feature vectors obtained with convolution kernel sizes of 2, 3, and 4, respectively, and A is the matrix obtained by concatenating the feature vectors; k It is the dimension of query, key, and value;

[0164] Multi-head pooling is calculated as follows:

[0165] P h =head h ×u g

[0166] a g =Softmax(P h )

[0167] head h =a g T ×head h

[0168] M=Concat(head′1, head′2,..., head′ h )

[0169] Among them, u g To initialize the parameter vector, set the head h with u g Perform matrix multiplication to obtain the joint vector P. h The probability distribution vector a between each row vector is calculated using the Softmax function. g ;a g T Let vector a g Transpose of a, then use a g T With matrix head h Multiplication, by adjusting the weights to increase the proportion of key information, achieves feature optimization and yields the vector head′. h The information obtained from each space is concatenated to obtain the pooled vector M.

[0170] S54. Finally, the pooled feature vector M is passed through a fully connected layer to calculate the score z of the i-th category. i :

[0171] z i =W i M+b i

[0172] Where z i W represents the score for the i-th category. i and b i The weights and biases corresponding to category i;

[0173] The score vector Z is obtained:

[0174] Z = [z1, z2, ..., z] i , ...z α ]

[0175] Where α represents the number of categories, which is determined by the category distribution of the data annotations;

[0176] The score vector Z is passed through the Softmax function to calculate the probability of each category:

[0177]

[0178] Where q i This represents the probability of the i-th category;

[0179] We obtain a probability vector Q of length α:

[0180] Q = [q1, q2, ..., q] i , ..., q α ]

[0181] Calculate the class with the highest probability value in the probability vector Q:

[0182] i max =argmax(Q)

[0183] Where i max This represents the index corresponding to the maximum probability value; i max The detection results are obtained by matching the categories labeled in the S2 data.

[0184] In step S6 of this embodiment, the malicious URL detection network that integrates a malicious dictionary and a BERT pre-trained model is trained using a URL page text content training set. Each training task obtains malicious URL and normal URL page text data in a ratio of approximately 1:3 from the training set. During training, malicious URLs and normal URLs are randomly sampled and input into the malicious URL detection network that integrates a malicious dictionary and a BERT pre-trained model to guide and optimize the malicious URL detection process until training is complete.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting malicious URLs by fusing a malicious dictionary and a Bert pre-training model, characterized in that: The method comprises the following steps: S1: Collecting malicious words to build a malicious dictionary and obtain URL page text information, and build a URL page text content dataset after preprocessing. S2: constructing a text content detection network fusing a malicious word dictionary and a Bert pre-training model, the text content detection network comprising an input layer, a Bert pre-training model, a malicious value calculation module, a multi-scale convolution module, and a Softmax classifier; S3: Use the Bert pre-training model to extract features from the text and generate text vectors containing context information ; S4: Obtain the malicious weight vector of the text sequence by the malicious value calculation module, and calculate the malicious weight vector by the malicious value calculation module S4: Obtain the malicious weight vector of the text sequence by the malicious value calculation module, and calculate the malicious weight vector by the malicious value calculation module In step S4, the malicious value calculation module calculates the malicious weight vector according to the following steps: S41, the malicious value calculation module first carries out the jieba word segmentation to the preprocessed text data, and obtains a to-be-matched word set and the to-be-matched word number set : wherein represents the word, represents the number of words; S42, map the word to be matched and the malicious word in the malicious word dictionary into a word vector through a Word2Vec model , , calculate the similarity between the word to be matched and the malicious word, and establish a mapping relationship between the malicious word and the word to be matched; the similarity calculation formula of the malicious word and the word to be matched is: wherein, represents the word, represents the word, represents the similarity between the words; the word in the malicious dictionary with the highest matching degree is found by similarity calculation ;​​ S43, define the malicious value of the word as: wherein, represents a score in a malicious dictionary, if is greater than or equal to a threshold value , then is ; otherwise, is ; wherein represents the number of characters of the th word Divide the malicious value of each word into each character in the word, and the malicious value of the character is represented as: in, Indicates the first word in the sentence The malicious value of the character; this character belongs to the word category. The malicious value of each word in the sentence and the identifiers [cls] and [sep] is calculated. The malicious value of the identifiers [cls] and [sep] is 1 by default. S44, construct a malicious value vector for the malicious value of each character calculated above: The resulting malicious value vector The resulting malicious weight vector For a sentence of length L, the weight vector One of is: Finally, the text vector output by the Bert pre-training model is multiplied by the malicious weight vector to obtain a malicious weight vector is:​​ wherein, is a malicious weight vector, is a text vector output by a Bert pre-training model; S5: malicious weight vector After the multi-scale convolution module, the features at different scales are extracted and the multi-scale feature vector is generated And the malicious probability is calculated by Softmax. S6: training the text content detection network using the constructed data set, retaining the parameter configuration file after training, and packaging into an interface for external calling.

2. The method of claim 1, wherein the method comprises: In step S1, for the malicious dictionary , the malicious words collected first, the collected malicious words are processed to form a malicious dictionary: wherein represents the word, total number of words in the malicious dictionary; For the URL page text content data set, malicious and normal URLs are obtained from public websites, and the obtained URL text content is preprocessed, wherein the preprocessing operations include removing irrelevant words, removing stop words, right truncation, and labeling processing, wherein the right truncation refers to padding with 0 for a statement with a length less than the maximum length L, and truncating the words after L for a statement with a length greater than the maximum length L; labeling refers to labeling the malicious URL page text as positive class 1 and the normal URL page text as negative class 0; The preprocessed text content is divided into a training set and a validation set according to a preset malicious and normal URL page text ratio and a preset training set and validation set data amount ratio.

3. The method of claim 2, wherein the method comprises: In step S2, a malicious URL text content detection network fusing a malicious word dictionary and a Bert pre-training model is established, wherein the text content detection network comprises an input layer, a Bert pre-training model, a malicious value calculation module, a multi-scale convolution module, and a Softmax classifier, the input layer inputs text content, the Bert pre-training model is used to extract features of the text, the malicious value calculation module is used to calculate a malicious weight vector, the multi-scale convolution module extracts features at different scales, and the Softmax outputs a final predicted malicious classification result.

4. The malicious URL detection method of claim 3, wherein: In step S3, the Bert pre-training model extracts features of the input text content according to the following steps: S31, segment the preprocessed text, and use the symbol [cls] to identify the beginning of the sentence and [sep] to identify the end of the sentence, to obtain a text sequence : wherein the text sequence has a length of , represents the th character in the text; S32, text sequence After word vector encoding, sentence encoding, and position encoding, the word vector is obtained. Sentence vector: Position vector: The vector representation of the final Bert pre-training model input is: ; S33, input vector The input vector is sequentially transmitted into multiple encoding layers, each of which performs multi-head self-attention calculation, layer normalization, residual connection and full connection layer processing, and finally generates a text vector containing context information .

5. The method of claim 4, wherein the method further comprises: In step S5, the processing process of the multi-scale convolution module comprises: S51, introducing the channel attention ECA to obtain a channel-weighted text vector matrix ; S52, the text vector matrix after the channel is weighted by using the convolution kernel with different row dimension sizes Convolution is performed, and the convolution kernel filters the part of words in the input sentence in a sliding order and generates features , and then the feature vector is obtained ; S53, the context relationship in the feature vector is reserved through multi-head self-attention pooling; S54. The feature vectors obtained after pooling... Calculate the first through the fully connected layer Scores for each category This constructs a score vector Z, and finally, the Softmax function is used to determine the malicious URL based on the score vector Z.

6. The method of claim 5, wherein the method further comprises: In step S51, the kernel size of one-dimensional convolution is first adaptively calculated according to the number of channels The calculation formula of the kernel size is as follows: wherein represents the number of channels of input features, and is a hyperparameter, represents take an odd number; The input text vector matrix is of size ; global features are extracted by global pooling to obtain a vector ; one-dimensional convolution operation is performed with a convolution kernel size of to capture cross-channel interaction information and obtain a channel weight vector ; the weights are normalized by using a Sigmoid function; and each generated channel weight is multiplied by the corresponding channel of the input word vector matrix to obtain a weighted text vector matrix .

7. The method of claim 5, wherein the method further comprises: In step S52, the final feature vector is expressed as: Features The calculation formula is: wherein, represents the convolution kernel size The generated first feature, ; is the matrix row to row, is a sliding window, representing the text vector at different positions; is the weight in the network, is the bias.

8. The method of claim 5, wherein the method further comprises: In step S53, the multi-head self-attention calculation is: wherein, represents a vector of queries, keys and values for the first attention head; , , is an initialized weight matrix; represents the transpose of a vector; are feature vectors obtained with a convolution kernel size of 2, 3, 4, respectively, is a matrix after concatenation of the feature vectors; is the dimension of queries, keys and values. Multi-head pooling calculation is: wherein, For initializing the parameter vector, the matrix multiplication operation is performed between and to obtain the joint vector ; the probability distribution vector between each row vector is calculated by using the Softmax function ; is the transpose of the vector , and then the vector is multiplied by the matrix to obtain the vector ; the information obtained by each space is spliced to obtain the pooled vector .

9. The method of claim 5, wherein the method further comprises: In step S54, the feature vector of the first class of the score is: wherein represents the score of the th category, and are the weight and bias corresponding to the category . the score vector is represented as: wherein represents the number of categories, which is determined by the category distribution of data labeling; The score vector is computed as The probabilities for each class are computed by the Softmax function: wherein represents the probability of the th class. a probability vector of length :​ Computing probability vectors Category with the highest probability value wherein represents the index corresponding to the maximum probability value; and is matched with the category of S2 data labeling, and a detection result is obtained.

Images