Malicious URL detection method fusing malicious dictionary and Bert pre-training model

Abstract

The invention relates to a malicious URL detection method fusing a malicious dictionary and a Bert pre-training model, and belongs to the technical field of information security. The method comprises the steps that malicious vocabularies are collected to construct a malicious dictionary, URL page text information is obtained, and a URL page text content data set is constructed after the URL page text information is preprocessed; constructing a text content detection network fusing the malicious dictionary and a Bert pre-training model, performing feature extraction on the text by using the Bert pre-training model, and generating a text vector containing context information; a malicious weight vector of the text sequence is obtained through a malicious value calculation module, and then a malicious weight vector of a text vector is obtained; the malicious weighted vector passes through a multi-scale convolution module, features under different scales are extracted, a multi-scale feature vector is generated, the malicious probability is calculated, and therefore the malicious URL is detected. According to the method, the accuracy and the stability of malicious URL detection are remarkably enhanced, and stronger protection is provided for information security.

Description

Technical Field

[0001] The invention belongs to the technical field of information security and relates to a malicious URL detection method integrating a malicious dictionary and a Bert pre-training model. Background Art

[0002] Malicious URLs are often used to spread viruses, carry out phishing attacks, steal information, etc., causing huge losses to users and enterprises. Although there are many malicious URL detection technologies, most methods still have certain limitations and fail to fully capture the complexity of URL text and the relationship between malicious words.

[0003] Most existing detection technologies rely on static features or simple rule matching, such as blacklists and keyword matching. However, these methods are often unable to cope with new malicious URL variants and disguises. At the same time, traditional machine learning methods often rely on manual selection in feature extraction, resulting in insufficient generalization of the model and difficulty in adapting to rapidly changing malicious attack patterns.

[0004] In recent years, the application of deep learning technology has brought new opportunities for malicious URL detection. In particular, the remarkable achievements of the BERT pre-training model in the field of natural language processing have demonstrated its powerful ability in understanding contextual information.

[0005] However, detection methods that rely solely on the Bert model or malicious dictionaries still have certain limitations and cannot fully consider the multi-dimensional characteristics of the text. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a malicious URL detection method that integrates a malicious dictionary and a Bert pre-training model. The method constructs a malicious dictionary by collecting sensitive words, pornographic words, violent and terrorist words, fraud-related words, reactionary words, etc., obtains URL page text data and pre-processes it, and constructs a text content data set. On this basis, a text content detection network that integrates a malicious dictionary and a Bert pre-training model is designed. The Bert pre-training model is used to extract features from the text, generate a text vector containing context information, and multiply it with the malicious value weight to obtain a malicious weighted vector. The weighted vector passes through a multi-scale convolution module to extract features at different scales, and outputs the final predicted malicious classification result through Softmax. Finally, accurate classification of malicious URLs is achieved.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A malicious URL detection method integrating a malicious dictionary and a BERT pre-training model, the method comprising the following steps:

[0009] S1: Collect malicious words to build a malicious dictionary W, obtain URL page text information, and preprocess it to build a URL page text content dataset;

[0010] S2: Build a text content detection network that integrates the malicious dictionary and the Bert pre-trained model. The text content detection network includes an input layer, a Bert pre-trained model, a malicious value calculation module, a multi-scale convolution module, and a Softmax classifier.

[0011] S3: Use the Bert pre-trained model to extract features from the text and generate a text vector H containing context information L ;

[0012] S4: The malicious weight vector of the text sequence is obtained through the malicious value calculation module and compared with the text vector H L Calculate and obtain the malicious weighted vector H′ L ;

[0013] S5: Add malicious weight vector H′ L After passing through the multi-scale convolution module, the features at different scales are extracted and a multi-scale feature vector M is generated, and the malicious probability is calculated through Softmax;

[0014] S6: Use the constructed dataset to train the text content detection network. After the training is completed, keep the parameter configuration file and encapsulate it into an interface for external calls.

[0015] Further, in step S1, for the malicious dictionary W, malicious words are first collected, and the collected malicious words are processed to form a malicious dictionary:

[0016] W={W1,W2,...,W j , ..., W M}

[0017] Where W j represents the jth word, and M is the total number of words in the malicious dictionary;

[0018] For the URL page text content dataset, malicious and normal URLs are obtained from public websites, and then the obtained URL text content is preprocessed. The preprocessing operations include: removing irrelevant words, removing stop words, right truncation, and annotation processing. Right truncation means padding sentences with a length less than the maximum length L with 0, and truncating the words after L for sentences with a length exceeding the maximum length L; annotation means that malicious URL page text is classified as positive class 1, and normal URL page text is classified as negative class 0;

[0019] The preprocessed text content is divided into a training set and a validation set according to a preset ratio of malicious and normal URL page texts and a preset ratio of training set and validation set data volumes.

[0020] Further, in step S2, a malicious URL text content detection network integrating a malicious dictionary and a Bert pre-trained model is established, wherein the text content detection network includes an input layer, a Bert pre-trained model, a malicious value calculation module, a multi-scale convolution module, and a Softmax classifier. The input layer inputs text content, and the Bert pre-trained model is used to extract features from the text. The malicious value calculation module is used to calculate a malicious weight vector. The multi-scale convolution module extracts features at different scales, and outputs the final predicted malicious classification result through Softmax.

[0021] Further, in step S3, the Bert pre-training model extracts features from the input text content according to the following steps:

[0022] S31. Segment the preprocessed text, and use the symbol [cls] to mark the beginning of the sentence and [sep] to mark the end of the sentence, to obtain a text sequence T:

[0023] T={x cls , x1, x2, ..., x i , ..., x L , x sep}

[0024] The length of the text sequence T is L+2, x i Represents the i-th word in the text;

[0025] S32, the text sequence T is encoded by word vector, sentence encoding, and position encoding to obtain word vectors:

[0026] E wi ={e wcls , e w1 , e w2 , ..., e wi , ..., e wL , e wsep}

[0027] Sentence vector:

[0028] E si ={e scls , e s1 , e s2 , ..., e si , ..., e sL , e ssep}

[0029] Position vector:

[0030] E pi ={e pcls , e p1 , e p2 , ..., e pi , ..., e pL , e psep}

[0031] The final vector representation of the Bert pre-training model input is: E bi =E wi +E si +E pi ;

[0032] S33, input vector E bi It is passed to multiple encoding layers in sequence, and each encoding layer performs a multi-head self-attention calculation, layer normalization, residual connection, and full connection layer processing, and finally generates a text vector H containing context information. L .

[0033] Further, in step S4, the malicious value calculation module calculates the malicious weight vector according to the following steps:

[0034] S41, the malicious value calculation module first performs word segmentation on the preprocessed text data to obtain a set of to-be-matched words V and a set of to-be-matched word counts V′:

[0035] V={V1,V2,...,V i , ..., V N}

[0036] V′={V′1, V′2,...,V′ i , ..., V′ N}

[0037] Where V i represents the i-th word, V′ i Indicates the number of words in the i-th word;

[0038] S42: Map the words to be matched and the malicious words in the malicious dictionary into word vectors through the Word2Vec model Calculate the similarity between the words to be matched and the malicious words, and establish a mapping relationship between the malicious words and the words to be matched; the similarity calculation formula between the malicious words and the words to be matched is:

[0039]

[0040] Among them, V i represents the i-th word, W j represents the jth word, sim(V i , Wj ) indicates V i With W j The similarity between words; find the words in the malicious dictionary that match V through similarity calculation i The most matching word W j ;

[0041] S43. Define the malicious value of the word:

[0042]

[0043] Among them, WMS (V i ) represents V i The score in the malicious dictionary, if max_sim(V i ) is greater than or equal to the threshold σ, then WMS(V i ) is 2V′ i Otherwise, it is V′ i ; where V′ i Indicates the number of words in the i-th word;

[0044] The malicious value of each word is evenly divided into each character in the word. The malicious value of the character is expressed as:

[0045]

[0046] Among them, P k Indicates the malicious value of the kth word in the sentence, which belongs to the word V i ; The malicious value of each word and identifiers [cls] and [sep] in the sentence is calculated, and the malicious value of identifiers [cls] and [sep] is 1 by default;

[0047] S44, construct a malicious value vector using the malicious value of each word calculated above:

[0048] P={P cls , P1, P2, ..., P k , ..., P L , P sep}

[0049] The malicious value vector P is normalized to obtain the malicious weight vector P′. For a sentence of length L, an item P′ in the weight vector P′ k for:

[0050]

[0051] Finally, the text vector H output by the Bert pre-training model L Multiply it with the malicious weight vector P′ to get the malicious weight vector H′ L for:

[0052] H′ L =P′H L

[0053] Among them, P′ is the malicious weight vector, H L The text vector output by the Bert pre-trained model.

[0054] Further, in step S5, the processing process of the multi-scale convolution module includes:

[0055] S51, introduce channel attention ECA to obtain the channel-weighted text vector matrix H″ L ;

[0056] S52, using convolution kernels of different row dimension sizes to weight the text vector matrix H″ after channel weighting L Convolution is performed, and the convolution kernel filters some words in the input sentence in a sliding order and generates features Then we get the feature vector A c ;

[0057] S53, preserve the contextual relationship in the feature vector through multi-head self-attention pooling;

[0058] S54, the feature vector M obtained after pooling is calculated through the fully connected layer to calculate the score z of the i-th category i , thereby constructing a score vector Z, and finally judging malicious URLs based on the score vector Z through the Softmax function.

[0059] Further, in step S51, the kernel size k of the one-dimensional convolution is firstly adaptively calculated according to the number of channels, and the calculation formula of the kernel size k is:

[0060]

[0061] Where C represents the number of channels of the input features, γ and β are hyperparameters, and |·| odd Indicates that k is an odd number;

[0062] The size of the input text vector matrix is ​​C×L×H; global features are extracted through global average pooling to obtain a 1×1×C vector; a one-dimensional convolution operation with a convolution kernel size of k is performed to capture cross-channel interaction information and obtain the channel weight vector ω; the weight is normalized using the Sigmoid function; each channel weight generated is multiplied by the corresponding channel of the input word vector matrix to obtain the weighted text vector matrix H″ L .

[0063] Further, in step S52, the final feature vector A c It is expressed as:

[0064]

[0065] feature The calculation formula is:

[0066]

[0067] in, represents the i-th feature generated by the convolution kernel size c, i = 1, 2, ..., L-c+1; The matrix is ​​from row i to row i+c-1, which is a sliding window representing text vectors at different positions; W is the weight in the network and b is the bias.

[0068] Further, in step S53, the multi-head self-attention is calculated as:

[0069]

[0070] A=Concat(A 2 , A 3 , A 4 )

[0071]

[0072] Among them, Q h , K h 、V h A vector representing the query, key, and value for the g-th attention head; is the initialization weight matrix; T represents the transpose of the vector; A 2 , A 3 , A 4 are the eigenvectors obtained when the convolution kernel size is 2, 3, and 4 respectively, and A is the matrix after the eigenvectors are concatenated; d k are the dimensions of query, key, and value;

[0073] The long pooling calculation is:

[0074] P h =head h ×u g

[0075] a g =Softmax(P h )

[0076] head′ h =a g T ×head h

[0077] M=Concat(head′1, head′2,..., head′ h)

[0078] Among them, u g To initialize the parameter vector, head h with u g Perform matrix multiplication to obtain the joint vector P h ; Use the Softmax function to calculate the probability distribution vector a between each row vector g ; a g T is the vector a g , and then a g T With the matrix head h Multiply to get vector head′ h ; The information obtained from each space is spliced ​​to obtain the pooled vector M.

[0079] Further, in step S54, the score z of the i-th category of the feature vector M is i for:

[0080] z i =W i M+b i

[0081] where z i represents the score of the i-th category, W i and b i is the weight and bias corresponding to category i;

[0082] Then the score vector Z is expressed as:

[0083] Z=[z1,z2,...,z i , ..., z α ]

[0084] Where α represents the number of categories, which is determined by the category distribution of the data annotation;

[0085] Pass the score vector Z through the Softmax function to calculate the probability of each category:

[0086]

[0087] where q i represents the probability of the i-th category;

[0088] Get the probability vector Q of length α:

[0089] Q=[q1,q2,...,q i , ..., q α ]

[0090] Calculate the category with the largest probability value in the probability vector Q:

[0091] i max = argmax(Q)

[0092] where i max Indicates the index corresponding to the maximum probability value; i max Match the categories marked with the S2 data to obtain the detection results.

[0093] The beneficial effects of the present invention are:

[0094] (1) Aiming at the problem that the page text content in the deep learning malicious URL detection method cannot fully utilize the malicious word information in the text, the present invention designs a malicious URL detection method that integrates a malicious dictionary and a BERT pre-training model, extracts contextual features of the text, and introduces a malicious dictionary for weighted processing, so that the malicious features are more prominent; the accuracy of malicious URL detection is improved, and URLs containing sensitive or malicious content can be more accurately identified.

[0095] (2) The present invention performs word similarity matching calculations on words that do not appear in the malicious dictionary. Even if the word to be matched is not a complete match of the malicious word, the potential association can still be eliminated through similarity judgment, thereby enhancing the coverage and flexibility of malicious detection; when the similarity exceeds the set threshold, the word to be matched is determined to be a malicious matching word, which improves the accuracy and robustness of detection and helps to identify malicious content with deformation, synonyms, or similar expressions.

[0096] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0098] Figure 1 It is a flow chart of a malicious URL detection method integrating a malicious dictionary and a Bert pre-training model of the present invention;

[0099] Figure 2 It is a schematic diagram of a text content detection network framework that integrates a malicious dictionary and a Bert pre-training model adopted by the present invention;

[0100] Figure 3 It is a schematic diagram of the structure of the Bert pre-training model used in the present invention;

[0101] Figure 4 Schematic diagram of the malicious value calculation module used in the present invention;

[0102] Figure 5 Schematic diagram of the multi-scale convolution module used in the present invention. DETAILED DESCRIPTION

[0103] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0104] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0105] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as a limitation on the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0106] See also Figure 1 to Figure 5 ,The present invention proposes a malicious URL detection method that integrates a malicious dictionary and a Bert pre-training model, such as Figure 1 As shown, the method specifically comprises the following steps:

[0107] S1: Collect malicious words to build a malicious dictionary W, obtain URL page text information, and preprocess it to build a URL page text content dataset;

[0108] S2: Build a text content detection network that integrates the malicious dictionary and the Bert pre-trained model. The model structure consists of the Bert pre-trained model, the malicious value calculation module, the multi-scale convolution module, and the Softmax classifier;

[0109] S3: Use the Bert pre-trained model to extract features from the text and generate a text vector H containing context information L ;

[0110] S4: The text vector H output by the above Bert pre-training model L Multiply it by the weight obtained by the malicious value calculation module to obtain the malicious weighted vector H′ L ;

[0111] S5: Add the malicious weighted vector H′ L After passing through the multi-scale convolution module, the features at different scales are extracted and a multi-scale feature vector M is generated, and the malicious probability is calculated through Softmax;

[0112] S6: Use the constructed dataset to train the text content detection network, retain the parameter configuration file of the trained best model, and encapsulate it into an interface for external calls.

[0113] Example

[0114] This embodiment describes the above method in detail in two aspects, mainly divided into S1 data preprocessing and S2-S6 detection network construction and training, wherein the following steps are included:

[0115] 1. Data preprocessing

[0116] In step S1 of this embodiment, for the construction of a malicious dictionary, malicious words are first collected, wherein the malicious words include sensitive words, pornographic words, violent and terrorist words, fraud-related words, reactionary words, etc.; then the malicious words are processed to form a malicious dictionary.

[0117] For the URL page text content dataset, malicious and normal URLs are obtained from public websites. Normal URLs are directly crawled from the site based on APPPC; malicious URLs are crawled from the site based on Cisco Talos Intelligence Group, Norton Safe Web, Clean-MX, and openphish. The crawled URLs simulate the browser to send HTTP requests to obtain HTML files, and then parse the HTML through Python scripts. <title>、< / title> , Extract the text content under the tags.

[0118] Considering that the crawled website data has special characters, repeated field content, messy format and other problems, it is necessary to preprocess the crawled website data. The preprocessing operations include:

[0119] 1) Remove irrelevant words: Considering that malicious website pages may contain some irrelevant information, most pages are similar to this meaningless information, so the text is simplified to remove this useless information;

[0120] 2) Remove stop words: Use a stop word list to remove punctuation, special symbols, modal particles, etc.;

[0121] 3) Right truncation: Set the maximum length of a sentence to L. If the length of a sentence does not meet L, it will be padded with 0. If the length of a sentence exceeds L, it will be right truncated.

[0122] 4) Labeling: Malicious URL page text is positive class 1, and normal URL page text is negative class 0.

[0123] Then, a URL page text content data set is constructed, which includes a training set and a validation set. In order to make the experimental effect of this embodiment close to the real network environment, the ratio of malicious and normal URL page texts in the training set and the validation set is guaranteed to be 1:3. The ratio of the training set and the validation set is 8:2 respectively. In order to better train and verify the detection effect of the network, the training set and the validation set are used to train and optimize the malicious URL detection network that integrates the malicious dictionary and the Bert pre-training model, so that the network can adapt to the distribution of malicious URLs and normal URL page texts.

[0124] (II) Construction of detection network

[0125] In step S2 of this embodiment, a malicious URL detection network integrating a malicious dictionary and a Bert pre-training model is constructed. Figure 2 It is the overall architecture of the malicious URL detection network that integrates the malicious dictionary and the Bert pre-training model adopted by the present invention. The text content detection network includes an input layer, a Bert pre-training model, a malicious value calculation module, a multi-scale convolution module and a Softmax classifier. It uses the Bert pre-training model to extract features from the text, generates a text vector containing context information, and multiplies it with the malicious value weight to obtain a malicious weighted vector; the weighted vector passes through the multi-scale convolution module to extract features at different scales, and outputs the final predicted malicious classification result through Softmax.

[0126] In step S3 of this embodiment, Figure 3 The Bert pre-training model structure diagram of the malicious URL detection network in the present invention that integrates the malicious dictionary and the Bert pre-training model is shown. The Bert pre-training model extracts features from the input text content, specifically including:

[0127] S31, segment the preprocessed text, and use the symbol [cls] to mark the beginning of the sentence and [sep] to mark the end of the sentence, to obtain a text sequence T = {x cls , x1, x2, ..., x i , ..., x L , x sep }; where the length of the text sequence is L+2, x i Represents the i-th word in the text;

[0128] S32, the text sequence T is encoded by word vector, sentence encoding, and position encoding to obtain word vector Ew i ={e wcls , e w1 , e w2 , ..., e wi ,..,,e wL , e wsep }; Sentence vector E si ={e scls , e s1 , e s2 , ..., e si , ..., e sL , e ssep }; Position vector E pi ={e pcls , e p1 , e p2 , ..., e pi , ..., e pL , e psep }; The final vector representation of the Bert pre-training model input is: E bi =E wi +E si +E pi ;

[0129] S33, input vector E bi The text is passed to 12 encoding layers in sequence. Each encoding layer undergoes a multi-head self-attention calculation, layer normalization, residual connection, and full connection layer. After 12 encoding layers, a text vector H containing context information is generated. L .

[0130] In step S4 of this embodiment, Figure 4 The schematic diagram of the malicious value calculation module in the malicious URL detection network that integrates the malicious dictionary and the Bert pre-training model used in the present invention is shown. The malicious value calculation module calculates the malicious weight vector of each word in the text sequence according to the input text sequence and the pre-built malicious dictionary. The specific process includes:

[0131] S41, the malicious value calculation module first performs word segmentation on the preprocessed text data to obtain a set of words to be matched V = {V1, V2, .., V i , ..., V N }, where V i represents the i-th word; the word count set V′={V′1, V′2, ..., V′ i , ..., V′ N }, where V′ i represents the number of characters in the i-th word. The malicious word set constructed based on S1 is W = {W1, W2, ..., W j , ..., W M }, where W j represents the jth word;

[0132] S42. Map the words to be matched and malicious words into word vectors through the Word2Vec model Calculate the similarity between the words to be matched and the malicious words, and establish a mapping relationship between the malicious words and the words to be matched; the similarity calculation formula between the malicious words and the words to be matched is:

[0133]

[0134] Among them, V i represents the i-th word, W j represents the jth word, sim(V i , W j ) indicates V i With W j The similarity between words; find the words in the malicious dictionary that match V through similarity calculation i The most matching word W j ;

[0135] S43. Define the word malicious score (WMS) as:

[0136]

[0137] Among them, WMS (V i ) represents V i The score in the malicious dictionary, if max_sim(V i ) is greater than or equal to the threshold σ, then WMS(V i ) is 2V′ i Otherwise, it is V′ i ; where V′ i Indicates the number of words in the i-th word;

[0138] Based on the above calculation results, the malicious value of each word is evenly distributed to each character in the word. The malicious value of the character is expressed as:

[0139]

[0140] Among them, P k Indicates the malicious value of the kth word in the sentence, which belongs to the word V i ; The maliciousness of each word and identifier [cls], [sep] in the sentence is calculated; if V i max_sim(V i ) is less than the threshold σ, then the malicious value of each character corresponding to the word is 1; conversely, if V i max_sim(V i ) is greater than or equal to the threshold σ, then the malicious value of each character corresponding to the word is 2; the malicious value of the identifiers [cls] and [sep] is 1 by default;

[0141] S44, construct a malicious value vector P = {P cls , P1, P2, ..., P k , ..., P L , P sep };

[0142] The malicious value vector P is normalized to obtain the malicious weight vector P′. For a sentence of length L, an item P′ in its weight vector P′ k for:

[0143]

[0144] Malicious weighted vector H′ L for:

[0145] H′ L =P′H L

[0146] Among them, P′ is the malicious weight vector, H L The text vector output by the Bert pre-trained model.

[0147] In step S5 of this embodiment, Figure 5 The schematic diagram of the multi-scale convolution module in the malicious URL detection network of the present invention that integrates the malicious dictionary and the Bert pre-training model is shown, and the obtained text malicious weighted vector H′ L As input, features at different scales are extracted, which specifically includes the following steps:

[0148] S51. First, introduce ECA (channel attention) to obtain the channel-weighted text vector matrix H″ L , so that the model can learn the importance of each channel feature, so that the network can pay more attention to favorable features;

[0149] The kernel size k of the one-dimensional convolution is adaptively calculated according to the number of channels. The calculation formula of the kernel size k is:

[0150]

[0151] Where C represents the number of channels of the input feature, γ and β are hyperparameters, and odd means that k is an odd number, which is conducive to central symmetry;

[0152] The size of the input text vector matrix is ​​C×L×H; global features are extracted through global average pooling to obtain a 1×1×C vector; a one-dimensional convolution operation with a convolution kernel size of k is performed to capture cross-channel interaction information and obtain the channel weight vector ω; the weight is normalized using the Sigmoid function; each channel weight generated is multiplied by the corresponding channel of the input word vector matrix to obtain the weighted text vector matrix H″ L ;

[0153] S52, convolution kernels with row dimensions of 2, 3, and 4 are used for convolution. The convolution kernels filter some words in the input sentence in a sliding order and generate features.

[0154]

[0155] in, represents the i-th feature generated by the convolution kernel size c; i = 1, 2, ..., L-c+1; The matrix row i to row i+c-1 is a sliding window, representing the text vectors at different positions; W is the weight in the network, and b is the bias;

[0156] Get the feature vector A c :

[0157]

[0158] S53. Then use multi-head self-attention pooling so that the integrated vector can retain the contextual relationship.

[0159] The multi-head self-attention calculation is:

[0160]

[0161] A=Concat(A 2 , A 3 , A 4 )

[0162]

[0163] Among them, Q h , K h 、V h A vector representing the query, key, and value for the g-th attention head; is the initialization weight matrix; T represents the transpose of the vector; A 2 , A 3 , A 4 are the eigenvectors obtained when the convolution kernel size is 2, 3, and 4 respectively, and A is the matrix after the eigenvectors are concatenated; d k are the dimensions of query, key, and value;

[0164] The long pooling calculation is:

[0165] P h =head h ×u g

[0166] a g =Softmax(P h )

[0167] head′ h =a g T ×head h

[0168] M=Concat(head′1, head′2,..., head′ h )

[0169] Among them, u g To initialize the parameter vector, head h with u g Perform matrix multiplication to obtain the joint vector P h ; Use the Softmax function to calculate the probability distribution vector a between each row vector g ; a g T is the vector a g , and then a g T With the matrix head h Multiply them together to increase the weight of key information by adjusting the weights, and achieve feature optimization to obtain the vector head′ h ; The information obtained from each space is spliced ​​to obtain the pooled vector M.

[0170] S54. Finally, the feature vector M obtained after pooling is passed through the fully connected layer to calculate the score z of the i-th category i :

[0171] z i =W i M+b i

[0172] where z i represents the score of the i-th category, W i and b i is the weight and bias corresponding to category i;

[0173] Get the score vector Z:

[0174] Z=[z1,z2,...,z i ,...z α ]

[0175] Where α represents the number of categories, which is determined by the category distribution of the data annotation;

[0176] Pass the score vector Z through the Softmax function to calculate the probability of each category:

[0177]

[0178] where q i represents the probability of the i-th category;

[0179] Get the probability vector Q of length α:

[0180] Q=[q1,q2,...,q i , ..., q α ]

[0181] Calculate the category with the largest probability value in the probability vector Q:

[0182] i max =argmax(Q)

[0183] where i max Represents the index corresponding to the maximum probability value; i max Match the categories marked with the S2 data to obtain the detection results.

[0184] In step S6 of the present embodiment, the malicious URL detection network integrating the malicious dictionary and the Bert pre-training model is trained using the URL page text content training set. Each training task obtains malicious URL and normal URL page text data in a ratio of approximately 1:3 from the training set. During training, malicious URLs and normal URLs are randomly batch sampled and input into the malicious URL detection network integrating the malicious dictionary and the Bert pre-training model to guide and optimize the malicious URL detection process until the training is completed.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A malicious URL detection method integrating a malicious dictionary and a BERT pre-trained model, characterized in that: The method comprises the following steps: S1: Collect malicious words to build a malicious dictionary W, obtain URL page text information, and preprocess it to build a URL page text content dataset; S2: Build a text content detection network that integrates the malicious dictionary and the Bert pre-trained model. The text content detection network includes an input layer, a Bert pre-trained model, a malicious value calculation module, a multi-scale convolution module, and a Softmax classifier. S3: Use the Bert pre-trained model to extract features from the text and generate a text vector H containing context information L ; S4: The malicious weight vector of the text sequence is obtained through the malicious value calculation module and compared with the text vector H L Calculate and obtain the malicious weighted vector H′ L ; S5: Add malicious weight vector H′ L After passing through the multi-scale convolution module, the features at different scales are extracted and a multi-scale feature vector M is generated, and the malicious probability is calculated through Softmax; S6: Use the constructed dataset to train the text content detection network. After the training is completed, keep the parameter configuration file and encapsulate it into an interface for external calls.

2. According to claim 1, a malicious URL detection method integrating a malicious dictionary and a BERT pre-training model is characterized in that: In step S1, for the malicious dictionary W, malicious words are first collected, and the collected malicious words are processed to form a malicious dictionary: W={W1,W2,...,W j ,...,IN M } Where W j represents the jth word, and M is the total number of words in the malicious dictionary; For the URL page text content dataset, malicious and normal URLs are obtained from public websites, and then the obtained URL text content is preprocessed. The preprocessing operations include: removing irrelevant words, removing stop words, right truncation, and annotation processing. Right truncation means padding sentences with a length less than the maximum length L with 0, and truncating the words after L for sentences with a length exceeding the maximum length L; annotation means that malicious URL page text is classified as positive class 1, and normal URL page text is classified as negative class 0; The preprocessed text content is divided into a training set and a validation set according to a preset ratio of malicious and normal URL page texts and a preset ratio of training set and validation set data volumes.

3. According to claim 2, a malicious URL detection method integrating a malicious dictionary and a BERT pre-training model is characterized in that: In step S2, a malicious URL text content detection network integrating a malicious dictionary and a Bert pre-trained model is established, wherein the text content detection network includes an input layer, a Bert pre-trained model, a malicious value calculation module, a multi-scale convolution module, and a Softmax classifier. The input layer inputs text content, and the Bert pre-trained model is used to extract features from the text. The malicious value calculation module is used to calculate the malicious weight vector, and the multi-scale convolution module extracts features at different scales, and outputs the final predicted malicious classification result through Softmax.

4. According to claim 3, a malicious URL detection method integrating a malicious dictionary and a BERT pre-training model is characterized in that: In step S3, the Bert pre-trained model extracts features from the input text content according to the following steps: S31. Segment the preprocessed text, and use the symbol [cls] to mark the beginning of the sentence and [sep] to mark the end of the sentence, to obtain a text sequence T: T={x cls ,x1,x2,…,x i ,…,x L ,x sep } The length of the text sequence T is L+2, x i Represents the i-th word in the text; S32, the text sequence T is encoded by word vector, sentence encoding, and position encoding to obtain word vectors: AND wi ={and wcls ,And w1 ,And w2 ,...,And wi ,...,And wL ,And wsep } Sentence vector: AND si ={and scls ,And s1 ,And s2 ,...,And si ,...,And sL ,And ssep } Position vector: AND pi ={and pcls ,And p1 ,And p2 ,...,And pi ,...,And pL ,And psep } The final vector representation of the Bert pre-training model input is: E bi =E wi +E si +E pi ; S33, input vector E bi It is passed to multiple encoding layers in sequence, and each encoding layer performs a multi-head self-attention calculation, layer normalization, residual connection, and full connection layer processing, and finally generates a text vector H containing context information. L .

5. According to claim 4, a malicious URL detection method integrating a malicious dictionary and a BERT pre-training model is characterized in that: In step S4, the malicious value calculation module calculates the malicious weight vector according to the following steps: S41, the malicious value calculation module first performs word segmentation on the preprocessed text data to obtain a set of to-be-matched words V and a set of to-be-matched word counts V′: V={V1,V2,...,V i ,...,V N } V′={V′1,V′2,...,V′ i ,...,V′ N } Where V i represents the i-th word, V′ i Indicates the number of words in the i-th word; S42: Map the words to be matched and the malicious words in the malicious dictionary into word vectors through the Word2Vec model Calculate the similarity between the words to be matched and the malicious words, and establish a mapping relationship between the malicious words and the words to be matched; the similarity calculation formula between the malicious words and the words to be matched is: Among them, V i represents the i-th word, W j represents the jth word, sim(V i ,W j ) indicates V i With W j The similarity between words; find the words in the malicious dictionary that match V through similarity calculation i The most matching word W j ; S43. Define the malicious value of the word: Among them, WMS (V i ) represents V i The score in the malicious dictionary, if max_sim(V i ) is greater than or equal to the threshold σ, then WMS(V i ) is 2V′ i Otherwise, it is V′ i ; where V′ i Indicates the number of words in the i-th word; The malicious value of each word is evenly divided into each character in the word. The malicious value of the character is expressed as: Among them, P k Indicates the malicious value of the kth word in the sentence, which belongs to the word V i ; The malicious value of each word and identifiers [cls] and [sep] in the sentence is calculated, and the malicious value of identifiers [cls] and [sep] is 1 by default; S44, construct a malicious value vector using the malicious value of each word calculated above: P={P cls ,P1,P2,...,P k ,...,P L ,P sep } The malicious value vector P is normalized to obtain the malicious weight vector P′. For a sentence of length L, an item P′ in the weight vector P′ k for: Finally, the text vector H output by the Bert pre-training model L Multiply it with the malicious weight vector P′ to get the malicious weight vector H′ L for: H′ L =P′H L Among them, P ′ is the malicious weight vector, H L The text vector output by the Bert pre-trained model.

6. According to claim 5, a malicious URL detection method integrating a malicious dictionary and a BERT pre-training model is characterized in that: In step S5, the processing process of the multi-scale convolution module includes: S51, introduce channel attention ECA to obtain the channel-weighted text vector matrix H″ L ; S52, using convolution kernels of different row dimension sizes to weight the text vector matrix H″ after channel weighting L Convolution is performed, and the convolution kernel filters some words in the input sentence in a sliding order and generates features Then we get the feature vector A c ; S53, preserve the contextual relationship in the feature vector through multi-head self-attention pooling; S54, the feature vector M obtained after pooling is calculated through the fully connected layer to calculate the score z of the i-th category i , thereby constructing a score vector Z, and finally judging malicious URLs based on the score vector Z through the Softmax function.

7. According to claim 6, a malicious URL detection method integrating a malicious dictionary and a BERT pre-training model is characterized in that: In step S51, the kernel size k of the one-dimensional convolution is firstly adaptively calculated according to the number of channels. The calculation formula of the kernel size k is: Where C represents the number of channels of the input features, γ and β are hyperparameters, and |·| odd Indicates that k is an odd number; The size of the input text vector matrix is ​​C×L×H; global features are extracted through global average pooling to obtain a 1×1×C vector; a one-dimensional convolution operation with a convolution kernel size of k is performed to capture cross-channel interaction information and obtain the channel weight vector ω; the weight is normalized using the Sigmoid function; each channel weight generated is multiplied by the corresponding channel of the input word vector matrix to obtain the weighted text vector matrix H″ L .

8. According to claim 6, a malicious URL detection method integrating a malicious dictionary and a BERT pre-training model is characterized in that: In step S52, the final feature vector A c It is expressed as: feature The calculation formula is: in, represents the i-th feature generated by the convolution kernel size c, i = 1, 2, ..., L-c+1; The matrix is ​​from row i to row i+c-1, which is a sliding window representing text vectors at different positions; W is the weight in the network and b is the bias.

9. According to claim 6, a malicious URL detection method integrating a malicious dictionary and a BERT pre-training model is characterized in that: In step S53, the multi-head self-attention is calculated as: A=Concat(A 2 ,A 3 ,A 4 ) Among them, Q h , K h 、V h A vector representing the query, key, and value for the h-th attention head; is the initialization weight matrix; T represents the transpose of the vector; A 2 , A 3 , A 4 are the eigenvectors obtained when the convolution kernel size is 2, 3, and 4 respectively, and A is the matrix after the eigenvectors are concatenated; d k are the dimensions of query, key, and value; The long pooling calculation is: P h =head h ×u h a h =Softmax(P h ) head′ h =a h T ×head h M=Concat(head′1,head′2,...,head′ h ) Among them, u h To initialize the parameter vector, head h with u h Perform matrix multiplication to obtain the joint vector P h ; Use the Softmax function to calculate the probability distribution vector a between each row vector h ; a h T is the vector a h , and then a h T With the matrix head h Multiply to get vector head′ h ; The information obtained from each space is spliced ​​to obtain the pooled vector M.

10. The method for detecting malicious URLs by integrating a malicious dictionary and a Bert pre-training model according to claim 6, characterized in that: In step S54, the score z of the i-th category of the feature vector M is i for: z i =W i M+b i where z i represents the score of the i-th category, W i and b i is the weight and bias corresponding to category i; Then the score vector Z is expressed as: Z=[z1,z2,...,z i ,...,With α ] Where α represents the number of categories, which is determined by the category distribution of the data annotation; Pass the score vector Z through the Softmax function to calculate the probability of each category: where q i represents the probability of the i-th category; Get the probability vector Q of length α: Q=[q1,q2,...,q i ,...,q α ] Calculate the category with the largest probability value in the probability vector Q: i max =argmax(Q) where i max Indicates the index corresponding to the maximum probability value; i max Match the categories marked with the S2 data to obtain the detection results.

Images