Phishing url detection method and system based on pre-trained language model

Abstract

The application discloses a phishing URL detection method and system based on a training language model, and the method comprises the following steps: acquiring a URL data source and constructing a URL corpus; constructing a URL word segmentation system, training the URL word segmentation system and initializing word embedding, and obtaining a URL initialized subword vector; constructing a multi-layer network encoder, training the multi-layer network editor based on a triple mask, and obtaining a fuzzy mask language model; performing fine-tuning processing on the fuzzy mask language model to obtain a phishing URL detection task model; and inputting a URL to be detected into the phishing URL detection task model to obtain a detection result. The application has a very high phishing website detection success rate, and the recognition rate reaches more than 99% on public data evaluation; only the original URL needs to be sent into the model, and the model can automatically perform word segmentation, generate a word vector, extract features and training; the model of the application improves the accuracy, and the fuzzy mask language model of the application can reduce the difficulty of pre-training.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data, and particularly relates to a phishing URL detection method and system based on a trained language model. BACKGROUND

[0002] Phishing attacks are the most common cyber attacks on the Internet, aiming to deceive and steal the private information of Internet users. Usually, attackers send phishing websites disguised as seemingly legitimate pages to unsuspecting users through email, social media or other online channels, luring them to submit bank accounts, identity information or download malicious software. In recent years, machine learning techniques have been widely used to automatically detect phishing URLs, thereby protecting Internet users from phishing attacks. Machine learning-based methods define phishing detection as a binary classification problem, using a set of labeled training samples (composed of normal URLs and phishing URLs) to train a binary classifier to infer whether a URL on the Internet is related to phishing.

[0003] Current anti-phishing machine learning methods still face many challenges. First, the amount of phishing URL data is extremely imbalanced with the amount of normal URL data in the real Internet world, while most machine learning algorithms for classification are designed around the assumption that the number of examples for each class is equal, resulting in the phishing URL detection model established being prone to high false positives. Second, to evade network regulation, phishing URLs usually appear in new variants by simply replacing characters after a short period of activity. However, current methods are difficult to detect these variants due to their inability to learn the semantic information of the URL. Three, the current phishing URL detection is severely limited by the size of the available phishing data. SUMMARY

[0004] The present application provides a phishing URL detection method and system based on a trained language model to solve the problems in the prior art.

[0005] To solve the above technical problems, the present application is solved by the following technical solutions:

[0006] A phishing URL detection method based on a trained language model, comprising the following steps:

[0007] Obtaining a URL data source and constructing a URL corpus;

[0008] Based on the URL corpus, a URL segmentation system is constructed, the URL segmentation system is trained and word embedding is initialized to obtain URL initialized subword vectors;

[0009] A multi-layer network encoder is constructed, the URL initialization subword vector is input into the multi-layer network encoder to obtain position encoding, a subword sequence containing position encoding is obtained, and the subword sequence containing position encoding is processed by multi-head attention information to obtain attention scores;

[0010] Perform a mask language model pre-training task: train the multi-layer network editor based on a triple mask to obtain a fuzzy mask language model, wherein the triple mask is formed based on URL initialization subword vector combination to obtain an average embedding vector;

[0011] Fine-tune the fuzzy mask language model to obtain a phishing URL detection task model;

[0012] Input the URL to be tested into the phishing URL detection task model to obtain a detection result.

[0013] As an implementable manner, the expression of the URL corpus is as follows:

[0014] D = unique (D1∪D2∪…∪D n )

[0015] Wherein, D1, D2, D3, D4…D n represent different sources of URL data sources.

[0016] As an implementable manner, the URL corpus is used to construct a URL tokenization system, including the following steps:

[0017] Generate a URL tokenization system based on a URL corpus: initialize a seed vocabulary, split the URLs in the URL corpus into single characters, and merge the single characters based on the BPE algorithm to form character blocks, wherein each character and special symbol is a seed word, and other seed words are generated;

[0018] Train the URL tokenization system and initialize the word embedding.

[0019] As an implementable manner, the training of the URL tokenization system and the initialization of the word embedding includes the following steps:

[0020] Based on the URL tokenization system, calculate the probability of each subword in the seed vocabulary in the URL sequence;

[0021] Assuming that each subword is discarded, the loss value generated when each subword is discarded is obtained by the maximum likelihood algorithm;

[0022] Discard the subword corresponding to the maximum loss value, and retain the subwords corresponding to the remaining loss values;

[0023] The above steps are repeated until the final dictionary reaches the preset size, or until the number of subwords does not change after consecutive iterations, obtaining the URL dictionary and the segmenter;

[0024] The initial numerical representation of the URL is obtained by initializing the word embedding based on the index of the URL dictionary.

[0025] As an implementable manner, the multi-layer network encoder is constructed, including the following steps:

[0026] The multi-layer network editor includes 12 Transformer encoders, each of which includes a multi-head attention component and a feed-forward layer component;

[0027] When the initial embedding representation of the URL is an average embedding vector input into the multi-layer network editor, a position encoding is obtained, which is a d-dimensional vector containing specific position information in the sequence, defined as follows:

[0028]

[0029] Where i represents the value of the ith element in the d-dimensional position encoding vector, n=10000, 0≤k<d / 2, w represents the position of the element in the input, represents the corresponding encoding, d represents the encoding dimension, represents a function that generates an output vector, N is a natural number, and R is a real number;

[0030] The position encoding is added to the actual embedding representation, and the input subword sequence is equipped with the corresponding position information, obtaining a subword sequence containing position encoding;

[0031] The multi-head attention component processes the subword sequence containing position encoding to obtain attention scores, and adds the attention scores to the actual embedding representation. The attention mechanism of the Transformer encoder is realized by three matrices, and the calculation formula of the attention mechanism is represented as follows:

[0032] Where Attention(Q,K,V) represents the weighted sum, Q represents the query, K represents the key, and V represents the value.

[0033] The feed-forward layer component includes two linear transformation functions and a ReLU activation function, and the feed-forward layer component is represented as: FFN(x) = ReLU(xW1+b1)W2+b2

[0034] Where X represents the input, W1, b1, W2, and b2 represent parameters, respectively.

[0035] As an implementable manner, the fine-tuning of the fuzzy mask language model comprises the following steps:

[0036] An application network component is added to the fuzzy mask language pre-training model.

[0037] The parameters of the application network component and the fuzzy mask language pre-training model are adjusted through supervised contrast learning, and a loss function of the supervised contrast learning is represented as follows:

[0038]

[0039] A loss function of cross entropy is represented as follows:

[0040]

[0041] Then, a whole loss function is represented as follows:

[0042]

[0043] Wherein, C represents a task category, N represents a batch of training samples, x i represents a sample in the batch, y i represents a label of the sample in the batch, and represents a Transformer encoder, an output of is a normalized result of a hidden layer l2, represents a total number of samples with the same label y i in the batch, is a scalar temperature parameter that can be adjusted, c represents a label of y i , y i,c represents a model output of a probability that the i th th sample belongs to the class c, and represents a scalar weighting hyperparameter that can be adjusted.

[0044] A phishing URL detection system based on a training language model comprises an acquisition and construction module, a construction word segmentation system module, a construction encoder module, an execution task module, a fine-tuning module, and a result detection module.

[0045] The acquisition and construction module is configured to acquire a URL data source and construct a URL corpus.

[0046] The construction word segmentation system module is configured to construct a URL word segmentation system based on the URL corpus, train the URL word segmentation system, and initialize word embedding to obtain a URL initialized subword vector.

[0047] The constructing encoder module is configured to construct a multi-layer network encoder, input the URL initialization subword vector into the multi-layer network encoder to obtain position encoding, obtain a subword sequence containing position encoding, and perform multi-head attention information processing on the subword sequence containing position encoding to obtain attention scores.

[0048] The executing task module is configured to perform a mask language model pre-training task, train the multi-layer network editor based on a triple mask to obtain a fuzzy mask language model, wherein the triple mask is obtained by combining URL initialization subword vectors to form a triple embedding vector and then obtaining an average embedding vector.

[0049] The fine-tuning processing module is configured to fine-tune the fuzzy mask language model to obtain a phishing URL detection task model.

[0050] The result detection module is configured to input a URL to be tested into the phishing URL detection task model to obtain a detection result.

[0051] As an implementable manner, the constructing encoder module is configured to:

[0052] The network model includes 12 Transformer encoders, and each Transformer encoder includes a multi-head attention component and a feed-forward layer component.

[0053] When the URL initialization embedding representation is input into the network model, position encoding is obtained, the position encoding is a d-dimensional vector containing specific position information in the sequence, and is defined as follows:

[0054]

[0055] wherein i represents the value of the i-th element in the d-dimensional position encoding vector, n = 10000, 0 ≤ k < d / 2, and w represents the position of the element in the input, represents the corresponding encoding, d represents the encoding dimension, represents a function for generating an output vector, N is a natural number, and R is a real number.

[0056] The position embedding is added to the actual embedding representation, and the input subword sequence is provided with corresponding position information to obtain a subword sequence containing position information.

[0057] The multi-head attention component performs multi-head attention information processing on the subword sequence containing position information to obtain attention scores, and adds the attention scores to the embedding representation. The attention mechanism of the Transformer encoder is realized by three matrices, and the calculation formula of the attention mechanism is represented as follows:

[0058] wherein Attention(Q,K,V) represents a weighted sum, Q represents a query, K represents a key, and V represents a value;

[0059] The feedforward layer component is provided with two linear transformation functions and a ReLU activation function, and a calculation formula of the feedforward layer component is: FFN(x) = ReLU(xW1+b1)W2+b2

[0060] wherein X represents an input, W1, b1, W2, and b2 represent parameters, respectively.

[0061] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method described below:

[0062] An URL data source is obtained, and an URL corpus is constructed;

[0063] Based on the URL corpus, an URL word segmentation system is constructed, the URL word segmentation system is trained, and word embedding is initialized to obtain an URL initial subword vector;

[0064] A multi-layer network encoder is constructed, the URL initial subword vector is input into the multi-layer network encoder to obtain a position encoding, a subword sequence containing the position encoding is obtained, and the subword sequence containing the position encoding is processed by multi-head attention information to obtain an attention score;

[0065] A mask language model pre-training task is performed: the multi-layer network editor is trained based on a triple mask to obtain a fuzzy mask language model, wherein the triple mask is: a triple embedding vector is formed based on the URL initial subword vector, and then an average embedding vector is obtained;

[0066] The fuzzy mask language model is fine-tuned to obtain a phishing URL detection task model;

[0067] The URL to be detected is input into the phishing URL detection task model to obtain a detection result.

[0068] A phishing URL detection device based on a trained language model, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the method described below when executing the computer program:

[0069] An URL data source is obtained, and an URL corpus is constructed;

[0070] Based on the URL corpus, an URL word segmentation system is constructed, the URL word segmentation system is trained, and word embedding is initialized to obtain an URL initial subword vector;

[0071] The URL initialization subword vector is input into the multi-layer network encoder to obtain position encoding, and a subword sequence containing position encoding is obtained.

[0072] The mask language model pre-training task is performed: the multi-layer network editor is trained based on a triple mask to obtain a fuzzy mask language model, wherein the triple mask is obtained by combining the URL initialization subword vector to form a triple embedding vector and then obtaining an average embedding vector.

[0073] The fuzzy mask language model is fine-tuned to obtain a phishing URL detection task model.

[0074] The URL to be tested is input into the phishing URL detection task model to obtain a detection result.

[0075] The present application has the following technical solutions, which has the following technical effects:

[0076] The present application has a very high phishing website detection success rate, and the recognition rate reaches more than 99% in public data evaluation; the present application only needs to send the original URL into the model, and the model will automatically perform word segmentation, generate word vectors, feature extraction and training; the method of the present application is a pre-training model completely driven by URL data, which is a truly complete URL-based pre-training model, which makes the model of the present application learn the fundamental features and deep semantic features of URL from almost all Internet distributed data corpus in a self-supervised manner, thereby improving the accuracy; the fuzzy mask language model of the present application can reduce the difficulty of pre-training; during fine-tuning, the contrast learning method is used to enhance the difference between the phishing URL and the normal URL, and only a small amount of labeled data is needed to train a good task model. BRIEF DESCRIPTION OF DRAWINGS

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0078] Figure 1 is a schematic diagram of the overall process of the method of the present application;

[0079] Figure 2 is a schematic diagram of the overall structure of the system of the present application;

[0080] Figure 3It is a fuzzy mask language model pre-training task schematic diagram of the present application;

[0081] Figure 4 It is a technical flow chart of the present application word segmentation system;

[0082] Figure 5 It is the overall framework diagram of the present application flow. DETAILED DESCRIPTION

[0083] The present application will be further described in detail below in combination with examples, and the following examples are an explanation of the present application and the present application is not limited to the following examples.

[0084] Example 1:

[0085] A phishing URL detection method based on a trained language model, as shown in Figure 1 , comprising the following steps:

[0086] S100, acquiring URL data source, constructing URL corpus;

[0087] S200, based on the URL corpus, constructing the URL word segmentation system, training the URL word segmentation system and initializing the word embedding, obtaining the URL initialized subword vector;

[0088] S300, constructing a multi-layer network encoder, inputting the URL initialized subword vector into the multi-layer network encoder to obtain a position encoding, obtaining a subword sequence containing a position encoding, and processing the subword sequence containing the position encoding to obtain an attention score;

[0089] S400, performing a mask language model pre-training task: training the multi-layer network editor based on a triple mask to obtain a fuzzy mask language model, wherein the triple mask is: forming a triple embedding vector based on the URL initialized subword vector and then obtaining an average embedding vector;

[0090] S500, fine-tuning the fuzzy mask language model to obtain a phishing URL detection task model;

[0091] S600, inputting the URL to be tested into the phishing URL detection task model to obtain a detection result.

[0092] In step S100, the expression of the URL corpus is as follows:

[0093] D=unique(D1∪D2∪…∪D n )

[0094] Wherein, D1, D2, D3, D4…D n represent different sources of URL data sources.

[0095] In the present application, various different URL data sources on the Internet are crawled to build a URL corpus of the largest scale so far to pre-train a language model. The corpus is denoted as D, and the resources covered by it are denoted as D i ,D=unique(D1∪D2∪…∪D n ). D1 contains 2.85 billion URLs of web pages extracted from Common Crawl; D2 contains a total of 330 million URLs, which are obtained by crawling the top 10 million websites in the Open PageRank program; D3 is a dataset containing 538.2 million URLs; and D4 is a set of 2.2 million URLs extracted from PhishTank.

[0096] In step S200, based on the URL corpus, a URL segmentation system is constructed, including the following steps:

[0097] Generating a URL segmentation system based on the URL corpus: initializing a seed vocabulary table, splitting the URLs in the URL corpus into single characters, and merging the single characters based on the BPE algorithm to form character blocks, wherein each character and special symbol is a seed vocabulary, and other seed vocabularies are generated; training the URL segmentation system and initializing word embedding.

[0098] Training the URL segmentation system and initializing the word embedding can be achieved by the following steps:

[0099] Based on the URL segmentation system, the probability of each subword in the seed vocabulary table in the URL sequence is calculated;

[0100] Suppose each subword is discarded, then the loss value generated when each subword is discarded is obtained by the maximum likelihood algorithm;

[0101] Discarding the subword corresponding to the maximum loss value and retaining the subword corresponding to the remaining loss value;

[0102] Repeat the above steps until the final dictionary reaches a preset size, or until the number of subwords does not change after continuous iterations, to obtain a URL dictionary and a segmenter;

[0103] Initializing the word embedding based on the index of the URL dictionary to obtain the initial digital representation of the URL.

[0104] The above process can be understood as the following steps:

[0105] 1) Calculate the probability of each word in the seed vocabulary table in the URL sequence;

[0106] 2) the loss value caused by discarding each subword calculated by the expectation-maximization algorithm;

[0107] 3) discarding the token with the largest loss value. In step 3), a certain percentage of subwords, such as 20% or 30% of subwords (of course, other data can also be selected, which is not limited here), are discarded according to the loss calculation;

[0108] 4) repeating steps 1) to 3) until the desired final dictionary size is reached, or until the number of subwords does not change after consecutive iterations, a URL dictionary and a segmenter can be obtained, and then the word embedding can be initialized based on the index of the URL dictionary, so as to obtain the initial digital representation of the URL.

[0109] The expectation-maximization algorithm (EM), or the Dempster-Laird-Rubin algorithm, is a class of optimization algorithms that iteratively perform maximum likelihood estimation (MLE), and is usually used as an alternative to the Newton-Raphson method for parameter estimation of probability models containing latent variables or incomplete data.

[0110] As shown in the accompanying drawings, Figure 4 Based on a corpus of more than 3 billion URLs obtained from the Internet, a URL segmentation system is first constructed, including 1) treating the URL string as a stream of unicode characters, which makes it unnecessary to specially process special symbols in the URL; 2) using the BPE algorithm to merge unicode characters and generating a large seed dictionary based on the frequency of the merged string in the URL corpus; 3) based on a unigram language model task, calculating the probability of each subword appearing in the entire sequence, and using the expectation-maximization algorithm to calculate the loss caused by discarding a specific subword; 4) according to the loss, discarding a part of the subwords, and taking the remaining subwords as the dictionary of the URL sequence; 5) implementing an automatic segmentation system based on the dictionary and the learned language model.

[0111] In step S300, a multi-layer network encoder is constructed, including the following steps:

[0112] The multi-layer network editor includes 12 Transformer encoders, each of which includes a multi-head attention component and a feedforward layer component;

[0113] When the initialization embedding representation of the URL is input into the multi-layer network editor, the position encoding is obtained, which is a d-dimensional vector containing specific position information in the sequence, and is defined as follows:

[0114]

[0115] where i represents the value of the ith element in the d-dimensional position encoding vector, n = 10000, 0 ≤ k < d / 2, and w represents the position of the element in the input, represents the corresponding encoding, d represents the encoding dimension, represents a function that generates an output vector, N is a natural number, and R is a real number;

[0116] The position encoding is added to the actual embedding representation, and the corresponding position information is provided for the input subword sequence, obtaining a subword sequence containing position encoding;

[0117] The multi-head attention component processes the subword sequence containing position encoding to obtain attention scores, and adds the attention scores to the actual embedding representation. The attention mechanism of the Transformer encoder is implemented through three matrices, and the calculation formula of the attention mechanism is represented as follows:

[0118] where Attention(Q, K, V) represents the weighted sum, Q represents the query, K represents the key, and V represents the value;

[0119] The feedforward layer component includes two linear transformation functions and a ReLU activation function, and the feedforward layer component is represented as: FFN(x) = ReLU(xW1+b1)W2+b2

[0120] where X represents the input, W1, b1, W2, and b2 represent parameters, respectively.

[0121] This multi-layer network encoder construction step is implemented using a basic Transformer encoder network model. In this embodiment, 12 standard Transformer encoders are used to form a deep and large network. The Transformer encoder network model includes two key components: a multi-head attention component and a feedforward layer component. The processing flow of the data stream through each component is as follows:

[0122] The initialization embedding representation of the URL is input into the multi-layer network editor model, and first, the position encoding is added to obtain the basic information of the position and order of the elements. This position encoding defines the syntax and thus defines the actual semantics of the sequence. This position encoding is not a number, but a d-dimensional vector containing specific position information in the sequence. Define w as the position of the element in the input, where d is the dimension of the encoding, and where represents the function that produces the output vector, the formula for the positional encoding is as follows:

[0123]

[0124] where i represents the value of the ith element in the d-dimensional positional encoding vector, n = 10000, and 0 < k < d / 2. This positional encoding scheme allows the model to effortlessly participate in relative positioning, and the computed positional embeddings are added on top of the actual embedding representations to equip the input subword sequences with their positional information.

[0125] The embedding representations configured with the positional encoding are then passed into the multi-layer network editor model, which is processed by the multi-head attention component. The multi-head attention component uses three trainable linear layers to compute three matrices, called queries, keys, and values, to generate embedding representations with attention scores. As the data flows through the later Transformer encoders in the stack, each multi-head attention component adds its own attention scores to the representation. The attention mechanism describes the way in which the neural network dynamically learns the weighted average of the sequence elements. The attention mechanism in the Transformer encoder is achieved through the three matrices mentioned above, usually denoted by Q (query), K (key), and V (value). The output of the attention is a weighted sum of the values, where the weight assigned to each value is computed by the query with the corresponding key, as follows:

[0126] After the multi-head attention component is a fully connected feed-forward layer component, which is provided with two linear transformation functions and a ReLU activation function. This feed-forward layer component shares parameters across each position, so a separate, identical linear transformation is applied to each element of a given sequence. The purpose of this is to project the output of the attention layer into a larger space, making it easier to disentangle information. The computation of the feed-forward layer component is as follows:

[0127] FFN(x) = ReLU(xW1 + b1)W2 + b2.

[0128] In step S400, the fuzzy mask language model is an improvement of the classic mask language model pre-training task, which aims to make the pre-training task more suitable for URL data and clearly describe the design intention and implementation details of the fuzzy mask language model. URL data contains protocols, domain names, file paths, and possibly parameters and anchors. Due to the extremely diverse characters of the components, the semantic relationship space of the token sequence can be very large, making self-supervised pre-training on URL data more difficult than on text. The fuzzy mask language model reduces the semantic space in URL data by using a triple mask method. As shown in Figure 3 The fuzzy mask language model eliminates the order relationship of tokens in the token triple, thereby reducing the original semantic space by 6 times compared with the standard mask language model. On the other hand, the triple method contains the neighborhood information of subwords, which also makes the triple-based model more accurate in context prediction. The fuzzy mask language model pre-training task masks 10% of the subword triples instead of individual subwords, and its training target is to train the model to predict the masked triple. 1) Initialize the vector after the original URL input; 2) use a sliding window of size 3 to lock three consecutive subwords one by one and calculate the average word vector one by one; 3) randomly mask the average word vector; 4) predict the masked vector.

[0129] In step S500, the trained multi-layer network editor model is fine-tuned to create a phishing URL detection task model, including the following steps:

[0130] In the fuzzy mask language model, the application network component is added, and the parameters of the application network component and the fuzzy mask language model are adjusted through supervised contrast learning. The loss function of supervised contrast learning is represented as follows:

[0131]

[0132] The loss function of cross-entropy is represented as follows:

[0133]

[0134] The overall loss function is represented as follows:

[0135]

[0136] Where C represents the task category, N represents the batch of training samples, x i represents the samples in the batch, and y iLet Φ(x) represent the labels of the samples in the batch, and let Φ(x) represent the Transformer encoder. The output of Φ(x) is the result of L2 normalization of the hidden layer. This indicates that the batch contains items with the same label y. i The total number of samples, θ>0 is an adjustable scalar temperature parameter; c represents y i The tag, y i,c Indicate i th The example outputs the model probability of belonging to class c, where β represents the scalar weighted hyperparameter that can be adjusted.

[0137] This process, known as fine-tuning, involves adding an application network to the multi-layer network editor model and then fine-tuning all model parameters end-to-end using a task training set containing both phishing and benign URL data. Considering that a good task model needs to capture the similarities between examples within a class and the differences between them and examples in other classes, supervised contrastive learning is introduced to improve the performance of the pre-training fine-tuning. This is achieved by calculating the Supervised Contrastive Learning (SCL) loss function, which is combined with the Cross-Entropy (CE) loss function to form the final learning objective of the fine-tuning stage. For a classification task with C classes, the batch size of training samples is N; x i ,y i Φ(x) represents the sample and the label of the sample in the batch, respectively; Φ(x) represents an encoder whose output is the L2 normalized result of the final hidden layer of the model. This indicates that the batch contains items with the same label y. i The total number of samples; θ>0 is an adjustable scalar temperature parameter that controls class separation; y i c represents y i The tag; y i,c Indicate i th The example outputs the model probability of belonging to class c; β is a scalar-weighted hyperparameter that is adjusted for each downstream task and setting, and the SCL loss function formula is as follows:

[0138]

[0139] here,

[0140] The loss due to cross-entropy is as follows:

[0141]

[0142] The overall loss is a weighted average of the SCL and CE losses, as follows:

[0143]

[0144] The method has a very high phishing website detection success rate, and the recognition rate reaches more than 99% in public data evaluation; the end-to-end and plug-and-play method does not require any manual data processing, only needs to send the original URL into the model, and the model can automatically perform word segmentation, generate word vectors, feature extraction and training; in addition, the pre-trained model trained based on the URL data makes the phishing URL detection task model learn the fundamental features and deep semantic features of the URL data in a self-supervised manner, so that the phishing URL detection task model achieves higher accuracy; the invention uses a fuzzy mask language model for the pre-training method of URL data, which can reduce the difficulty of pre-training; in addition, a fine-tuning process is added, and the difference between phishing URLs and normal URLs is enhanced by using the contrast learning method during fine-tuning, so that a good task model can be trained from a small amount of labeled data, and then a phishing URL detection task model is obtained.

[0145] Embodiment 2:

[0146] A phishing URL detection system based on a trained language model, as shown in Figure 2 The system includes an acquisition and construction module 100, a construction word segmentation system module 200, a construction encoder module 300, an execution task module 400, a fine-tuning processing module 500 and a result detection module 600.

[0147] The acquisition and construction module 100 is used to acquire a URL data source and construct a URL corpus.

[0148] The construction word segmentation system module 200 is used to construct a URL word segmentation system based on the URL corpus, train the URL word segmentation system and initialize word embedding, and obtain a URL initialized subword vector.

[0149] The construction encoder module 300 is used to construct a multi-layer network encoder, input the URL initialized subword vector into the multi-layer network encoder to obtain a position encoding, obtain a subword sequence containing the position encoding, and process the subword sequence containing the position encoding by multi-head attention information to obtain an attention score.

[0150] The execution task module 400 is used to perform a mask language model pre-training task: training the multi-layer network editor based on a triple mask to obtain a fuzzy mask language model, wherein the triple mask is: combining the URL initialized subword vector to form a triple embedding vector and then obtaining an average embedding vector.

[0151] The fine-tuning processing module 500 is used to fine-tune the fuzzy mask language model to obtain a phishing URL detection task model.

[0152] The result detection module 600 is configured to input the to-be-detected URL into the phishing URL detection task model to obtain a detection result.

[0153] In one embodiment, the construction encoder module 300 is configured to:

[0154] The network model comprises 12 Transformer encoders, each of which comprises a multi-head attention component and a feed-forward layer component.

[0155] When the initial embedding representation of the URL is input into the network model, a position encoding is obtained, which is a d-dimensional vector containing specific position information in the sequence and is defined as follows:

[0156]

[0157] where i represents the value of the ith element in the d-dimensional position encoding vector, n = 10000, 0 ≤ k < d / 2, and w represents the position of the element in the input, represents the corresponding encoding, d represents the encoding dimension, represents a function that generates an output vector, N is a natural number, and R is a real number.

[0158] The position embedding is added to the actual embedding representation, and the input subword sequence is provided with corresponding position information to obtain a subword sequence containing position information.

[0159] The multi-head attention component processes the subword sequence containing position information to obtain an attention score, and adds the attention score to the embedding representation. The attention mechanism of the Transformer encoder is implemented through three matrices, and the calculation formula of the attention mechanism is as follows:

[0160] where Attention(Q, K, V) represents a weighted sum, Q represents a query, K represents a key, and V represents a value.

[0161] The feed-forward layer component is provided with two linear transformation functions and a ReLU activation function, and the calculation formula of the feed-forward layer component is FFN(x) = ReLU(xW1+b1)W2+b2.

[0162] where X represents an input, W1, b1, W2, and b2 represent parameters, respectively.

[0163] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, and the like) having computer-usable program code embodied in the medium.

[0164] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0165] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0166] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0167] In addition, it should be noted that the specific embodiments described in the specification, the shape of the zero, components, and the name taken, etc. can be different. Any equivalent or simple change made in accordance with the structure, features and principles described in the patent concept of the present application is included in the protection scope of the present application. Those skilled in the art can make various modifications or supplements to the described specific embodiments or use similar ways to replace, as long as it does not deviate from the structure of the present application or exceed the scope defined by the present claims.

Claims

1. A method for detecting phishing URLs based on a trained language model, characterized in that, Includes the following steps: Obtain URL data sources and build a URL corpus; Based on the URL corpus, a URL segmentation system is constructed, and the URL segmentation system and initial word embeddings are trained to obtain URL initial sub-word vectors. A multi-layer network encoder is constructed. The URL initialization sub-word vector is input into the multi-layer network encoder to obtain position encoding, resulting in a sub-word sequence containing position encoding. The sub-word sequence containing position encoding is then processed with multi-head attention information to obtain an attention score. The multi-layer network encoder includes 12 Transformer encoders, each Transformer encoder including a multi-head attention component and a feedforward layer component; When the initial embedding representation of the URL is input into a multi-layer network encoder, a positional encoding is obtained, which contains specific positional information within the sequence. A dimensional vector is defined as follows: in, Indicates the first The elements are in The value in the dimensional positional encoding vector, =10000, , Indicates the position of the element in the input. This indicates the corresponding encoding. Indicates the encoding dimension. This represents a function that produces an output vector. For natural numbers, It is a real number; The positional encoding is added to the actual embedding representation, thereby equipping the input subword sequence with corresponding positional information, resulting in a subword sequence containing positional encoding; The multi-head attention component processes the position-encoded word sequence with multi-head attention information to obtain an attention score, which is then added to the actual embedding representation. The attention mechanism of the Transformer encoder is implemented through three matrices, and the calculation formula of the attention mechanism is as follows: in, Indicates a weighted sum. Indicates a query. Indicates key, Represents the value; If the feedforward layer component includes two linear transformation functions and a ReLU activation function, then the feedforward layer component is represented as follows: in, Indicates input, , , , They represent parameters respectively; Perform the masked language model pre-training task: train the multi-layer network encoder based on triple mask to obtain a fuzzy masked language model, wherein the triple mask is: based on the combination of sub-word vectors initialized by URL to form triple embedding vectors and then to obtain the average embedding vector; The fuzzy mask language model is fine-tuned to obtain a phishing URL detection task model; Input the URL to be tested into the phishing URL detection task model to obtain the detection result.

2. The phishing URL detection method based on a trained language model according to claim 1, characterized in that, The expressions in the URL corpus are as follows: in, , , , … This indicates URL data sources from different origins.

3. The phishing URL detection method based on a trained language model according to claim 1, characterized in that, The construction of a URL segmentation system based on a URL corpus includes the following steps: A URL segmentation system based on a URL corpus is generated by: initializing a seed vocabulary, splitting URLs in the URL corpus into individual characters, and merging these individual characters into character blocks based on the BPE algorithm. Each character and special symbol serves as a seed vocabulary to generate other seed vocabulary. Train the URL segmentation system and initialize word embeddings.

4. The phishing URL detection method based on a trained language model according to claim 3, characterized in that, The training URL segmentation system and the initialization of word embeddings include the following steps: Based on the URL segmentation system, calculate the probability of each subword in the seed vocabulary in the URL sequence; Assuming each subword is discarded, the loss value generated when each subword is discarded is obtained using the expectation-maximization algorithm; Discard the sub-words corresponding to the maximum loss value and retain the sub-words corresponding to the remaining loss values; Repeat the above steps until the final dictionary reaches the preset size, or until the number of subwords does not change after continuous iteration, to obtain the URL dictionary and the word segmenter; Initialize word embeddings based on the URL dictionary index to obtain the initial numerical representation of the URL.

5. The phishing URL detection method based on a trained language model according to claim 1, characterized in that, The fine-tuning of the fuzzy mask language model includes the following steps: Add an application network component to the fuzzy mask language pre-trained model; The parameters of the application network components and the fuzzy mask language pre-trained model are adjusted through supervised contrastive learning. The loss function of the supervised contrastive learning is expressed as follows: The cross-entropy loss function is expressed as follows: The overall loss function is then expressed as follows: in, Indicates the type of task. Indicates the batch of training samples. Indicates the samples in the batch. Labels indicating samples within a batch. Indicates Transformer encoder, The output is the hidden layer. The result of normalization, This indicates that the batch contains the same label. The total number of samples, It is an adjustable scalar temperature parameter; express The tag, express The example belongs to the class The model output of the probability, This represents a scalar-weighted hyperparameter that can be adjusted.

6. A phishing URL detection system based on a trained language model, characterized in that, It includes a module for acquiring the building block, a module for building the word segmentation system, a module for building the encoder, a module for executing tasks, a module for fine-tuning, and a module for detecting results; The acquisition and construction module is used to acquire URL data sources and build a URL corpus; The module for constructing the word segmentation system is based on a URL corpus, constructs a URL word segmentation system, trains the URL word segmentation system and initializes word embeddings, and obtains URL initialization sub-word vectors; The encoder construction module is used to construct a multi-layer network encoder. The URL initialization sub-word vector is input into the multi-layer network encoder to obtain position encoding, resulting in a sub-word sequence containing position encoding. The sub-word sequence containing position encoding is then processed with multi-head attention information to obtain an attention score. The multi-layer network encoder includes 12 Transformer encoders, each Transformer encoder including a multi-head attention component and a feedforward layer component; When the initial embedding representation of the URL is input into a multi-layer network encoder, a positional encoding is obtained, which contains specific positional information within the sequence. A dimensional vector is defined as follows: in, Indicates the first The elements are in The value in the dimensional positional encoding vector, =10000, , Indicates the position of the element in the input. This indicates the corresponding encoding. Indicates the encoding dimension. This represents a function that produces an output vector. For natural numbers, It is a real number; The positional encoding is added to the actual embedding representation, thereby equipping the input subword sequence with corresponding positional information, resulting in a subword sequence containing positional encoding; The multi-head attention component processes the position-encoded word sequence with multi-head attention information to obtain an attention score, which is then added to the actual embedding representation. The attention mechanism of the Transformer encoder is implemented through three matrices, and the calculation formula of the attention mechanism is expressed as follows: in, Indicates a weighted sum. Indicates a query. Indicates key, Represents the value; If the feedforward layer component includes two linear transformation functions and a ReLU activation function, then the feedforward layer component is represented as follows: in, Indicates input, , , , They represent parameters respectively; The execution task module is used to perform the masked language model pre-training task: train the multi-layer network encoder based on triple mask to obtain a fuzzy masked language model, wherein the triple mask is: based on the combination of sub-word vectors initialized by URL to form triple embedding vectors and then to obtain the average embedding vector; The fine-tuning processing module is used to fine-tune the fuzzy mask language model to obtain the phishing URL detection task model; The result detection module is used to input the URL to be tested into the phishing URL detection task model to obtain the detection result.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 5.

8. A phishing URL detection device based on a trained language model, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 5.

Images