Training method of URL (Uniform Resource Locator) classification model

Abstract

The invention provides a URL classification model training method. The method comprises the steps that token replacement detection training and token disruption detection training are executed on a first URL classification model through unmarked sample URLs to obtain a second URL classification model; mask learning, self-supervised contrast learning and virtual adversarial training are executed on the second URL classification model through the unmarked sample URL, and a third URL classification model is obtained; training a convolutional neural network based on the coding result of the first labeled sample URL and the real classification label, and linking the third URL classification model and the convolutional neural network to form a fourth URL classification model; and based on the second marked sample URL and the real classification mark thereof, carrying out fine adjustment on the model parameters of the fourth URL classification model. According to the technical scheme, the classification capability of the URL classification model on the specific URL classification task is improved on the basis of greatly reducing the number of marked samples.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a training method for a URL classification model. Background Art

[0002] URL (Uniform Resource Locator) is the cornerstone of the Internet and the basic element for browsing and understanding the network. It plays a vital role in key areas such as guiding users, network security, personalized online services, and network engineering. As a rich and easily accessible data source, URL has long been used to train various classifiers to complete specific tasks, such as malicious URL detection, web page topic classification, and advertising URL identification.

[0003] Existing URL classifiers are often trained for specific tasks using data related to the task, which requires manual labeling of the data related to the task in order to increase the URL classifier's ability to understand URL features.

[0004] However, with the rapid development of the Internet, the tasks that URL classifiers need to handle are diverse. If classifiers are trained separately for each task, it is quite time-consuming and laborious. On the other hand, URL classifiers trained for specific tasks cannot meet the diverse URL classification needs of various tasks. At the same time, when training classifiers, a large amount of labor costs are required to mark massive sample data. This process is time-consuming, labor-intensive, and prone to errors, which affects the classifier's learning level for samples, and ultimately limits the classification ability of the resulting URL classifier.

[0005] Therefore, how to efficiently and accurately train URL classifiers has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] The embodiment of the present application provides a method for training a URL classification model, aiming to solve the technical problems faced in the related art in training URL classifiers for specific tasks, such as time-consuming and labor-intensive sample labeling and insufficient classifier capabilities.

[0007] In a first aspect, an embodiment of the present application provides a method for training a URL classification model, comprising: In response to the training instruction for the first URL classification model, performing word segmentation processing on the unlabeled sample URL to obtain multiple tokens of the unlabeled sample URL; Performing token replacement detection training and token scrambling detection training on the first URL classification model through multiple tokens of the unmarked sample URL to obtain a second URL classification model; Using the unlabeled sample URL, performing mask learning, self-supervised contrastive learning and virtual adversarial training on the second URL classification model to obtain a third URL classification model; Obtaining a first labeled sample URL and a true classification label of the first labeled sample URL, and outputting an encoding result of the first labeled sample URL through the third URL classification model; Based on the encoding result and the real classification mark, training a convolutional neural network so that the third URL classification model and the convolutional neural network are linked to form a fourth URL classification model, wherein the convolutional neural network is used to reflect the association relationship between the encoding result of the first labeled sample URL and the predicted classification mark; Obtain a second labeled sample URL and a true classification label of the second labeled sample URL; Based on the second labeled sample URL and the true classification mark of the second labeled sample URL, the model parameters of the fourth URL classification model are fine-tuned, wherein the fourth URL classification model is used to reflect the association between the second labeled sample URL and the predicted classification mark of the second labeled sample URL.

[0008] In a second aspect, an embodiment of the present application provides a training device for a URL classification model, including: A word segmentation unit, configured to perform word segmentation processing on the unlabeled sample URL in response to a training instruction for the first URL classification model, to obtain a plurality of tokens of the unlabeled sample URL; A first unlabeled training unit is used to perform token replacement detection training and token scrambling detection training on the first URL classification model through multiple tokens of the unlabeled sample URL to obtain a second URL classification model; A second unlabeled training unit is used to perform mask learning, self-supervised contrastive learning and virtual adversarial training on the second URL classification model through the unlabeled sample URL to obtain a third URL classification model; A first acquisition unit is used to acquire a first labeled sample URL and a real classification label of the first labeled sample URL, and output an encoding result of the first labeled sample URL through the third URL classification model; A first labeled training unit, configured to train a convolutional neural network based on the encoding result and the true classification mark, so that the third URL classification model is linked to the convolutional neural network to form a fourth URL classification model, wherein the convolutional neural network is configured to reflect the association between the encoding result of the first labeled sample URL and the predicted classification mark; A second obtaining unit, configured to obtain a second labeled sample URL and a true classification label of the second labeled sample URL; The second labeled training unit is used to fine-tune the model parameters of the fourth URL classification model based on the second labeled sample URL and the real classification label of the second labeled sample URL, wherein the fourth URL classification model is used to reflect the association relationship between the second labeled sample URL and the predicted classification label of the second labeled sample URL.

[0009] In a third aspect, an embodiment of the present application provides a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are configured to execute the method described in the first aspect above.

[0010] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the method described in the first aspect above.

[0011] The above technical solution aims at the technical problems of time-consuming and laborious sample labeling and insufficient classifier capability faced by the related technology in training URL classifiers for specific tasks. The URL classification model of the Transformer architecture is trained through unlabeled sample URLs. In the first stage of the training process, a three-classification method is used to realize the learning ability of the model for replacement token detection and scrambled token detection. In the second stage of the training process, a combination of mask learning, self-supervised contrast learning and virtual adversarial training is used to further train the model's learning and analysis capabilities for the features of unlabeled sample URLs. On the basis of this training, a convolutional neural network is linked to the model, and the ability of the convolutional neural network to handle specific URL classification tasks is optimized through labeled sample URLs. Then, the parameters of the model and the convolutional neural network are optimized as a whole. In this way, the number of labeled samples required for the training process of the URL classification model can be greatly reduced, the manual labeling work required for training can be reduced, the difficulty of model training can be reduced, and the problem of inaccurate training results caused by excessive manual participation can be avoided. At the same time, through the additional convolutional neural network, the classification ability of the URL classification model for specific URL classification tasks can still be guaranteed on the basis of greatly reducing the number of labeled samples, ensuring the smooth progress of specific URL classification tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A flowchart of a method for training a URL classification model according to an embodiment of the present application is shown; Figure 2 A schematic diagram of the ROC curve of the URLBERT model in Table 2 under different data scales is shown; Figure 3 The figure shows the changes in the core performance indicators of the URLBERT model in different training time periods in the phishing URL detection task according to an embodiment of the present application; Figure 4 The figure shows the changes in the core performance indicators of the BiLSTM model in the related art in different training time periods in the phishing URL detection task; Figure 5 The changes in the core performance indicators of the TextCNN model in the related art in different training time periods in the phishing URL detection task are shown; Figure 6 A block diagram of a computer device according to an embodiment of the present application is shown; Figure 7 A block diagram of a computer device according to another embodiment of the present application is shown. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] Figure 1 A flowchart of a method for training a URL classification model according to an embodiment of the present application is shown.

[0016] like Figure 1 As shown, the training method of the URL classification model according to an embodiment of the present application includes: Step 102, in response to the training instruction for the first URL classification model, perform word segmentation processing on the unlabeled sample URL to obtain multiple tokens of the unlabeled sample URL.

[0017] The first URL classification model is an untrained initial BERT model based on the Transformer architecture, or a model to be optimized that uses unlabeled sample URLs and / or labeled sample URLs for URL classification training and whose URL classification capabilities are still insufficient. In this technical solution, the unlabeled sample URLs after word segmentation processing are directly used as training samples for the first URL classification model, and the process of manually labeling the training samples is omitted. It should be noted that any training process for unlabeled sample URLs described in the context is performed for each unlabeled sample URL set in the massive unlabeled sample URL set.

[0018] Specifically, the word segmentation process can be implemented by a word segmenter. Before step 102, a URL word segmenter can be trained based on a URL corpus set and a WordPiece algorithm, and then the URL word segmenter maps the characters of the unlabeled sample URL into a word sequence, and obtains the encoding representation information corresponding to each word in the word sequence to form a corresponding first token sequence.

[0019] Suppose the unmarked sample URL is ,in Indicates the first The URL tokenizer maps the character sequence in the unlabeled sample URL to a subword sequence based on the trained vocabulary and rules. ,in, represents the jth subword. This mapping process can be represented by a function f, that is, Each subword There is a corresponding encoding representation in the vocabulary. If the vocabulary is V, the subword The code is ,but .

[0020] During the word segmentation process, the URL word segmenter determines the encoding of each subword by looking up the vocabulary, thereby converting the unlabeled sample URL into an encoded representation to facilitate the computational processing of the model.

[0021] In addition, when encountering characters or subwords that are not in the vocabulary, the URL tokenizer can process them according to predefined rules. For example, the URL tokenizer can decompose the unknown subword into individual characters, or approximate the unknown subword based on existing subwords. Let the unknown subword be u, the URL tokenizer will find a set of subwords , so that u is approximately expressed as a combination of s. This process can be achieved by optimizing an objective function, which can be selected as a method to minimize the edit distance between u and the combination of s, expressed as: ,in, is the set of all possible subwords, is a function that converts a subword set s into an actual string, is a distance metric function. In this way, the word segmenter can handle various unknown situations that appear in unlabeled sample URLs, thereby ensuring the completeness and accuracy of word segmentation.

[0022] In addition, during the word segmentation process, a CLS token may be added before the first token of the first token sequence. , when After adding the CLS token, the new sequence becomes .

[0023] Among them, the hidden state of the CLS token It can be used as a basis for predicting URL categories. For example, in a URL classification model based on a neural network, there is a fully connected layer FC. The prediction result of the URL classification model is It can be expressed as .

[0024] During the word segmentation process, several PAD tokens can be added after the last token of the first token sequence to make the length of the first token sequence reach the specified length, so as to unify the length of the samples used in the training process, reduce the impact of unlabeled sample URLs of different lengths on the URL classification model training, and help improve the accuracy of URL classification model training.

[0025] Step 104: Perform token replacement detection training and token scrambling detection training on the first URL classification model using the multiple tokens of the unlabeled sample URL to obtain a second URL classification model.

[0026] After word segmentation, first, the first URL classification model is made to learn the two capabilities of replacement token detection and scrambled token detection as the first stage of training the URL classification model. Subsequent further training is based on the second URL classification model obtained in the first stage.

[0027] In one possible design, the model can be trained directly through a three-class classification approach to learn the difference between shuffled tokens, randomly replaced tokens, and original tokens.

[0028] Specifically, in step 102, first, multiple tokens of the untagged sample URL are scrambled and replaced to obtain a second token sequence, wherein the scrambling includes randomly selecting a first percentage of tokens from the multiple tokens to perform a scrambling operation, and the replacing tokens includes replacing a second percentage of tokens from the multiple tokens with random tokens.

[0029] Optionally, the first percentage and the second percentage are both 5%.

[0030] Optionally, the positions of the first percentage of shuffled tokens and the second percentage of replaced tokens in the second token sequence do not overlap.

[0031] Next, each token in the second token sequence is subjected to a three-classification process by the first URL classification model, and iterative training is performed until the first loss function is minimized, wherein the three-classification process is used to select the token type of the token from the shuffled token, the randomly replaced token, and the original token. Let the current model be M. For the first token in a single unlabeled sample URL, Tokens , M predicts the probabilities of the three situations as follows: (Scrambled tokens), (randomly replace tokens), (original tokens), and the corresponding true labels are .in, , is the first loss function, is the total number of tokens in the second token sequence, is the token number of the untagged sample URL, Indicates the first Tokens, is the serial number of the token type, is the first Token belongs to The probability of a token type, is the first The actual token type of the token.

[0032] The first loss function is the cross entropy loss function. By minimizing the first loss function, the current model can learn to identify the replacement and scrambling of tokens, and preliminarily understand the structure and semantic characteristics of the unlabeled sample URLs.

[0033] In the URL classification task, since URLs often serve as direct identifiers of web resources, even a change in a single token may direct users to a completely different and potentially harmful destination. Training with token replacement detection enables the current model to identify subtle changes within URLs, especially when key components within URLs are cleverly tampered with to mimic legitimate sites. The model trained with token replacement detection can effectively detect these changes, thereby enhancing its ability to prevent deceptive network behavior and improving the accuracy of malicious URL detection.

[0034] For example, when detecting phishing URLs, token replacement detection training can help the model identify the replaced domain part of the URL, thereby judging its possible maliciousness. Taking the example domain name "https: / / pilipalabiu.com" as an example, for "https: / / pilipalabiu.com", a portion of tokens (such as 5%) are randomly selected for replacement. Assuming that the token "com" is selected and replaced with a randomly selected word (such as "net"), the perturbed URL is "https: / / pilipalabiu.net".

[0035] At the same time, the order of tokens in the URL is used to reflect the hierarchical structure of the accessed network resources, so scrambling token detection training can make the model understand the positional significance of the token order in the URL. Similar to the impact of word order on grammar and readability in standard text, the order of tokens or markers in the URL will also affect the navigation path of the web page content. Through scrambling token detection training, the model's ability to identify and evaluate the order of markers can be improved, increasing the accuracy and reliability of the model's URL classification.

[0036] Step 106: Using the unlabeled sample URL, perform mask learning, self-supervised contrastive learning and virtual adversarial training on the second URL classification model to obtain a third URL classification model.

[0037] Mask learning allows the current model to learn context and fill in missing or blocked parts of the URL, thereby enhancing the URL classification model's ability to infer and understand URL patterns; self-supervised contrastive learning aims to learn subtle differences between samples and learn distinguishable representations by simultaneously maximizing the consistency between various transformations of the same instance (positive samples) and minimizing the consistency between different instances (negative samples); virtual adversarial learning aims to improve the generalization ability of the URL classification model when parsing slightly different URLs.

[0038] In one possible design, mask learning, self-supervised contrastive learning and virtual adversarial training are performed successively on the second URL classification model.

[0039] In another possible design, mask learning is performed on the second URL classification model to obtain an intermediate model, and self-supervised contrastive learning and virtual adversarial training are performed on the intermediate model simultaneously; or In another possible design, mask learning, self-supervised contrastive learning and virtual adversarial training are performed simultaneously on the second URL classification model.

[0040] It can be seen that mask learning, self-supervised contrastive learning and virtual adversarial training can be executed in parallel, or they can be performed successively in any structural manner that meets the actual URL classification training needs, as long as the purpose of improving the classification ability of the URL classification model can be achieved.

[0041] The specific method of mask learning is as follows: randomly replacing a third percentage of tokens with masked tags among multiple tokens of the untagged sample URL to obtain a masked third token sequence; outputting predicted tokens of the masked tags in the third token sequence through the second URL classification model, and updating the parameters of the second URL classification model based on the difference between the predicted token and the original real token corresponding to the masked tag, and iterating training until the second loss function is minimized, wherein, , is the second loss function, is the total number of tokens in the third token sequence, Indicates the first The real token at the token position, Indicates that the second URL classification model The predicted results of the token probability distribution at the token position.

[0042] The task of mask learning is to mask some tokens in the token sequence of the unlabeled sample URL and then predict the masked tokens. Suppose the token sequence of the unlabeled sample URL is , in this sequence, a specified proportion of positions are randomly selected for masking. Let the original token be , the masked token is ,but

[0043] Where p is the mask probability, which can be selected as 15%.

[0044] For the position selected to be masked, its original token is replaced by a special mask marker, such as [MASK], to obtain the masked sequence At this point, the second URL classification model needs to predict the original token at the masked position based on the context information, that is, other original tokens that are not masked in the token sequence of the unmarked sample URL.

[0045] The current model learns a large number of unlabeled sample URLs in mask learning and adjusts its internal parameters to maximize the accuracy of predicting masked tokens. It can be understood that the current model learns the mapping function in mask learning. , so that ,in, The parameters of the current model The function of determination is , further, the current model optimizes the parameters by minimizing the difference between the predicted result and the original token at the masked position , achieving effective learning of the semantic and structural features of unlabeled sample URLs.

[0046] Among them, self-supervised contrastive learning aims to learn the feature representation of the data by comparing positive and negative samples, so that the model can understand the subtle differences between samples and obtain distinguishable representations by simultaneously maximizing the consistency between various transformations of the same instance (positive samples) and minimizing the consistency between different instances (negative samples).

[0047] Specifically, in the self-supervised contrastive learning, firstly, the enhanced data samples of the unlabeled sample URL are obtained by random dropout and fast gradient sign method.

[0048] The random loss method is the Dropout technology. Let the original unlabeled sample URL be , the Dropout probability is , then the samples after Dropout processing as follows: , in One with A tensor of the same shape, where each element takes the value 1 with probability p and takes the value 0 with probability 1-p.

[0049] Furthermore, the fast gradient sign method (FGSM) is used for adversarial learning. Let the original unlabeled sample URL be The disturbance intensity is , the reasoning process of the current model on the mask learning task is , then the adversarial sample as follows: , Among them, the sign function is used to regularize the gradient.

[0050] At this point, the number of enhanced data samples obtained is twice the number of tokens in the unmarked sample URL, and the enhanced data samples include one positive sample and multiple negative samples.

[0051] Next, based on the second URL classification model, the distance between the positive samples and the distance between the negative samples in the unlabeled sample URL and the enhanced data sample are determined, and iterative training is performed until the third loss function is minimized, wherein: , is the third loss function, is the total number of tokens of the unmarked sample URL, and They are respectively the first The augmented data samples, The augmented data samples and augmented data samples, sim() is the cosine similarity function, is the temperature coefficient, is the serial number of the enhanced data sample. The third loss function is a normalized temperature scaled cross entropy loss function, which aims to maximize the distance between negative samples while minimizing the distance between positive samples.

[0052] Taking the example domain name "https: / / pilipalabiu.com" as an example, Dropout is used to generate the initial enhanced sample. Specifically, for "https: / / pilipalabiu.com", Dropout technology is first applied. If the Dropout probability is p=0.2, after Dropout, some tokens may be randomly set to zero to obtain an enhanced sample, such as ["https",": / / ","0",".","com"], where "0" represents the result after Dropout processing. This is just a simplified example. In the actual calculation scenario, "0" can be selected as a representation content randomly selected according to probability.

[0053] Continuing with the example domain name "https: / / pilipalabiu.com", FGSM is used to generate adversarial samples as positive and negative samples. Specifically, the obtained positive sample can be the correct address of another example domain name "https: / / fafabook.com", and one of the obtained negative samples is ["https",": / / ","piliopalabiu",".","com"], where "piliopalabiu" is obtained by adding perturbations and is different from the original "pilipalabiu". This negative sample has a certain similarity with the original URL, but there are subtle differences. For another example, the generated negative sample may also be ["https",": / / ","fasebook",".","com"]. Similarly, fasebook is the result of fafabook in https: / / fafabook.com after perturbation.

[0054] The specific method of virtual adversarial training is: based on the fast gradient sign method, a perturbation is added in the gradient ascending direction of the third token sequence to obtain an adversarial sample of the third token sequence; based on the similarity between the third token sequence and the adversarial sample of the third token sequence, the second URL classification model is iteratively trained until the fourth loss function is minimized, wherein, , is the fourth loss function, is the total number of tokens, is the token sequence number of the third token sequence, is the first Tokens, For the said The adversarial example after adding perturbations to tokens, is the output result of the second URL classification model for the third token sequence, The second URL classification model is The output distribution of adversarial examples after adding perturbations to tokens.

[0055] Virtual adversarial training introduces regularization perturbations in the embedding space to generate adversarial samples corresponding to unlabeled sample URLs. In the virtual adversarial training process, FGSM is used as the data augmentation algorithm. In order to improve the generalization ability of the model when parsing slightly different unlabeled sample URLs, the KL divergence loss function is used as the fourth loss function to measure the difference between the output values ​​of the original data sample and the enhanced data sample after model inference.

[0056] The KL divergence loss function is used as the fourth loss function to calculate the gradient of the input data and add small perturbations in the direction of gradient ascent to generate adversarial samples. as follows: , Among them, the disturbance is Norm constraint.

[0057] Taking the example domain name “https: / / pilipalabiu.com” as an example, similar to the task of self-supervised contrastive learning, we first perturb the data of “https: / / pilipalabiu.com” to generate adversarial samples, such as ["https",": / / ","pilipalabi",".","com"], where “pilipalabi” is the result after adding the perturbation.

[0058] Based on the above three training methods, after completing one iteration of training in any training method, the total loss function of the current model is calculated, that is, based on the second loss function, the third loss function and the fourth loss function, the loss function of the third URL classification model is determined. Specifically, the second loss function, the third loss function and the fourth loss function can be weighted and summed, that is, ,in, is the loss function of the third URL classification model, b, c and d are the weights of the second loss function, the third loss function and the fourth loss function respectively.

[0059] When the total loss function of the model, that is, the loss function of the third URL classification model, is minimized, the iteration process is stopped. At this point, the obtained third URL classification model can effectively learn a feature representation that is more robust to input changes of unlabeled sample URLs.

[0060] Step 108: Obtain the first labeled sample URL and the true classification label of the first labeled sample URL, and output the encoding result of the first labeled sample URL through the third URL classification model.

[0061] Step 110: Based on the encoding result and the true classification mark, a convolutional neural network is trained to link the third URL classification model with the convolutional neural network to form a fourth URL classification model.

[0062] That is to say, after the third URL classification model is trained, the parameters of the third URL classification model are frozen, and based on the output result (predicted classification mark) of the third URL classification model and its original real classification mark, a convolutional neural network is trained and linked to the third URL classification model, so that the third URL classification model can be better adapted to specific downstream tasks. In other words, when performing specific URL classification tasks with different classification requirements such as malicious URL detection, advertising link identification, and web page classification, the output result of the current model can be adjusted using the labeled sample URL corresponding to the specific URL classification task. In other words, the current model can further learn the characteristics of the specific URL classification task through the labeled sample URL, so that the current model can adapt to the specific URL classification task and improve the accuracy of the current model in processing the specific URL classification task.

[0063] It should be noted that the number of labeled sample URLs used at this time is greatly reduced compared to the number of labeled samples required for directly training the URL classification model to handle specific URL classification tasks. In this way, by increasing the model's pre-learning of unlabeled sample URLs, it can reduce its demand for labeled samples required for specific URL classification tasks, thereby reducing the difficulty of manual labeling operations during model training.

[0064] The convolutional neural network is used to reflect the correlation between the encoding result of the first labeled sample URL and the predicted classification label.

[0065] Specifically, first, the convolutional neural network can output the predicted classification label corresponding to the encoding result.

[0066] Based on the above steps, the training dataset for the downstream task (i.e., the specific URL classification task) is determined as ,in, is the first labeled sample URL represented by word segmentation and tokenization encoding, yes The corresponding true classification label, for example, in the phishing URL detection task, Indicates that the URL is malicious. Indicates that the URL is benign. The goal of the model is to minimize the difference between the predicted classification label and the true classification label of the first labeled sample URL, which is achieved by minimizing the fifth loss function.

[0067] At this point, a convolutional neural network (CNN) is first constructed. The input of the convolutional neural network is the encoding result of the first labeled sample URL output by the third URL classification model. , , the input layer of the convolutional neural network receives Among them, convolutional neural networks are usually composed of convolutional layers, pooling layers and fully connected layers. Local features are extracted through convolution operations, the pooling layer performs downsampling to reduce the number of parameters, and the fully connected layer maps the features to a task-specific output space.

[0068] At this time, based on the real classification label and predicted classification label of the first labeled sample URL, the first AdamW optimizer is used to update the parameters of the convolutional neural network, and the convolutional neural network is iteratively trained until the fifth loss function of the convolutional neural network is minimized, wherein, , , is the fifth loss function, is the total number of tokens of the first marked sample URL, is the true classification label of the first labeled sample URL, is the predicted classification label of the first labeled sample URL, represents the parameters of the convolutional neural network in the t-th iteration training, is the learning rate of the first AdamW optimizer, for Regarding the gradient of the parameters of the convolutional neural network in the tth iteration training, Represents the parameters of the convolutional neural network in the t+1th iteration training.

[0069] In this way, the parameters of the convolutional neural network are optimized.

[0070] Step 112: Obtain a second labeled sample URL and a true classification label of the second labeled sample URL.

[0071] Step 114: fine-tune the model parameters of the fourth URL classification model based on the second labeled sample URL and the true classification label of the second labeled sample URL.

[0072] After optimizing the parameters of the convolutional neural network and linking it with the third URL classification model, a fourth URL classification model is obtained. It can be said that the fourth URL classification model is used to reflect the association between the second labeled sample URL and the predicted classification tag of the second labeled sample URL. At this time, the parameters of the original third URL classification model in the fourth URL classification model are unfrozen, and the fourth URL classification model is taken as a whole to optimize its parameters.

[0073] First, the predicted classification label corresponding to the second labeled sample URL is output through the fourth URL classification model. That is, the second labeled sample URL is input to the third URL classification model, and the feature representation of the second labeled sample URL is output through the third URL classification model. , and then the feature is expressed as As the input of the convolutional neural network, the convolutional neural network predicts the predicted classification tag corresponding to the second labeled sample URL .

[0074] Next, the cross entropy loss function is used to measure the difference between the predicted classification mark and the true classification mark of the entire fourth URL classification model, that is, based on the true classification mark and the predicted classification mark of the second labeled sample URL, the second AdamW optimizer is used to update the parameters of the fourth URL classification model, and iterative training is performed until the sixth loss function of the fourth URL classification model is minimized, wherein, , , is the sixth loss function, is the total number of tokens of the second marked sample URL, is the true classification label of the second labeled sample URL, is the predicted classification label of the second labeled sample URL, represents the parameters of the fourth URL classification model in the t-th iteration training, is the learning rate of the second AdamW optimizer, for Regarding the gradient of the parameters of the fourth URL classification model in the tth iteration training, Represents the parameters of the fourth URL classification model in the t+1th iteration training.

[0075] In a possible design, the first labeled sample URL and the second labeled sample URL are the same sample set.

[0076] In another possible design, the first labeled sample URL and the second labeled sample URL are different sample sets.

[0077] In one possible design, after using the first AdamW optimizer, its learning rate is adjusted to , and get the second AdamW optimizer.

[0078] At this point, this stage adjusts the parameters of the fourth URL classification model so that the fourth URL classification model can better adapt to the personalized needs of specific URL classification tasks and improve the classification performance of the model on specific URL classification tasks.

[0079] The above technical solution trains the URL classification model of the Transformer architecture through unlabeled sample URLs. In the first stage of the training process, a three-classification method is used to realize the model's learning ability for replacement token detection and scrambled token detection. In the second stage of the training process, a combination of mask learning, self-supervised contrast learning and virtual adversarial training is used to further train the model's learning and analysis capabilities for the features of unlabeled sample URLs. On the basis of this training, a convolutional neural network is linked to the model, and the ability of the convolutional neural network to handle specific URL classification tasks is optimized through labeled sample URLs. Then, the parameters of the model and the convolutional neural network are optimized as a whole. In this way, the number of labeled samples required for the training process of the URL classification model can be greatly reduced, the manual labeling work required for training can be reduced, the difficulty of model training can be reduced, and the problem of inaccurate training results caused by excessive manual participation can be avoided. At the same time, through the additional convolutional neural network, the classification ability of the URL classification model for specific URL classification tasks can still be guaranteed on the basis of greatly reducing the number of labeled samples, ensuring the smooth progress of specific URL classification tasks.

[0080] As shown in Table 1 below, URLBERT and URLBERT-E are two URL classification models trained by the above technical solutions. BERT, RoBERTa, XLNet, BiLSTM, TextRNN and TextCNN are existing URL classification models in related technologies. Table 1 shows the ACC (accuracy) and AUC (Area UnderCurve, area under the ROC curve) of the above models in different URL classification scenarios. Both are model quality evaluation parameters.

[0081] In the experiment, all models were optimized using the Adamw optimizer, and the datasets used for training and testing of all models were the same. The learning rate of each model structure was fine-tuned, with BiLSTM being 0.005, TextCNN and TextRNN being 0.002, and URLBERT, URLBERT-E, BERT, RoBERTa, and XLNet being 0.00005. In the experiment, the batch size of each model was standardized to 5 and evaluated after 5 training epochs.

[0082] Table 1

[0083] From Table 1, it can be seen that URLBERT and URLBERT-E of this application have shown clear advantages in various specific classification tasks.

[0084] Furthermore, in the phishing detection task, as shown in Table 2 below, the empirical performance of the URLBERT model is quantitatively evaluated as a function of the dataset size, where training is performed on datasets containing 30,000, 320,000, and 520,000 annotated examples, and corresponding test sets are made to measure the generation volume. In order to highlight the scale sensitivity of URLBERT, the model performance indicators of the three models at different training set sizes are experimented with TextCNN and BiLSTM models as benchmarks. Among them, based on the experiments involved in Table 1, a unified training configuration is still retained, and the time span is set to five times.

[0085] Table 2

[0086] Table 2 lists the performance indicators of the three models on different numbers of training sets. Among all the models, the performance improvement of the URLBERT model is the most significant. Figure 2 The diagram shows the ROC curve of the URLBERT model under different data scales. When the data scale is 3W or 30K, the AUC value of the URLBERT model is 0.9864; when the data scale is 32W or 320K, the AUC value of the URLBERT model is 0.9918; when the data scale is 52W or 520K, the AUC value of the URLBERT model is 0.9943. The ROC curve (Receiver Operating Characteristic Curve) is a curve with the false positive rate (FPR) as the horizontal axis and the true positive rate (TPR) as the vertical axis, which is used to evaluate the performance of the URLBERT model.

[0087] in addition, Figures 3 to 5 The changes in the core performance indicators of the URLBERT model, BiLSTM model, and TextCNN model in different training time periods in the phishing URL detection task are shown respectively. The core performance indicators described here include: Accuracy, Precision, and F1-Score. Figure 3It can be seen that the URLBERT model established stable high performance early in the training phase, and its accuracy peaked in the second training period, which highlights its robustness. All performance indicators of the BiLSTM model rose steadily during the training process, indicating that the learning ability of the BiLSTM model is highly dependent on the training data and requires more training cycles to reach a suitable performance level. In addition, the TextCNN model showed a completely different trend. Its accuracy and F1 score peaked in the second training cycle and then dropped significantly, indicating that there was overfitting in the training process of the TextCNN model, that is, the performance of the TextCNN model on the training data generalized to the test set to a certain extent, so it was necessary to stop early or use regularization techniques to maintain peak performance. In contrast, the URLBERT model performed the best.

[0088] In addition, the embodiment of the present application also provides a training device for a URL classification model, including: A word segmentation unit, configured to perform word segmentation processing on the unlabeled sample URL in response to a training instruction for the first URL classification model, to obtain a plurality of tokens of the unlabeled sample URL; A first unlabeled training unit is used to perform token replacement detection training and token scrambling detection training on the first URL classification model through multiple tokens of the unlabeled sample URL to obtain a second URL classification model; A second unlabeled training unit is used to perform mask learning, self-supervised contrastive learning and virtual adversarial training on the second URL classification model through the unlabeled sample URL to obtain a third URL classification model; A first acquisition unit is used to acquire a first labeled sample URL and a real classification label of the first labeled sample URL, and output an encoding result of the first labeled sample URL through the third URL classification model; A first labeled training unit, configured to train a convolutional neural network based on the encoding result and the true classification mark, so that the third URL classification model is linked to the convolutional neural network to form a fourth URL classification model, wherein the convolutional neural network is configured to reflect the association between the encoding result of the first labeled sample URL and the predicted classification mark; A second obtaining unit, configured to obtain a second labeled sample URL and a true classification label of the second labeled sample URL; The second labeled training unit is used to fine-tune the model parameters of the fourth URL classification model based on the second labeled sample URL and the real classification label of the second labeled sample URL, wherein the fourth URL classification model is used to reflect the association relationship between the second labeled sample URL and the predicted classification label of the second labeled sample URL.

[0089] In one embodiment of the present application, optionally, the device further includes: A word segmenter training unit, used for training a URL word segmenter based on a URL corpus set and a WordPiece algorithm before the word segmentation unit performs word segmentation processing on an unlabeled sample URL; The word segmentation unit is used to: map the characters of the unmarked sample URL into a word sequence through the URL word segmenter, and obtain the encoding representation information corresponding to each word in the word sequence to form a corresponding first token sequence.

[0090] In one embodiment of the present application, optionally, the word segmentation unit further includes: The token adding unit is used to add a CLS token before the first token of the first token sequence, and to add several PAD tokens after the last token of the first token sequence, so that the length of the first token sequence reaches a specified length.

[0091] In one embodiment of the present application, optionally, the first unlabeled training unit is specifically used to: shuffle and replace tokens of multiple tokens of the unlabeled sample URL to obtain a second token sequence, wherein the shuffle includes randomly selecting a first percentage of tokens from the multiple tokens to perform a shuffle operation, and the replacement token includes replacing a second percentage of tokens from the multiple tokens with random tokens; perform three-category processing on each token in the second token sequence through the first URL classification model, and iteratively train until the first loss function is minimized, wherein the three-category processing is used to select the token type of the token from the shuffled tokens, the randomly replaced tokens, and the original tokens, , is the first loss function, is the total number of tokens in the second token sequence, is the token number of the untagged sample URL, Indicates the first Tokens, is the serial number of the token type, is the first Token belongs to The probability of a token type, is the first The actual token type of the token.

[0092] In one embodiment of the present application, optionally, the second unlabeled training unit is used to: successively perform mask learning, self-supervised contrast learning and virtual adversarial training on the second URL classification model, or perform mask learning on the second URL classification model to obtain an intermediate model, and synchronously perform self-supervised contrast learning and virtual adversarial training on the intermediate model, or synchronously perform mask learning, self-supervised contrast learning and virtual adversarial training on the second URL classification model.

[0093] In one embodiment of the present application, optionally, the second unlabeled training unit includes: The mask learning unit is used to randomly replace a third percentage of tokens with masked tags among multiple tokens of the untagged sample URL to obtain a masked third token sequence, and output the predicted tokens of the masked tags in the third token sequence through the second URL classification model, and update the parameters of the second URL classification model based on the difference between the predicted token and the original real token corresponding to the masked tag, and iteratively train until the second loss function is minimized, wherein, , is the second loss function, is the total number of tokens in the third token sequence, Indicates the first The real token at the token position, Indicates that the second URL classification model The predicted results of the token probability distribution at the token position.

[0094] In one embodiment of the present application, optionally, the second unlabeled training unit includes: A self-supervised contrastive learning unit is used to obtain an enhanced data sample of the unlabeled sample URL by random inactivation and a fast gradient sign method, wherein the number of the enhanced data samples is twice the number of tokens in the unlabeled sample URL, the enhanced data sample includes a positive sample and multiple negative samples, and the distance between the positive samples and the distance between the negative samples in the unlabeled sample URL and the enhanced data sample is determined based on the second URL classification model, and iterative training is performed until a third loss function is minimized, wherein, , is the third loss function, is the total number of tokens of the unmarked sample URL, and They are respectively the first The augmented data samples, The augmented data samples and augmented data samples, sim() is the cosine similarity function, is the temperature coefficient, is the number of the enhanced data sample.

[0095] In one embodiment of the present application, optionally, the second unlabeled training unit includes: A virtual adversarial training unit is used to add perturbations in the gradient ascent direction of the third token sequence based on the fast gradient sign method to obtain an adversarial sample of the third token sequence, and iteratively train the second URL classification model based on the similarity between the third token sequence and the adversarial sample of the third token sequence until a fourth loss function is minimized, wherein, , is the fourth loss function, is the total number of tokens, is the token sequence number of the third token sequence, is the first Tokens, For the said The adversarial example after adding perturbations to tokens, is the output result of the second URL classification model for the third token sequence, The second URL classification model is The output distribution of adversarial examples after adding perturbations to tokens.

[0096] In one embodiment of the present application, optionally, the device further includes: A loss function weighted calculation unit is used to determine the loss function of the third URL classification model based on the second loss function, the third loss function and the fourth loss function, wherein: , is the loss function of the third URL classification model, b, c and d are the weights of the second loss function, the third loss function and the fourth loss function respectively.

[0097] In one embodiment of the present application, optionally, the first labeled training unit is specifically used to: output the predicted classification label corresponding to the encoding result through the convolutional neural network, and based on the real classification label and the predicted classification label of the first labeled sample URL, use the first AdamW optimizer to update the parameters of the convolutional neural network, and iteratively train until the fifth loss function of the convolutional neural network is minimized, wherein, , , is the fifth loss function, is the total number of tokens of the first marked sample URL, is the true classification label of the first labeled sample URL, is the predicted classification label of the first labeled sample URL, represents the parameters of the convolutional neural network in the t-th iteration training, is the learning rate of the first AdamW optimizer, for Regarding the gradient of the parameters of the convolutional neural network in the tth iteration training, Represents the parameters of the convolutional neural network in the t+1th iteration training.

[0098] In one embodiment of the present application, optionally, the second labeled training unit is specifically used to: output the predicted classification label corresponding to the second labeled sample URL through the fourth URL classification model, and based on the real classification label and the predicted classification label of the second labeled sample URL, use the second AdamW optimizer to update the parameters of the fourth URL classification model, and iteratively train until the sixth loss function of the fourth URL classification model is minimized, wherein, , , is the sixth loss function, is the total number of tokens of the second marked sample URL, is the true classification label of the second labeled sample URL, is the predicted classification label of the second labeled sample URL, represents the parameters of the fourth URL classification model in the t-th iteration training, is the learning rate of the second AdamW optimizer, for Regarding the gradient of the parameters of the fourth URL classification model in the tth iteration training, Represents the parameters of the fourth URL classification model in the t+1th iteration training.

[0099] The device uses any one of the solutions described in the above embodiments, and therefore has all the above technical effects, which will not be repeated here.

[0100] In addition, in one embodiment, the present application provides a computer device, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, the method described in any of the above embodiments can be implemented.

[0101] In one embodiment, the present application further provides a computer device, which may be a client, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, the method described in any of the above embodiments can be implemented.

[0102] Any of the above-mentioned computer devices in the embodiments of the present application may exist in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by their mobile communication functions and their main purpose is to provide voice and data communications. These terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.

[0103] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, such as iPad.

[0104] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys, wearable devices, and portable car navigation devices.

[0105] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0106] (5) Other electronic devices with data interaction functions.

[0107] In addition, an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to perform the following steps: In response to the training instruction for the first URL classification model, performing word segmentation processing on the unlabeled sample URL to obtain multiple tokens of the unlabeled sample URL; Performing token replacement detection training and token scrambling detection training on the first URL classification model through multiple tokens of the unmarked sample URL to obtain a second URL classification model; Using the unlabeled sample URL, performing mask learning, self-supervised contrastive learning and virtual adversarial training on the second URL classification model to obtain a third URL classification model; Obtaining a first labeled sample URL and a true classification label of the first labeled sample URL, and outputting an encoding result of the first labeled sample URL through the third URL classification model; Based on the encoding result and the real classification mark, training a convolutional neural network so that the third URL classification model and the convolutional neural network are linked to form a fourth URL classification model, wherein the convolutional neural network is used to reflect the association relationship between the encoding result of the first labeled sample URL and the predicted classification mark; Obtain a second labeled sample URL and a true classification label of the second labeled sample URL; Based on the second labeled sample URL and the true classification mark of the second labeled sample URL, the model parameters of the fourth URL classification model are fine-tuned, wherein the fourth URL classification model is used to reflect the association between the second labeled sample URL and the predicted classification mark of the second labeled sample URL.

[0108] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant description in the aforementioned method embodiment. In order to avoid repetition, they will not be described one by one here.

[0109] The technical solution of the present application is described in detail above in conjunction with the accompanying drawings. The technical solution of the present application is to train the URL classification model of the Transformer architecture through unlabeled sample URLs. In the first stage of the training process, a three-classification method is used to realize the model's learning ability for replacement token detection and scrambled token detection. In the second stage of the training process, a combination of mask learning, self-supervised contrast learning and virtual adversarial training is used to further train the model's learning and analysis capabilities for the features of unlabeled sample URLs. On the basis of this training, a convolutional neural network is linked to the model, and the ability of the convolutional neural network to handle specific URL classification tasks is optimized through labeled sample URLs, and then the parameters of the model and the convolutional neural network are optimized as a whole. In this way, the number of labeled samples required for the training process of the URL classification model can be greatly reduced, the manual labeling work required for training can be reduced, the difficulty of model training can be reduced, and the problem of inaccurate training results caused by too much manual participation can also be avoided. At the same time, through the additional convolutional neural network, on the basis of greatly reducing the number of labeled samples, the classification ability of the URL classification model for specific URL classification tasks can still be guaranteed, ensuring the smooth progress of specific URL classification tasks.

[0110] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0111] It should be understood that although the terms first, second, etc. may be used to describe URL classification models in the embodiments of the present application, these URL classification models should not be limited to these terms. These terms are only used to distinguish URL classification models from each other. For example, without departing from the scope of the embodiments of the present application, the first URL classification model may also be referred to as the second URL classification model, and similarly, the second URL classification model may also be referred to as the first URL classification model.

[0112] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0113] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0114] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0115] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0116] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0117] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for training a URL classification model, characterized in that: include: In response to the training instruction for the first URL classification model, performing word segmentation processing on the unlabeled sample URL to obtain multiple tokens of the unlabeled sample URL; Performing token replacement detection training and token scrambling detection training on the first URL classification model through multiple tokens of the unmarked sample URL to obtain a second URL classification model; Using the unlabeled sample URL, performing mask learning, self-supervised contrastive learning and virtual adversarial training on the second URL classification model to obtain a third URL classification model; Obtaining a first labeled sample URL and a true classification label of the first labeled sample URL, and outputting an encoding result of the first labeled sample URL through the third URL classification model; Based on the encoding result and the real classification mark, training a convolutional neural network so that the third URL classification model and the convolutional neural network are linked to form a fourth URL classification model, wherein the convolutional neural network is used to reflect the association relationship between the encoding result of the first labeled sample URL and the predicted classification mark; Obtain a second labeled sample URL and a true classification label of the second labeled sample URL; Based on the second labeled sample URL and the true classification mark of the second labeled sample URL, the model parameters of the fourth URL classification model are fine-tuned, wherein the fourth URL classification model is used to reflect the association between the second labeled sample URL and the predicted classification mark of the second labeled sample URL.

2. The method according to claim 1, characterized in that Before the word segmentation process of the unmarked sample URL is performed, the following steps are included: Train the URL word segmenter based on the URL corpus and the WordPiece algorithm; The word segmentation processing of the unmarked sample URL includes: The characters of the unmarked sample URL are mapped into a word sequence through the URL tokenizer, and the encoding representation information corresponding to each word in the word sequence is obtained to form a corresponding first token sequence.

3. The method according to claim 2, characterized in that Also includes: Add a CLS token before the first token of the first token sequence; as well as A number of PAD tokens are added after the last token of the first token sequence so that the length of the first token sequence reaches a specified length.

4. The method according to claim 1, characterized in that: The step of performing token replacement detection training and token disruption detection training on the first URL classification model using multiple tokens of the unlabeled sample URL includes: Performing scrambling and token replacement processing on multiple tokens of the unmarked sample URL to obtain a second token sequence, wherein the scrambling processing includes randomly selecting a first percentage of tokens from the multiple tokens to perform a scrambling operation, and the token replacement processing includes replacing a second percentage of tokens from the multiple tokens with random tokens; Each token in the second token sequence is subjected to a three-classification process by the first URL classification model, and iterative training is performed until the first loss function is minimized, wherein the three-classification process is used to select the token type of the token from the shuffled token, the randomly replaced token, and the original token, wherein, ， is the first loss function, is the total number of tokens in the second token sequence, is the token number of the untagged sample URL, Indicates the first Tokens, is the serial number of the token type, is the first Token belongs to The probability of a token type, is the first The actual token type of the token.

5. The method according to claim 4, characterized in that The performing mask learning, self-supervised contrastive learning and virtual adversarial training on the second URL classification model includes: sequentially performing mask learning, self-supervised contrastive learning, and virtual adversarial training on the second URL classification model; or Performing mask learning on the second URL classification model to obtain an intermediate model, and simultaneously performing self-supervised contrastive learning and virtual adversarial training on the intermediate model; or Mask learning, self-supervised contrastive learning and virtual adversarial training are performed synchronously on the second URL classification model.

6. The method according to claim 1, characterized in that The step of performing mask learning on the second URL classification model includes: Randomly replacing a third percentage of tokens in the unmarked sample URL with masked tokens to obtain a masked third token sequence; The predicted token of the mask mark in the third token sequence is output by the second URL classification model, and the parameters of the second URL classification model are updated based on the difference between the predicted token and the original real token corresponding to the mask mark, and the training is iterated until the second loss function is minimized, wherein, ， is the second loss function, is the total number of tokens in the third token sequence, Indicates the first The real token at the token position, Indicates that the second URL classification model The predicted results of the token probability distribution at the token position.

7. The method according to claim 6, characterized in that The step of performing self-supervised contrastive learning on the second URL classification model includes: Obtaining enhanced data samples of the unlabeled sample URL by random inactivation and fast gradient sign method, wherein the number of the enhanced data samples is twice the number of tokens in the unlabeled sample URL, and the enhanced data samples include a positive sample and multiple negative samples; Based on the second URL classification model, the distance between the positive samples and the distance between the negative samples in the unlabeled sample URL and the enhanced data sample are determined, and the training is iterated until the third loss function is minimized, wherein, ， is the third loss function, is the total number of tokens of the untagged sample URL, and They are respectively the first The augmented data samples, The augmented data samples and augmented data samples, sim() is the cosine similarity function, is the temperature coefficient, is the number of the enhanced data sample.

8. The method according to claim 7, characterized in that The step of performing virtual adversarial training on the second URL classification model includes: Based on the fast gradient sign method, adding disturbance in the gradient ascent direction of the third token sequence to obtain an adversarial sample of the third token sequence; Based on the similarity between the third token sequence and the adversarial sample of the third token sequence, the second URL classification model is iteratively trained until a fourth loss function is minimized, wherein, ， is the fourth loss function, is the total number of tokens, is the token sequence number of the third token sequence, is the first Tokens, For the The adversarial example after adding perturbations to tokens, is the output result of the second URL classification model for the third token sequence, The second URL classification model is The output distribution of adversarial examples after adding perturbations to tokens.

9. The method according to claim 8, characterized in that Also includes: Based on the second loss function, the third loss function and the fourth loss function, determining the loss function of the third URL classification model, in, ， is the loss function of the third URL classification model, b, c and d are the weights of the second loss function, the third loss function and the fourth loss function respectively.

10. The method according to any one of claims 1 to 9, characterized in that The step of training a convolutional neural network based on the encoding result and the true classification label comprises: Outputting a predicted classification label corresponding to the encoding result through the convolutional neural network; Based on the true classification label and the predicted classification label of the first labeled sample URL, the parameters of the convolutional neural network are updated using a first AdamW optimizer, and the convolutional neural network is iteratively trained until the fifth loss function of the convolutional neural network is minimized, wherein: ， ， is the fifth loss function, is the total number of tokens of the first marked sample URL, is the true classification label of the first labeled sample URL, is the predicted classification label of the first labeled sample URL, Indicates The parameters of the convolutional neural network in the training iteration, is the learning rate of the first AdamW optimizer, for The parameters of the convolutional neural network are The gradient in the training iteration, Represents the parameters of the convolutional neural network in the t+1th iteration training.

11. The method according to claim 10, characterized in that The fine-tuning of the model parameters of the fourth URL classification model based on the second labeled sample URL and the real classification label of the second labeled sample URL includes: Outputting a predicted classification label corresponding to the second labeled sample URL through the fourth URL classification model; Based on the real classification label and predicted classification label of the second labeled sample URL, the parameters of the fourth URL classification model are updated using the second AdamW optimizer, and the training is iterated until the sixth loss function of the fourth URL classification model is minimized, wherein, ， ， is the sixth loss function, is the total number of tokens of the second marked sample URL, is the true classification label of the second labeled sample URL, is the predicted classification label of the second labeled sample URL, Indicates The parameters of the fourth URL classification model in the iteration training, is the learning rate of the second AdamW optimizer, for Regarding the gradient of the parameters of the fourth URL classification model in the tth iteration training, Represents the parameters of the fourth URL classification model in the t+1th iteration training.

12. A computer device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are configured to enable the processor to execute the method described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that: Computer executable instructions are stored, and the computer executable instructions are configured to execute the method according to any one of claims 1 to 11.

Images