Multi-modal phishing website detection method based on large model

Abstract

The invention discloses a multi-modal phishing website detection method based on a large model, and the method comprises the steps: extracting a domain name text of a URL address of a target website and a webpage screenshot corresponding to the URL address through a server, and then carrying out the preprocessing of the obtained domain name text and webpage screenshot, and obtaining a text initial feature and a picture initial feature; text features and picture features are obtained through processing of a neural network deep learning model CLIP; calculating the similarity between the obtained text features and the picture features to obtain an image-text similarity matrix, and normalizing the image-text similarity matrix to obtain an image-text probability matrix; generating a decision set and an extension set from the image-text probability matrix through an analysis selector, and finally judging whether the target website belongs to the phishing website or not by analyzing the decision set and the extension set.

Description

Technical Field

[0001] The present invention belongs to network security technology, and in particular relates to a multimodal phishing website detection method based on a large model. Background Art

[0002] Nowadays, with the rapid development of network technology, more and more online trading platforms, online shopping platforms, banking services, etc. are deployed on the Internet, which enriches daily life and provides people with great convenience. However, in the current network environment, there are attackers who use various attack methods to steal users' sensitive information, causing great losses to users' property. Stealing sensitive information through phishing web pages and phishing emails is the most common method. Although with the continuous development of phishing detection technology, the attackers' attack methods are also constantly updated and changed. In the face of such a phishing threat, effective countermeasures are urgently needed.

[0003] In order to reduce the threat posed by phishing to the public, the academic community has proposed many phishing website detection technologies, which are mainly divided into four categories: blacklist technology, heuristic rule algorithms, machine learning methods and deep learning technology. However, these methods all have certain limitations and defects.

[0004] Blacklist-based detection technology

[0005] A blacklist is a list of phishing website URLs. It detects whether the input URL is in the list to identify whether it is a phishing website. Currently, many browsers have the function of detecting blacklists to protect users from phishing attacks and avoid losses. Blacklist detection is very efficient because it does not require analysis of the specific content of the web page. However, it is limited by the need for frequent maintenance and cannot detect zero-day phishing websites, that is, it cannot respond to newly emerging malicious URLs immediately.

[0006] Detection technology based on heuristic rule algorithm

[0007] Heuristic algorithms are a technology that does not rely on predefined blacklists, but instead predicts and identifies phishing websites by analyzing the characteristics and patterns of the website. These characteristics may include but are not limited to the structure of the website URL, page content, domain name registration information, and similarities with known phishing websites. The core of the heuristic algorithm is to identify potential phishing websites through these characteristics, even if they have not yet been added to the blacklist. Heuristic algorithms are usually combined with machine learning to detect phishing web pages. However, heuristic rule algorithms can be deliberately circumvented by attackers by obtaining heuristic algorithms for analysis.

[0008] Machine learning-based detection technology

[0009] The machine learning detection method is to learn from the characteristics of a website and predict whether it is a phishing website. However, machine learning is limited by the training data set. The quality of the training set directly affects the effect of the model itself. In addition, the generalization ability of machine learning is poor, and it cannot be applied to zero-day phishing detection.

[0010] Detection technology based on deep learning

[0011] Deep learning can detect phishing websites better than other methods due to its black box mechanism of neural network, and it makes it impossible for attackers to evade by finding its patterns. However, deep learning has high requirements for platform training hardware. At present, the phishing website detection model based on visual information includes visual similarity, which uses the degree of visual similarity between web pages and legitimate web pages as the basis for its detection. This type of method is generally called single modality. This type of method causes the model to be heavily dependent on the training data set. There are also deep models that combine images and texts. However, this type of model lacks semantic information between text and images, resulting in poor results for zero-day phishing detection, and no recognition ability for phishing websites that do not exist in the training data set. Therefore, there is an urgent need for a deep model with low training pressure and high accuracy, combined with visual text semantic multimodality, and a model that also has a certain recognition ability for websites that do not exist in the training set data. Summary of the invention

[0012] Purpose of the invention: The purpose of the present invention is to address the deficiencies in the prior art and to provide a multimodal phishing website detection method based on a large model. Based on the multimodal deep learning large model method and the large model's zero-sample reasoning capability, the URL address and the web page screenshot are used as multimodal features to detect whether the original website is a phishing website, thereby greatly improving the accuracy and speed of detecting phishing websites and being able to perform zero-day phishing detection.

[0013] Technical solution: A multimodal phishing website detection method based on a large model of the present invention comprises the following steps:

[0014] Step 1: First extract the URL address of the target website that the user wants to visit and send it to the server for processing; the URL address includes the URL protocol, subdomain, domain name and domain name suffix, port number, and resource path;

[0015] Step 2: The server extracts the domain name of the URL address as the original text (row) information, and uses the automated tool selenium to capture the webpage screenshot corresponding to the URL address as the original image (row) information. The original text information and the original image information are then preprocessed by their respective methods to obtain the text initial feature Token and the image initial feature Image.

[0016] Step 3: For the obtained text initial feature Token and image initial feature Image, the server processes them through the Text Transformer layer and VIT (Vision Transformer) layer of the neural network deep learning model CLIP to obtain the text feature T and image feature I respectively;

[0017] Step 4: The server calculates the similarity between the text feature T and the image feature I to obtain the image-text similarity matrix S, and then uses the normalized exponential function (Softmax Normalization) to process the image-text similarity matrix to obtain the image-text probability matrix P;

[0018] Step 5: The graph-text probability matrix is ​​analyzed by the selector to generate a decision set and an expansion set, and by analyzing the decision set and the expansion set, it is determined whether the target website is a phishing website;

[0019] Step 6: Return the judgment result to the browser. If it is a phishing website, a warning dialog box will pop up to prompt the user. If it is a legitimate website, no operation will be performed and the user can access it directly.

[0020] The above step 1 simultaneously intercepts the URL of the target website that the user is about to browse based on the user inputting the URL address into the browser and pressing Enter or entering the web page through a hyperlink. Web page hyperlinks, usually referred to as links, are elements used in the World Wide Web to jump from one web page to another web page or a specific part of a web page. Hyperlinks are one of the basic features of the Internet, which allow users to access information on other pages or sites by clicking on link text or images. The main types of hyperlinks include: Internal links: links to other pages within the same website. External links: links to pages on different websites. Anchor links: links to specific parts of the same page, usually used for navigation of long pages.

[0021] Furthermore, the detailed method for obtaining the initial features of the text in step 2 is:

[0022] A) Construct an expandable brand category list StringDB; after storing each brand category in StringDB, use the position of each brand category in StringDB as its label Label. There are n strings in the brand category list StringDB; for example, StringDB: [adidas, absa, adobe.....ziggo]. Assuming that the number of strings in this StringDB is 155, then n is 155, and the domain name position of the target website to be detected is n+1.

[0023] B) Preprocess the URL address text. The process is as follows:

[0024] First, obtain the domain name string from the target website URL U and use it as the detection category of the target website; for example, for the URL "https: / / www.alibaba.com", remove the domain name suffix and subdomain name based on the domain of the website through regular expression to obtain the final domain name 'alibaba';

[0025] Then, the detected category string of the target website is merged with the category string of StringDB to form an extended category string list L, that is, L = {L1, L2, L3...L n ,L n+1};

[0026] Among them, L i is the i-th category string stored in StringDB, i = 1, 2, ..., n, L n+1 The target website domain name (category) string obtained from the URL;

[0027] Next, the obtained extended category string list L is combined with the prompt word P to generate a string list with prompt words. Accurate prompt words can make the model recognition more accurate. Assuming that the prompt word used is P, the new string list L with prompt words formed by combining L and P is new for:

[0028] L new ={[P]L1,[P]L2,…,[P]L n ,[P]L n+1};

[0029] Finally, for the string list L new Perform tokenization and convert it into a tokenized form, i.e., the initial feature Token of the text: Token = {Token1, Token2, ..., Token n ,Token n+1};

[0030] Among them, Token i For [P]L i ∈L new The result after word segmentation processing is i=1,2,…,n,n+1.

[0031] Furthermore, the detailed method of processing the original (row) information of the picture in step 2 to obtain the initial features of the picture is:

[0032] The original (row) information of the image is segmented, cropped, and padded in turn, and the entire large image is cut into several sub-images; assuming that N*M cutting is used, the final image is segmented into N*M sub-images, and the initial feature Image of the preprocessed image is obtained as follows:

[0033] Image={Image1,Image2,Image3,Image4...Image N*M}.

[0034] Furthermore, the specific method of calculating the text features and the image features in step 3 is:

[0035] The text encoding layer Text_Encoder of the CLIP model encodes and extracts the initial feature Token text obtained in step 2, and then generates a text feature vector list T of the website brand category to obtain the text features:

[0036] T = Text_Encoder (Token)

[0037] T={T1,T2,T3...T n ,T n+1}

[0038] The text encoding layer Text_Encoder adopts the Text Transformer architecture;

[0039] The image encoding layer Image_Encoder of the CLIP model encodes and extracts the initial features Image obtained in step 2, and then maps the Image to a higher-dimensional space to obtain the image features;

[0040] I = Image_Encoder(Image)

[0041] I={I1,I2,I3...I N*M}

[0042] The picture encoding layer adopts the Vision Transformer architecture.

[0043] Furthermore, the core of the Text Transformer architecture and the Vision Transformer architecture both adopt the self-attention mechanism. For a certain input original feature X, the calculation formula of multi-head self-attention is:

[0044] First, the original feature X is linearly transformed to obtain query (Query, Q), key (Key, K), value (Value, V), where W Q,W K ,W V is the weight matrix of the model, which is learned during the training process: Q = XW Q ,K=XW K ,V=XW V ;

[0045] Then, we calculate the dot product of Q and K to get the attention score and divide it by the scaling factor where d k is the dimension of the key K, and then the score is normalized by Softmax and multiplied by the value V to get the Attention score, where the Attention formula is as follows:

[0046] Next, Multi-Head Attention replicates the self-attention mechanism multiple times (heads), each head calculates the attention of a different representation subspace, concatenates the results and passes another linear transformation to get the final output:

[0047] MultiHead(Q,K,V)=Concat(head1,...,head h )W0

[0048] The expression of each head is: head i =Attention(QW i Q ,KW i K ,VW i V );

[0049] Finally, the feed-forward network (FFN) and residual connection, as well as layer normalization LayerNorm are performed to obtain the dimensional features:

[0050] FFN(x)=LayerNorm(max(0,xW1+b1)W2+b2)

[0051] The final output feature of vector X is:

[0052] X1=LayerNorm(X+MultiHead(X))

[0053] X2=LayerNorm(X1+FFN(X1))

[0054] That is, the original feature X is transformed into feature X2 after the transformer architecture.

[0055] Furthermore, the detailed process of step 4 is as follows:

[0056] Step 4.1: Each vector T in the text feature T j And each vector I in the image feature I i Perform dot product operations to obtain the similarity matrix S between each sub-graph and the category prompt text:

[0057] S={S1,S2,…,S N×M};

[0058] Among them, S i = {I i ●T1,I i ●T2,……,I i ●T n ,I i ●T n+1},i=1,2,…,N×M;j=1,2,…,n+1;T={T1,T2,T3…T n ,T n+1}; I = {I1, I2, I3...I N*M};

[0059] Step 4.2, the similarity matrix S is normalized by the feature normalization exponential function Softmax Normalization to obtain the final probability matrix P;

[0060] The similarity matrix S is normalized by the feature normalization exponential function to obtain the image-text probability matrix P, which describes the probability that each sub-image of the target image belongs to the corresponding category;

[0061] P={P1,P2,…,P N×M} T

[0062] in, i=1,2,...,N×M, that is

[0063]

[0064] in, τ is the temperature coefficient, is the element in the similarity matrix S that represents the similarity of subgraph i belonging to category k, P i j It represents the normalized probability value when the model predicts the i-th sub-graph as category j. There are a total of N×M×(n+1) elements in the graph-text probability matrix P.

[0065] Furthermore, the detailed method of step 5 is:

[0066] Step 5.1, generate a subgraph category probability decision set DecisionSet, and use the decision set DecisionSet to provide a basis for judging the legal websites of unknown categories;

[0067] For a legitimate website, if its category is unknown, that is, the category represented by the domain name in the URL of the website is not stored in StringDB, then the probability of predicting the subgraph i containing key information as the category to which the URL domain name belongs (i.e., n+1) will be significantly higher than other categories stored in StringDB; the method for generating the subgraph category probability decision set DecisionSet is:

[0068] Set the highest prediction probability set MaxProbSet to save the maximum probability value of each sub-graph i belonging to different categories Among them, for subgraph i (i = 1, 2, ..., N × M), the category number max to which the maximum probability value belongs is i (max i =1,2,…,n,n+1) is found as follows:

[0069] By traversing each row of the graph-text probability matrix P through the above formula, the maximum value of the model's predicted probability of the corresponding subgraph belonging to each category is taken to complete the selection of the highest predicted probability value of the category to which the subgraph belongs;

[0070] The MaxProbSet obtained by the above process is expressed as:

[0071]

[0072] The probability threshold ρ1 is set based on the obtained highest prediction probability set MaxProbSet, and the elements with sub-graph category probability values ​​less than ρ1 in MaxProbSet are deleted to complete the construction of the sub-graph category decision probability set DecisionSet;

[0073] Step 5.2, generate the sub-graph category probability expansion set ExpandSet;

[0074] For a legitimate website, if its category is known, that is, the category represented by the domain name in the URL of the website has been stored in StringDB and the category is k, then the probability of predicting subgraph i containing the key information of the website as category k is The probability that the model predicts the category to which the URL domain belongs is Since the predicted probability of the known class k The partial probability value of the prediction of the unknown category n+1 is amortized, then and Will not exceed max iThe threshold ρ1 is preset to find the maximum probability value;

[0075] The specific method of constructing the subgraph category probability decision expansion set ExpandSet is:

[0076] By setting the sub-image category probability threshold ρ2, we traverse each sub-image of the target image, that is, each row of the matrix P; during the traversal process, if the category of a sub-image i is predicted to be the probability of an existing category k in StringDB and If they are close and both are in the interval [ρ2,1], then the elements representing the probability information of the two categories are stored in the ExpandSet set;

[0077] The setting of ρ2 depends on the value of ρ1, that is, in normal cases 2ρ2 should be close to ρ1;

[0078] Step 5.3: Generate a phishing detection report;

[0079] The subgraph category probability decision set DecisionSet and the subgraph category probability expansion set ExpandSet are combined to generate a detection report reflecting whether the target website is a phishing website.

[0080] Furthermore, the phishing detection report in step 5.3 includes the following situations:

[0081] Case (1), DecisionSet is not empty and there is a subgraph prediction probability of category n+1, that is, there is Such elements; since the values ​​of these elements are all greater than ρ1, it is concluded that (a) the model's prediction of the category to which the relevant subgraph belongs is highly consistent with the category described by the input URL domain name; (b) the category of the input URL does not share probability with the website category in StringDB, and therefore it is not a website in StringDB;

[0082] At this point, the final detection report is: the target website is a legitimate website of unknown category;

[0083] Case (2), ExpandSet is not empty and there is a subgraph prediction probability of category n+1, that is, there is This type of paired elements; since the values ​​of these elements are all greater than ρ2, it can be concluded that (a) the model's prediction of the category to which the relevant subgraph belongs is consistent with the category described by the input URL domain name; (b) the category of the input URL shares the probability with the website category in StringDB, that is, the category of the URL already exists in StringDB;

[0084] At this point, the final detection report is: the target website is a legitimate website of a known category;

[0085] In case (3), at least one of DecisionSet and ExpandSet is not empty, and there is no subgraph prediction probability of category n+1, which means that the webpage has at least one key information and the key information is irrelevant to the webpage domain name. Since the values ​​of these elements are all greater than ρ1 or ρ2, it can be judged that (a) the category they predict is not the category described by the input URL domain name; (b) the target website has at least one key information that is intended to imitate the key information of the legitimate website;

[0086] At this point, the final detection report is: the target website is a phishing website that imitates the key information of a legitimate website;

[0087] In case (4), both DecisionSet and ExpandSet are empty, which indicates that there is no obvious logo or key text description in the webpage. For phishing attackers, webpages without key information will not serve the purpose of imitating phishing websites, that is, phishing attackers will not imitate such webpages without key information. Based on the above analysis, such webpages without key information are directly considered to be legitimate.

[0088] At this point, the final test report is: a legitimate website without any critical information.

[0089] Beneficial effects: The present invention introduces the large model CLIP into the field of phishing detection for the first time, and uses the advantages of the large model to solve the problem that other classification models cannot detect phishing websites of untrained categories, and solves the problem of lack of semantic correlation between images and texts compared to other models, and realizes efficient phishing detection. The present invention improves the preprocessing stage. In order to prevent the loss of overall information of the image, we use cropping, segmentation, and padding methods to ensure that the information of the image is not missing. Make up for the problem that key features of large-size web page images will be seriously lost after being compressed. The present invention analyzes the characteristics of web page screenshots, and therefore analyzes the website by adding an analysis selector, and finally generates a decision set and an extension set to determine whether it is a legitimate or phishing website, and our model has a high phishing detection function for zero-day phishing websites. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 It is the overall flow chart of the present invention;

[0091] Figure 2 It is the basic composition structure diagram of the URL of the present invention;

[0092] Figure 3 Schematic diagram of the text preprocessing process of the present invention;

[0093] Figure 4 Schematic diagram of the image preprocessing process in the present invention;

[0094] Figure 5 It is a schematic diagram of the overall detection framework of the server in the present invention;

[0095] Figure 6 This is the process of extracting features using the attention mechanism in the present invention. DETAILED DESCRIPTION

[0096] The technical solution of the present invention is described in detail below, but the protection scope of the present invention is not limited to the embodiments.

[0097] like Figure 1 and Figure 5 As shown, the multimodal phishing website detection method based on a large model of the present invention comprises the following steps:

[0098] Step 1: First extract the URL address of the target website that the user wants to visit and send it to the server for processing; the URL address includes the URL protocol, subdomain, domain name and domain name suffix, port number, and resource path;

[0099] Step 2: The server extracts the domain name of the URL address as the original text (row) information, and uses the automated tool selenium to capture the webpage screenshot corresponding to the URL address as the original image (row) information. The original text information and the original image information are then preprocessed by their respective methods to obtain the text initial feature Token and the image initial feature Image.

[0100] Step 3: For the obtained text initial feature Token and image initial feature Image, the server processes them through the Text Transformer layer and VIT (Vision Transformer) layer of the neural network deep learning model CLIP to obtain the text feature T and image feature I respectively;

[0101] Step 4: The server calculates the similarity between the text feature T and the image feature I to obtain the image-text similarity matrix S, and then uses the normalized exponential function (Softmax Normalization) to process the image-text similarity matrix to obtain the image-text probability matrix P;

[0102] Step 5: The graph-text probability matrix is ​​analyzed by the selector to generate a decision set and an expansion set, and by analyzing the decision set and the expansion set, it is determined whether the target website is a phishing website;

[0103] Step 6: Return the judgment result to the browser. If it is a phishing website, a warning dialog box will pop up to prompt the user. If it is a legitimate website, no operation will be performed and the user can access it directly.

[0104] The URL address of the present invention is composed of four key components: Figure 2 : First is the URL protocol, such as http or ftp, which defines the way to access resources; second is the subdomain, domain name and domain suffix, which are used to identify a specific host in the network; then comes the port number: used to specify access.

[0105] The present invention applies the large model CLIP of multimodal zero-sample reasoning of text and images to the field of identification of phishing pages. With the help of the high inference accuracy of the CLIP model in the zero-sample case, the fine-tuning of parameters in the migration process does not require large-scale training samples, and through the construction of a maintainable brand category list (the name of the website brand is used as its category in this paper), the research on page image segmentation and sub-image probability matrix generation and selection, the accurate classification and identification of phishing pages in the case of small samples is achieved. Through comparative learning, the model can learn the correlation between images and texts, so as to identify matching images given a text description, or find matching text descriptions given an image.

[0106] The CLIP model in the present invention is pre-trained on a large-scale image and text pair dataset, using two sub-networks, an image encoder and a text encoder, to extract feature representations of images and texts respectively, and then by contrasting the learned objective function, the model is able to identify matching image-text pairs and distinguish unmatched pairs.

[0107] The detailed method for obtaining the initial features of the text in step 2 of this embodiment is:

[0108] A) Construct an extensible brand category list StringDB; after storing each brand category in StringDB, use the position of each brand category in StringDB as its label Label. There are n strings in the brand category list StringDB;

[0109] B) Preprocess the URL address text, such as Figure 3 The process is:

[0110] First, obtain the domain name string from the target website URL U and use it as the detection category of the target website ；

[0111] Then, the detected category string of the target website is merged with the category string of StringDB to form an extended category string list L, that is, L = {L1, L2, L3...L n ,L n+1};

[0112] Among them, L iis the i-th category string stored in StringDB, i = 1, 2, ..., n, L n+1 The target website domain name (category) string obtained from the URL;

[0113] Next, the obtained extended category string list L is combined with the prompt word P to generate a string list with prompt words; assuming that the prompt word used is P, the new string list with prompt words formed by combining L and P is L new for:

[0114] L new ={[P]L 1 ,[P]L2,…,[P]L n ,[P]L n+1};

[0115] Finally, for the string list L new Perform tokenization and convert it into a tokenized form, i.e., the initial feature Token of the text: Token = {Token1, Token2, ..., Token n ,Token n+1};

[0116] Among them, Token i For [P]L i ∈L new The result after word segmentation processing is i=1,2,…,n,n+1.

[0117] The detailed method of processing the original (row) information of the picture to obtain the initial features of the picture in step 2 of this embodiment is:

[0118] The original (row) information of the image is segmented, cropped, and padded in turn, and the entire large image is cut into several sub-images; assuming that N*M cutting is used, the final image is divided into N*M sub-images, such as Figure 4 The final initial feature Image of the preprocessed image is as follows:

[0119] Image={Image1,Image2,Image3,Image4...Image N*M}.

[0120] The specific method of calculating text features and image features in step 3 of this embodiment is:

[0121] The text encoding layer Text_Encoder of the CLIP model encodes and extracts the initial feature Token text obtained in step 2, and then generates a text feature vector list T of the website brand category to obtain the text features:

[0122] T = Text_Encoder (Token)

[0123] T={T1,T2,T3...T n ,T n+1}

[0124] The text encoding layer Text_Encoder adopts the Text Transformer architecture;

[0125] The image encoding layer Image_Encoder of the CLIP model encodes and extracts the initial features Image obtained in step 2, and then maps the Image to a higher-dimensional space to obtain the image features;

[0126] I = Image_Encoder(Image)

[0127] I={I1,I2,I3...I N*M}

[0128] The picture encoding layer adopts the Vision Transformer architecture.

[0129] like Figure 6 As shown, the core of the Text Transformer architecture and the Vision Transformer architecture described in this embodiment both adopt the self-attention mechanism. For a certain input original feature X, the calculation formula of the multi-head self-attention is:

[0130] First, the original feature X is linearly transformed to obtain query (Query, Q), key (Key, K), value (Value, V), where W Q ,W K ,W V is the weight matrix of the model, which is learned during the training process: Q = XW Q ,K=XW K ,V=XW V

[0131] Then, we calculate the dot product of Q and K to get the attention score and divide it by the scaling factor where d k is the dimension of the key K, and then the score is normalized by Softmax and multiplied by the value V to get the Attention score, where the Attention formula is as follows:

[0132] Next, Multi-Head Attention replicates the self-attention mechanism multiple times (heads), each head calculates the attention of a different representation subspace, concatenates the results and passes another linear transformation to get the final output:

[0133] MultiHead(Q,K,V)=Concat(head1,...,head h )W0

[0134] The expression of each head is: head i =Attention(QW i Q ,KW i K ,VW i V );

[0135] Finally, the feed-forward network (FFN) and residual connection, as well as layer normalization LayerNorm are performed to obtain the dimensional features:

[0136] FFN(x)=LayerNorm(max(0,xW1+b1)W2+b2)

[0137] The final output feature of vector X is:

[0138] X1=LayerNorm(X+MultiHead(X))

[0139] X2=LayerNorm(X1+FFN(X1))

[0140] That is, the original feature X is transformed into feature X2 after the transformer architecture.

[0141] The detailed process of step 4 above is:

[0142] Step 4.1: Each vector T in the text feature T j And each vector I in the image feature I i Perform dot product operations to obtain the similarity matrix S between each sub-graph and the category prompt text:

[0143] S={S 1 ,S2,…,S N×M};

[0144] Among them, S i = {I i ●T1,I i ●T2,……,I i ●T n ,Ii ●T n+1},i=1,2,…,N×M;j=1,2,…,n+1;T={T1,T2,T3…T n ,T n+1}; I = {I1, I2, I3...I N*M};

[0145] Step 4.2, the similarity matrix S is normalized by the feature normalization exponential function Softmax Normalization to obtain the final probability matrix P;

[0146] The similarity matrix S is normalized by the feature normalization exponential function to obtain the image-text probability matrix P, which describes the probability that each sub-image of the target image belongs to the corresponding category;

[0147] P={P1,P2,…,P N×M} T

[0148] in, i=1,2,...,N×M, that is

[0149]

[0150] in, τ is the temperature coefficient. is the element in the similarity matrix S that represents the similarity of subgraph i belonging to category k, P i j It represents the normalized probability value when the model predicts the i-th sub-graph as category j. There are a total of N×M×(n+1) elements in the graph-text probability matrix P.

[0151] In the above preprocessing, the target image is first cut into N×M sub-graphs. In StringDB, there are n well-known brands (i.e., categories), plus one category to which the URL domain name to be detected belongs, so there are a total of n+1 categories. Therefore, the "sub-graph-category" similarity probability prediction matrix P has a total of N×M×(n+1) elements. If we directly use the "P i j "To predict the category j to which subgraph i belongs will inevitably bring a large time overhead. Therefore, it is necessary to select the subgraph category prediction probability that is useful for judging the nature of webpage phishing from the similarity probability prediction matrix P, and then provide a basis for judging whether the target website is a phishing website.

[0152] To improve efficiency, this embodiment applies the probability reasoning results of the CLIP model about the category to which the subgraph belongs to to phishing detection based on the following assumption: if the key information in a subgraph of the webpage screenshot is highly correlated with the website category reflected by the domain name in the input URL, then the website is considered to be a legitimate website. The detailed method of step 5 of this embodiment is:

[0153] Step 5.1, generate a subgraph category probability decision set DecisionSet, and use the decision set DecisionSet to provide a basis for judging the legal websites of unknown categories;

[0154] For a legitimate website, if its category is unknown, that is, the category represented by the domain name in the URL of the website is not stored in StringDB, then the probability of predicting the subgraph i containing key information as the category to which the URL domain name belongs (i.e., n+1) will be significantly higher than other categories stored in StringDB; the method for generating the subgraph category probability decision set DecisionSet is:

[0155] Set the highest prediction probability set MaxProbSet to save the maximum probability value of each sub-graph i belonging to different categories Among them, for subgraph i (i = 1, 2, ..., N × M), the category number max to which the maximum probability value belongs is i (max i =1,2,…,n,n+1) is found as follows:

[0156] By traversing each row of the graph-text probability matrix P through the above formula, the maximum value of the model's predicted probability of the corresponding subgraph belonging to each category is taken to complete the selection of the highest predicted probability value of the category to which the subgraph belongs;

[0157] The MaxProbSet obtained by the above process is expressed as:

[0158]

[0159] Based on the obtained highest prediction probability set MaxProbSet, the probability threshold ρ1 is set, and the elements with sub-graph category probability values ​​less than ρ1 in MaxProbSet are deleted to complete the construction of the sub-graph category decision probability set DecisionSet. The content of step 5.1 can be expressed by Algorithm 1:

[0160] Algorithm 1: Generation of subgraph category probability decision set DecisionSet

[0161]

[0162] Step 5.2, generate the sub-graph category probability expansion set ExpandSet;

[0163] For a legitimate website, if its category is known, that is, the category represented by the domain name in the URL of the website has been stored in StringDB and the category is k, then the probability of predicting subgraph i containing the key information of the website as category k is The probability that the model predicts it as the category to which the URL domain belongs is apportioned Since the predicted probability of the known class k The partial probability value of the prediction of the unknown category n+1 is amortized, then and Will not exceed max i The threshold ρ1 is preset to find the maximum probability value;

[0164] The specific method of constructing the subgraph category probability decision expansion set ExpandSet is:

[0165] By setting the sub-image category probability threshold ρ2, we traverse each sub-image of the target image, that is, each row of the matrix P; during the traversal process, if the category of a sub-image i is predicted to be the probability of an existing category k in StringDB and If they are close and both are in the interval [ρ2,1], then the elements representing the probability information of the two categories are stored in the ExpandSet set;

[0166] The setting of ρ2 depends on the value of ρ1, that is, under normal circumstances 2ρ2 should be close to ρ1.

[0167] The above process can be expressed by Algorithm 2:

[0168] Algorithm 2: Generation of subgraph category probability expansion set ExpandSet.

[0169]

[0170] Step 5.3: Generate a phishing detection report;

[0171] The subgraph category probability decision set DecisionSet and the subgraph category probability expansion set ExpandSet are combined to generate a detection report reflecting whether the target website is a phishing website.

[0172] Furthermore, the phishing detection report in step 5.3 includes the following situations:

[0173] Case (1), DecisionSet is not empty and there is a subgraph prediction probability of category n+1, that is, there is Such elements; since the values ​​of these elements are all greater than ρ1, it is concluded that (a) the model's prediction of the category to which the relevant subgraph belongs is highly consistent with the category described by the input URL domain name; (b) the category of the input URL does not share probability with the website category in StringDB, and therefore it is not a website in StringDB;

[0174] At this point, the final detection report is: the target website is a legitimate website of unknown category;

[0175] Case (2), ExpandSet is not empty and there is a subgraph prediction probability of category n+1, that is, there is This type of paired elements; since the values ​​of these elements are all greater than ρ2, it can be concluded that (a) the model's prediction of the category to which the relevant subgraph belongs is consistent with the category described by the input URL domain name; (b) the category of the input URL shares the probability with the website category in StringDB, that is, the category of the URL already exists in StringDB;

[0176] At this point, the final detection report is: the target website is a legitimate website of a known category;

[0177] In case (3), at least one of DecisionSet and ExpandSet is not empty, and there is no subgraph prediction probability of category n+1, which means that the webpage has at least one key information and the key information is irrelevant to the webpage domain name. Since the values ​​of these elements are all greater than ρ1 or ρ2, it can be judged that (a) the category they predict is not the category described by the input URL domain name; (b) the target website has at least one key information that is intended to imitate the key information of the legitimate website;

[0178] At this point, the final detection report is: the target website is a phishing website that imitates the key information of a legitimate website;

[0179] In case (4), both DecisionSet and ExpandSet are empty, which indicates that there is no obvious logo or key text description in the webpage. For phishing attackers, webpages without key information will not serve the purpose of imitating phishing websites, that is, phishing attackers will not imitate such webpages without key information. Based on the above analysis, such webpages without key information are directly considered to be legitimate.

[0180] At this point, the final test report is: a legitimate website without any critical information.

Claims

1. A multimodal phishing website detection method based on a large model, characterized in that: The following steps are involved: Step 1: First extract the URL address of the target website that the user wants to visit and send it to the server for processing; The URL address includes the URL protocol, subdomain, domain name and domain name suffix, port number, and resource path; Step 2: The server extracts the domain name of the URL address as the original text information, and intercepts the screenshot of the web page corresponding to the URL address as the original image information, and then obtains the text initial feature Token and the image initial feature Image by respective preprocessing methods for the original text information and the original image information; Step 3: For the obtained text initial feature Token and image initial feature Image, the server processes them through the Text Transformer layer and VIT layer of the neural network deep learning model CLIP to obtain the text feature T and image feature I respectively; Step 4: The server calculates the similarity between the text feature T and the image feature I to obtain the image-text similarity matrix S, and then uses the normalized exponential function to process the image-text similarity matrix to obtain the image-text probability matrix P; Step 5: The graph-text probability matrix P is used to generate a decision set and an expansion set by analyzing the selector. By analyzing the decision set and the expansion set, it is determined whether the target website is a phishing website. Step 6: Return the judgment result to the browser. If it is a phishing website, a warning dialog box will pop up to prompt the user. If it is a legitimate website, no operation will be performed and the user can access it directly.

2. The large model-based multimodal phishing website detection method according to claim 1, characterized in that: The detailed method for obtaining the initial features of the text in step 2 is: A) Construct an extensible brand category list StringDB; after storing each brand category in StringDB, use the position of each brand category in StringDB as its label Label. There are n strings in the brand category list StringDB; B) Preprocess the URL address text. The process is as follows: First, obtain the domain name string from the target website URL U and use it as the detection category of the target website ； Then, the detected category string of the target website is merged with the category string of StringDB to form an extended category string list L, that is, L = {L1, L2, L3...L n ,L n+1 }; Among them, L i is the i-th category string stored in StringDB, where i = 1, 2, ..., n, L n+1 The target website domain name string obtained from the URL; Next, the obtained extended category string list L is combined with the prompt word P to generate a string list with prompt words; assuming that the prompt word used is P, the new string list with prompt words formed by combining L and P is L new for: L new ={[P]L1,[P]L2,…,[P]L n ,[P]L n+1 }; Finally, for the string list L new Perform word segmentation processing and convert it into word segmentation form, that is, the initial feature Token of the text: Token = {Token1, Token2, ..., Token n ,Token n+1 }; Among them, Token i For [P]L i ∈L new The result after word segmentation processing is i=1,2,…,n,n+1.

3. The large model-based multimodal phishing website detection method according to claim 1, characterized in that: The detailed method of processing the original information of the image in step 2 to obtain the initial features of the image is: segmenting, cropping, and padding the original information of the image in sequence, and cutting the entire large image into several sub-images; assuming that N*M cutting is adopted, a picture is finally segmented into N*M sub-images, and the initial features of the image after preprocessing Image are finally obtained as follows: Image={Image1,Image2,Image3,Image4...Image N*M }。 4. The large model-based multimodal phishing website detection method according to claim 1, characterized in that: The specific method of calculating the text features and image features in step 3 is: The initial feature Token text obtained in step 2 is encoded and feature extracted through the text encoding layer Text_Encoder, and then a text feature vector list T of the website brand category is generated to obtain the text feature: T = Text_Encoder (Token) T={T1,T2,T3...T n ,T n+1 } The text encoding layer Text_Encoder adopts the Text Transformer architecture; The initial features Image obtained in step 2 are encoded and extracted through the image encoding layer Image_Encoder, and then the Image is mapped to a higher dimensional space to obtain the image features; I = Image_Encoder(Image) I={I1,I2,I3...I N*M } The picture encoding layer adopts the Vision Transformer architecture.

5. The large model-based multimodal phishing website detection method according to claim 4, characterized in that: The core of the Text Transformer architecture and the Vision Transformer architecture both use the self-attention mechanism. For a certain input original feature X, the calculation formula of multi-head self-attention is: First, the original feature X is linearly transformed to obtain the query (Query, Q), key (Key, K), and value (Value, V). Where W Q ,W K ,W V is the weight matrix of the model; Q = XW Q ,K=XW K ,V=XW V ; Then, we calculate the dot product of Q and K to get the attention score and divide it by the scaling factor where d k is the dimension of the key K, and then the score is normalized by Softmax and multiplied by the value V to get the Attention score, where the Attention formula is as follows: Next, the multi-head attention replicates the self-attention mechanism multiple times, each head calculates the attention of a different representation subspace, concatenates the results and passes another linear transformation to get the final output: MultiHead(Q,K,V)=Concat(head1,...,head h )W0 The expression for each head is: Finally, the feedforward network FFN and residual connection, as well as layer normalization LayerNorm are performed to obtain the dimensional features: FFN(x)=LayerNorm(max(0,xW1+b1)W2+b2) The final output feature of vector X is: X1=LayerNorm(X+MultiHead(X)) X2=LayerNorm(X1+FFN(X1)) That is, the original feature X is transformed into feature X2 after the transformer architecture.

6. The large model-based multimodal phishing website detection method according to claim 1, characterized in that: The detailed process of step 4 is as follows: Step 4.1: Each vector T in the text feature T j And each vector I in the image feature I i Perform dot product operations to obtain the similarity matrix S between each sub-graph and the category prompt text: S={S1,S2,…,S N×M }; Among them, S i = {I i ● T1, I i ● T2, ……, I i ● T n , I i ● T n+1}, i = 1, 2, ……, N × M; j = 1, 2, ……, n + 1; T = {T1, T2, T3... T n , T n+1}; I = {I1, I2, I3... I N*M}; Step 4.2, the similarity matrix S is normalized by the feature normalization exponential function Softmax Normalization to obtain the final probability matrix P; The similarity matrix S is normalized by the feature normalization exponential function to obtain the image-text probability matrix P, which describes the probability that each sub-image of the target image belongs to the corresponding category; P={P1,P2,…,P N×M } T in, Right now in, τ is the temperature coefficient, is the element in the similarity matrix S that represents the similarity of subgraph i belonging to category k, It represents the normalized probability value when the model predicts the i-th sub-graph as category j. There are a total of N×M×(n+1) elements in the graph-text probability matrix P.

7. The large model-based multimodal phishing website detection method according to claim 1, characterized in that: The detailed method of step 5 is as follows: Step 5.1, generate a subgraph category probability decision set DecisionSet, and use the decision set DecisionSet to provide a basis for judging the legal websites of unknown categories; For a legitimate website, if its category is unknown, that is, the category represented by the domain name in the URL of the website is not stored in StringDB, then the probability of predicting the subgraph i containing key information as the category to which the URL domain name belongs will be significantly higher than other categories stored in StringDB; the method for generating the subgraph category probability decision set DecisionSet is: Set the highest prediction probability set MaxProbSet to save the maximum probability value of each sub-graph i belonging to different categories Among them, for sub-graph i, the category number max to which its maximum probability value belongs is i The search method is: By traversing each row of the graph-text probability matrix P through the above formula, the maximum value of the model's predicted probability of the corresponding subgraph belonging to each category is taken to complete the selection of the highest predicted probability value of the category to which the subgraph belongs; The MaxProbSet obtained by the above process is expressed as: The probability threshold ρ1 is set based on the obtained highest prediction probability set MaxProbSet, and the elements with sub-graph category probability values ​​less than ρ1 in MaxProbSet are deleted to complete the construction of the sub-graph category decision probability set DecisionSet; Step 5.2, generate the sub-graph category probability expansion set ExpandSet; For a legitimate website, if its category is known, that is, the category represented by the domain name in the URL of the website has been stored in StringDB and the category is k, then the probability of predicting subgraph i containing the key information of the website as category k is The probability that the model predicts the category to which the URL domain belongs is Since the predicted probability of the known class k The partial probability value of the prediction of the unknown category n+1 is amortized, then and Will not exceed max i The threshold ρ1 is preset to find the maximum probability value; The specific method of constructing the subgraph category probability decision expansion set ExpandSet is: By setting the sub-image category probability threshold ρ2, we traverse each sub-image of the target image, that is, each row of the matrix P; during the traversal process, if the category of a sub-image i is predicted to be the probability of an existing category k in StringDB and If they are close and both are in the interval [ρ2,1], then the elements representing the probability information of the two categories are stored in the ExpandSet set; The setting of ρ2 depends on the value of ρ1, that is, in normal cases 2ρ2 should be close to ρ1; Step 5.3: Generate a phishing detection report; The subgraph category probability decision set DecisionSet and the subgraph category probability expansion set ExpandSet are combined to generate a detection report reflecting whether the target website is a phishing website.

8. The large model-based multimodal phishing website detection method according to claim 7, characterized in that: The phishing detection report in step 5.3 includes the following situations: Case (1), DecisionSet is not empty and there is a subgraph prediction probability of category n+1, that is, there is …such elements; since the values ​​of these elements all exceed ρ1, it is concluded that (a) the model's prediction of the category to which the relevant subgraph belongs is highly consistent with the category described by the input URL domain name; (b) the category of the input URL does not share probability with the website category in StringDB, and therefore it is not a website in StringDB; At this point, the final detection report is: the target website is a legitimate website of unknown category; Case (2), ExpandSet is not empty and there is a subgraph prediction probability of category n+1, that is, there is …such paired elements; since the values ​​of these elements all exceed ρ2, it can be concluded that (a) the model's prediction of the category to which the relevant subgraph belongs is consistent with the category described by the input URL domain name; (b) the category of the input URL shares the probability with the website category in StringDB, that is, the category of the URL already exists in StringDB; At this point, the final detection report is: the target website is a legitimate website of a known category; In case (3), at least one of DecisionSet and ExpandSet is not empty, and there is no subgraph prediction probability of category n+1, which means that the webpage has at least one key information and the key information is irrelevant to the webpage domain name. Since the values ​​of these elements are all greater than ρ1 or ρ2, it can be judged that (a) the category they predict is not the category described by the input URL domain name; (b) the target website has at least one key information that is intended to imitate the key information of the legitimate website; At this point, the final detection report is: the target website is a phishing website that imitates the key information of a legitimate website; In case (4), both DecisionSet and ExpandSet are empty, which indicates that there is no obvious logo or key text description in the webpage. For phishing attackers, webpages without key information will not serve the purpose of imitating phishing websites, that is, phishing attackers will not imitate such webpages without key information. Based on the above analysis, such webpages without key information are directly considered to be legitimate. At this point, the final test report is: a legitimate website without any critical information.

Images