Multimodal phishing website detection method based on large model

Abstract

The present invention discloses a multimodal phishing website detection method based on a large model. The server extracts the domain name text of the URL address of the target website and the web page screenshot corresponding to the URL address, and then pre-processes the obtained domain name text and web page screenshot respectively to obtain text initial features and image initial features; the text features and image features are obtained by CLIP processing using a neural network deep learning model; the similarity between the obtained text features and image features is calculated to obtain a graph-text similarity matrix, and normalized to obtain a graph-text probability matrix; the graph-text probability matrix is ​​used to generate a decision set and an extension set through an analysis selector, and finally, by analyzing the decision set and the extension set, it is determined whether the target website is a phishing website.

Description

Technical Field

[0001] The present invention relates to network security technology, and in particular to a multimodal phishing website detection method based on a large model. Background Art

[0002] Nowadays, with the rapid development of network technology, more and more online trading platforms, online shopping platforms, and banking services are deployed on the Internet, enriching daily life and providing people with great convenience. However, in the current network environment, there are attackers who use various attack methods to steal users' sensitive information, causing great losses to users' property. Stealing sensitive information through phishing web pages and phishing emails is the most common method. Although phishing detection technology continues to develop, the attackers' attack methods are also constantly changing. Faced with such a phishing threat, effective countermeasures are urgently needed.

[0003] In order to reduce the threat posed by phishing to the public, the academic community has proposed many phishing website detection technologies, which are mainly divided into four categories: blacklist technology, heuristic rule algorithms, machine learning methods and deep learning technology. However, these methods have certain limitations and defects.

[0004] Blacklist-based detection technology

[0005] A blacklist is a list of previously identified phishing URLs. This list is used to identify phishing sites by checking whether the input URL is on it. Many browsers now include blacklist detection capabilities to protect users from phishing attacks and avoid losses. Blacklist detection is highly efficient because it doesn't require analyzing the specific content of web pages. However, it requires constant maintenance and cannot detect zero-day phishing sites, meaning it can't immediately respond to newly emerging malicious URLs.

[0006] Detection technology based on heuristic rule algorithm

[0007] Heuristic algorithms are a technology that predicts and identifies phishing websites by analyzing website characteristics and patterns, rather than relying on predefined blacklists. These characteristics may include, but are not limited to, the structure of the website's URL, page content, domain name registration information, and similarities to known phishing websites. The core of heuristic algorithms is to use these characteristics to identify potential phishing websites, even those not yet blacklisted. Heuristic algorithms are often combined with machine learning to detect phishing pages. However, heuristic rule-based algorithms can be deliberately circumvented by attackers by analyzing and obtaining heuristics.

[0008] Machine learning-based detection technology

[0009] Machine learning detection methods learn from the characteristics of a website and predict whether it is a phishing website. However, machine learning is limited by the training data set. The quality of the training set directly affects the effectiveness of the model itself. In addition, machine learning has poor generalization ability and cannot be applied to zero-day phishing detection.

[0010] Detection technology based on deep learning

[0011] Due to its black-box neural network mechanism, deep learning can better detect phishing websites than other methods, making it difficult for attackers to circumvent them by identifying patterns. However, deep learning places high demands on platform training hardware. Currently, phishing website detection models based on visual information include visual similarity, which determines the degree of visual similarity between a webpage and a legitimate webpage as the basis for detection. These methods are generally referred to as single-modal and result in a heavy reliance on training datasets. There are also deep models that combine images and text, but these models lack the semantic information between text and images, resulting in poor performance in zero-day phishing detection and inability to identify phishing websites not present in the training dataset. Therefore, there is an urgent need for a deep model with low training load, high accuracy, and a combination of visual and textual semantic multimodality, which can also recognize websites not present in the training dataset. Summary of the Invention

[0012] Purpose of the invention: The purpose of the present invention is to address the deficiencies in the prior art and to provide a large-model-based multimodal phishing website detection method. Based on the multimodal deep learning large-model method and the large-model zero-sample inference capability, the method uses the URL address and the webpage screenshot as multimodal features to detect whether the original website is a phishing website, thereby greatly improving the accuracy and speed of phishing website detection and enabling zero-day phishing detection.

[0013] Technical solution: The present invention provides a multimodal phishing website detection method based on a large model, comprising the following steps:

[0014] Step 1: First, extract the URL address of the target website that the user wants to visit and send it to the server for processing; the URL address includes the URL protocol, subdomain, domain name and domain name suffix, port number, and resource path;

[0015] Step 2: The server extracts the domain name of the URL address as the text original (row) information, and uses the automated tool selenium to intercept the webpage screenshot corresponding to the URL address as the image original (row) information. The text original information and the image original information are then preprocessed by their respective methods to obtain the text initial feature Token and the image initial feature Image;

[0016] Step 3: For the obtained initial text feature Token and initial image feature Image, the server processes them through the Text Transformer layer and VIT (Vision Transformer) layer of the neural network deep learning model CLIP to obtain text feature T and image feature I respectively;

[0017] Step 4: The server calculates the similarity between the text feature T and the image feature I to obtain the image-text similarity matrix S. Then, the image-text similarity matrix is ​​processed using the softmax normalization function to obtain the image-text probability matrix P.

[0018] Step 5: The graph-text probability matrix is ​​analyzed by the selector to generate a decision set and an expansion set. By analyzing the decision set and the expansion set, it is determined whether the target website is a phishing website.

[0019] Step 6: Return the judgment result to the browser. If it is a phishing website, a warning dialog box will pop up to remind the user. If it is a legitimate website, no operation will be performed and the user can access it directly.

[0020] The above step 1 intercepts the URL of the target website that the user is about to browse at the same time based on the user entering the URL address into the browser and pressing Enter or entering the web page through a hyperlink. Web page hyperlinks, usually referred to as links, are elements in the World Wide Web that are used to jump from one web page to another web page or a specific part of a web page. Hyperlinks are one of the basic features of the Internet, allowing users to access information on other pages or sites by clicking on linked text or images. The main types of hyperlinks include: internal links: links to other pages within the same website. External links: links to pages on different websites. Anchor links: links to specific parts within the same page, usually used for navigation on long pages.

[0021] Furthermore, the detailed method for obtaining the initial features of the text in step 2 is:

[0022] A) Construct an extensible brand category list StringDB; after storing each brand category in StringDB, use the position of each brand category in StringDB as its label Label. There are n strings in the brand category list StringDB; for example, StringDB: [adidas, absa, adobe.....ziggo]. Assuming that the number of strings in this StringDB is 155, then n is 155, and the domain name position of the target website to be detected is n+1.

[0023] B) Preprocess the URL address text. The process is as follows:

[0024] First, the domain name string is obtained from the target website URL U and used as the detection category of the target website. For example, for the URL "https: / / www.alibaba.com", the domain name suffix and subdomain are removed based on the obtained domain name through regular expressions to obtain the final domain name "alibaba".

[0025] Then, the detected category string of the target website is merged with the category string of StringDB to form an extended category string list L, that is, L={L1, L2, L3...L n ,L n+1};

[0026] Among them, L i is the i-th category string stored in StringDB, i=1,2,…,n, L n+1 The target website domain name (category) string obtained from the URL;

[0027] Next, the obtained extended category string list L is combined with the prompt word P to generate a string list with prompt words. Accurate prompt words can make the model recognition more accurate. Assuming that the prompt word used is P, the new string list L with prompt words formed by combining L and P is new for:

[0028] L new ={[P]L1,[P]L2,…,[P]L n ,[P]L n+1};

[0029] Finally, for the string list L new Perform token processing and convert it into a token form, that is, the initial feature Token of the text: Token = {Token1, Token2, ..., Token n ,Token n+1};

[0030] Among them, Token i For [P]L i ∈L new The result after word segmentation processing is i=1,2,…,n,n+1.

[0031] Furthermore, the detailed method for processing the original (row) information of the image in step 2 to obtain the initial features of the image is:

[0032] The original (row) information of the image is segmented, cropped, and padded in sequence to cut the entire large image into several sub-images. Assuming that N*M cutting is used, the final image is divided into N*M sub-images. The initial feature image of the preprocessed image is as follows:

[0033] Image={Image1,Image2,Image3,Image4...Image N*M}.

[0034] Furthermore, the specific method of calculating text features and image features in step 3 is:

[0035] The text encoding layer Text_Encoder of the CLIP model encodes and extracts the initial feature Token text obtained in step 2, and then generates a text feature vector list T of the website brand category to obtain the text features:

[0036] T = Text_Encoder(Token)

[0037] T={T1,T2,T3...T n ,T n+1}

[0038] The text encoding layer Text_Encoder adopts the Text Transformer architecture;

[0039] The image encoding layer Image_Encoder of the CLIP model encodes and extracts the initial features Image obtained in step 2, and then maps the Image to a higher-dimensional space to obtain image features;

[0040] I = Image_Encoder(Image)

[0041] I={I1,I2,I3...I N*M}

[0042] The picture coding layer adopts the Vision Transformer architecture.

[0043] Furthermore, the core of the Text Transformer architecture and the Vision Transformer architecture both use the self-attention mechanism. For a certain input original feature X, the calculation formula of multi-head self-attention is:

[0044] First, the original feature X is linearly transformed to obtain query (Query, Q), key (Key, K), and value (Value, V), where W Q,W K ,W V Is the weight matrix of the model, which is learned during the training process: Q = XW Q ,K=XW K ,V=XW V ;

[0045] Then, we calculate the dot product of Q and K to get the attention score and divide it by the scaling factor where d k is the dimension of the key K, and then the score is normalized by Softmax and multiplied by the value V to get the Attention score, where the Attention formula is as follows:

[0046] Next, Multi-Head Attention replicates the self-attention mechanism multiple times (heads), each head computes the attention of a different representation subspace, concatenates the results, and passes another linear transformation to obtain the final output:

[0047] MultiHead(Q,K,V)=Concat(head1,...,head h )W0

[0048] The expression of each head is: head i =Attention(QW i Q ,KW i K ,VW i V );

[0049] Finally, the feed-forward network (FFN) and residual connection, as well as layer normalization LayerNorm are performed to obtain dimensional features:

[0050] FFN(x)=LayerNorm(max(0,xW1+b1)W2+b2)

[0051] The final output features of vector X are:

[0052] X1=LayerNorm(X+MultiHead(X))

[0053] X2=LayerNorm(X1+FFN(X1))

[0054] That is, the original feature X is transformed into feature X2 after the transformer architecture.

[0055] Furthermore, the detailed process of step 4 is as follows:

[0056] Step 4.1: Each vector T in the text feature T j and each vector I in the image feature I i Perform dot product operations to obtain the similarity matrix S between each subgraph and the category prompt text:

[0057] S={S1,S2,…,S N×M};

[0058] Among them, S i ={I i T1, I i ●T2,……,I i ●T n ,I i ●T n+1},i=1,2,…,N×M;j=1,2,…,n+1;T={T1,T2,T3…T n ,T n+1}; I={I1,I2,I3...I N*M};

[0059] Step 4.2: Apply the Softmax Normalization function to the similarity matrix S to obtain the final probability matrix P.

[0060] The similarity matrix S is normalized by the feature normalization exponential function to obtain the image-text probability matrix P, which describes the probability that each sub-image of the target image belongs to the corresponding category;

[0061] P={P1,P2,…,P N×M} T

[0062] in, i=1,2,...,N×M, that is

[0063]

[0064] in, τ is the temperature coefficient, is the element in the similarity matrix S that represents the similarity of subgraph i belonging to category k, P i j It represents the normalized probability value when the model predicts the i-th sub-graph as category j. The graph-text probability matrix P has a total of N×M×(n+1) elements.

[0065] Furthermore, the detailed method of step 5 is:

[0066] Step 5.1: Generate the subgraph category probability decision set DecisionSet, and use the decision set DecisionSet to provide a basis for judging the legal websites of unknown categories;

[0067] For a legitimate website, if its category is unknown, that is, the category represented by the domain name in the URL of the website is not stored in StringDB, then the probability of predicting subgraph i containing key information as the category to which the URL domain name belongs (i.e., n+1) will be significantly higher than other categories stored in StringDB. The method for generating the subgraph category probability decision set DecisionSet is:

[0068] Set the highest prediction probability set MaxProbSet to save the maximum probability value of each subgraph i belonging to different categories Among them, for subgraph i (i=1,2,…,N×M), the category number max to which the maximum probability value belongs is i (max i =1,2,…,n,n+1) is found as follows:

[0069] By traversing each row of the image-text probability matrix P through the above formula, the maximum value of the model's predicted probability of the corresponding sub-image belonging to each category is taken to complete the selection of the highest predicted probability value of the category to which the sub-image belongs;

[0070] The MaxProbSet obtained by the above process is expressed as:

[0071]

[0072] Based on the obtained highest prediction probability set MaxProbSet, a probability threshold ρ1 is set. By deleting the elements in MaxProbSet whose subgraph category probability value is less than ρ1, the subgraph category decision probability set DecisionSet is constructed.

[0073] Step 5.2: Generate the subgraph category probability expansion set ExpandSet;

[0074] For a legitimate website, if its category is known, that is, the category represented by the domain name in the URL of the website has been stored in StringDB and the category is k, then the probability of predicting the subgraph i containing the key information of the website as category k is The probability that the apportioned model predicts it as the category to which the URL domain belongs Since the predicted probability of the known class k The partial probability value of the prediction of the unknown category n+1 is amortized, then and Will not exceed max iThe threshold ρ1 is preset to find the maximum probability value;

[0075] The specific method of constructing the subgraph category probability decision expansion set ExpandSet is:

[0076] By setting the sub-image category probability threshold ρ2, we traverse each sub-image of the target image, that is, each row of the matrix P; during the traversal process, if the category of a sub-image i is predicted to be the probability of the existing category k in StringDB and If they are close and both are in the interval [ρ2,1], then the elements representing the probability information of the two categories are stored in the ExpandSet set;

[0077] The setting of ρ2 depends on the value of ρ1, that is, under normal circumstances, 2ρ2 should be close to ρ1;

[0078] Step 5.3: Generate a phishing detection report;

[0079] The subgraph category probability decision set DecisionSet and the subgraph category probability expansion set ExpandSet are combined to generate a detection report reflecting whether the target website is a phishing website.

[0080] Furthermore, the phishing detection report in step 5.3 includes the following situations:

[0081] Case (1), DecisionSet is not empty and there is a subgraph prediction probability of category n+1, that is, there is Since the values ​​of these elements all exceed ρ1, we conclude that (a) the model's prediction of the category of the relevant subgraph is highly consistent with the category described by the input URL domain name; (b) the category of the input URL does not share probability with the website category in StringDB, and therefore it is not a website in StringDB;

[0082] At this point, the final detection report is: the target website is a legitimate website of unknown category;

[0083] Case (2), ExpandSet is not empty and there is a subgraph prediction probability of category n+1, that is, there is Since the values ​​of these elements all exceed ρ2, we can conclude that (a) the model's prediction of the category of the relevant subgraph is consistent with the category described by the input URL domain name; (b) the category of the input URL and the website category in StringDB share the probability, that is, the category of the URL already exists in StringDB;

[0084] At this point, the final detection report is: the target website is a legitimate website of a known category;

[0085] Case (3): At least one of the DecisionSet and ExpandSet is not empty, and there is no subgraph prediction probability of category n+1, indicating that the webpage has at least one key information and this key information is not related to the webpage domain name. Since the values ​​of these elements are all greater than ρ1 or ρ2, it can be judged that (a) the category they predict is not the category described by the input URL domain name; (b) the target website has at least one key information that is intended to imitate the key information of the legitimate website;

[0086] The final detection report at this time is: the target website is a phishing website that imitates the key information of the legitimate website;

[0087] In case (4), both DecisionSet and ExpandSet are empty, which indicates that there is no obvious logo or key text description in the webpage. For phishing attackers, webpages without key information will not serve the purpose of imitating phishing websites. That is, phishing attackers will not imitate such webpages without key information. Based on the above analysis, such webpages without key information are directly considered legitimate.

[0088] The final test report at this time is: a legal website without any key information.

[0089] Beneficial effects: The present invention introduces the large model CLIP into the field of phishing detection for the first time, and uses the advantages of the large model to solve the problem that other classification models cannot detect phishing websites of untrained categories, and solves the problem of lack of semantic correlation between images and texts compared to other models, and realizes efficient phishing detection. The present invention improves the preprocessing stage. In order to prevent the loss of overall information of the image, we use cropping, segmentation, and padding methods to ensure that the information of the image is not missing. It makes up for the problem that key features of large-size web page images will be severely lost after compression. The present invention analyzes the characteristics of web page screenshots, and therefore analyzes the website by adding an analysis selector, and finally generates a decision set and an extension set to determine whether it is a legitimate or phishing website, and our model has a high phishing detection function for zero-day phishing websites. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 It is the overall flow chart of the present invention;

[0091] Figure 2 The basic structure diagram of the URL of the present invention;

[0092] Figure 3 Schematic diagram of the text preprocessing process in the present invention;

[0093] Figure 4 Schematic diagram of the image preprocessing process in the present invention;

[0094] Figure 5 Schematic diagram of the overall detection framework of the server in the present invention;

[0095] Figure 6 This is the process of extracting features using the attention mechanism in this invention. DETAILED DESCRIPTION

[0096] The technical solution of the present invention is described in detail below, but the protection scope of the present invention is not limited to the embodiments.

[0097] like Figure 1 and Figure 5 As shown, the multimodal phishing website detection method based on a large model of the present invention includes the following steps:

[0098] Step 1: First, extract the URL address of the target website that the user wants to visit and send it to the server for processing; the URL address includes the URL protocol, subdomain, domain name and domain name suffix, port number, and resource path;

[0099] Step 2: The server extracts the domain name of the URL address as the text original (row) information, and uses the automated tool selenium to intercept the webpage screenshot corresponding to the URL address as the image original (row) information. The text original information and the image original information are then preprocessed by their respective methods to obtain the text initial feature Token and the image initial feature Image;

[0100] Step 3: For the obtained initial text feature Token and initial image feature Image, the server processes them through the Text Transformer layer and VIT (Vision Transformer) layer of the neural network deep learning model CLIP to obtain text feature T and image feature I respectively;

[0101] Step 4: The server calculates the similarity between the text feature T and the image feature I to obtain the image-text similarity matrix S. Then, the image-text similarity matrix is ​​processed using the softmax normalization function to obtain the image-text probability matrix P.

[0102] Step 5: The graph-text probability matrix is ​​analyzed by the selector to generate a decision set and an expansion set. By analyzing the decision set and the expansion set, it is determined whether the target website is a phishing website.

[0103] Step 6: Return the judgment result to the browser. If it is a phishing website, a warning dialog box will pop up to remind the user. If it is a legitimate website, no operation will be performed and the user can access it directly.

[0104] The URL address of the present invention is composed of four key components: Figure 2 : First is the URL protocol, such as http or ftp, which defines how to access resources; second is the subdomain, domain name and domain suffix, which are used to identify a specific host in the network; then comes the port number: used to specify access.

[0105] The present invention applies the large text-image multimodal zero-sample inference model CLIP to the field of phishing page recognition. With the help of the CLIP model's high inference accuracy in zero-sample conditions, the parameter fine-tuning of the migration process does not require large-scale training samples, and through the construction of a maintainable brand category list (this article uses the name of the website brand as its category), page image segmentation, and sub-image probability matrix generation and selection research, accurate classification and recognition of phishing pages in small sample conditions are achieved. Through comparative learning, the model can learn the correlation between images and text, so that given a text description, it can identify the image that matches it, or given an image, it can find the text description that matches it.

[0106] The CLIP model in this paper is pre-trained on a large-scale dataset of image and text pairs. It uses two sub-networks, an image encoder and a text encoder, to extract feature representations of images and text, respectively. Then, through the contrastive learning objective function, the model is able to identify matching image-text pairs and distinguish mismatched pairs.

[0107] The detailed method for obtaining the initial features of the text in step 2 of this embodiment is as follows:

[0108] A) Construct an extensible brand category list StringDB; after storing each brand category in StringDB, use the position of each brand category in StringDB as its label. The brand category list StringDB contains n strings;

[0109] B) Preprocess the URL address text, such as Figure 3 The process is:

[0110] First, obtain the domain name string from the target website URL U and use it as the detection category of the target website ；

[0111] Then, the detected category string of the target website is merged with the category string of StringDB to form an extended category string list L, that is, L={L1, L2, L3...L n ,L n+1};

[0112] Among them, L iis the i-th category string stored in StringDB, i=1,2,…,n, L n+1 The target website domain name (category) string obtained from the URL;

[0113] Next, the obtained extended category string list L is combined with the prompt word P to generate a string list with prompt words; assuming that the prompt word used is P, the new string list with prompt words formed by combining L and P is L new for:

[0114] L new ={[P]L 1 ,[P]L2,…,[P]L n ,[P]L n+1};

[0115] Finally, for the string list L new Perform token processing and convert it into a token form, that is, the initial feature Token of the text: Token = {Token1, Token2, ..., Token n ,Token n+1};

[0116] Among them, Token i For [P]L i ∈L new The result after word segmentation processing is i=1,2,…,n,n+1.

[0117] The detailed method for processing the original (row) information of the image in step 2 of this embodiment to obtain the initial features of the image is as follows:

[0118] The original (row) information of the image is divided, cropped, and padded in sequence to cut the entire large image into several sub-images. Assuming that N*M cutting is used, the final image is divided into N*M sub-images, such as Figure 4 The final preprocessed image initial features Image are as follows:

[0119] Image={Image1,Image2,Image3,Image4...Image N*M}.

[0120] The specific method for calculating text features and image features in step 3 of this embodiment is:

[0121] The text encoding layer Text_Encoder of the CLIP model encodes and extracts the initial feature Token text obtained in step 2, and then generates a text feature vector list T of the website brand category to obtain the text features:

[0122] T = Text_Encoder(Token)

[0123] T={T1,T2,T3...T n ,T n+1}

[0124] The text encoding layer Text_Encoder adopts the Text Transformer architecture;

[0125] The image encoding layer Image_Encoder of the CLIP model encodes and extracts the initial features Image obtained in step 2, and then maps the Image to a higher-dimensional space to obtain image features;

[0126] I = Image_Encoder(Image)

[0127] I={I1,I2,I3...I N*M}

[0128] The picture coding layer adopts the Vision Transformer architecture.

[0129] like Figure 6 As shown in the figure, the core of the Text Transformer architecture and the Vision Transformer architecture described in this embodiment both adopt the self-attention mechanism. For a certain input original feature X, the calculation formula of the multi-head self-attention is:

[0130] First, the original feature X is linearly transformed to obtain query (Query, Q), key (Key, K), and value (Value, V), where W Q ,W K ,W V Is the weight matrix of the model, which is learned during the training process: Q = XW Q ,K=XW K ,V=XW V

[0131] Then, we calculate the dot product of Q and K to get the attention score and divide it by the scaling factor where d k is the dimension of the key K, and then the score is normalized by Softmax and multiplied by the value V to get the Attention score, where the Attention formula is as follows:

[0132] Next, Multi-Head Attention replicates the self-attention mechanism multiple times (heads), each head computes the attention of a different representation subspace, concatenates the results, and passes another linear transformation to obtain the final output:

[0133] MultiHead(Q,K,V)=Concat(head1,...,head h )W0

[0134] The expression of each head is: head i =Attention(QW i Q ,KW i K ,VW i V );

[0135] Finally, the feed-forward network (FFN) and residual connection, as well as layer normalization LayerNorm are performed to obtain dimensional features:

[0136] FFN(x)=LayerNorm(max(0,xW1+b1)W2+b2)

[0137] The final output features of vector X are:

[0138] X1=LayerNorm(X+MultiHead(X))

[0139] X2=LayerNorm(X1+FFN(X1))

[0140] That is, the original feature X is transformed into feature X2 after the transformer architecture.

[0141] The detailed process of step 4 above is as follows:

[0142] Step 4.1: Each vector T in the text feature T j and each vector I in the image feature I i Perform dot product operations to obtain the similarity matrix S between each subgraph and the category prompt text:

[0143] S={S 1 ,S2,…,S N×M};

[0144] Among them, S i ={I i T1, I i ●T2,……,I i ●T n ,Ii ●T n+1},i=1,2,…,N×M;j=1,2,…,n+1;T={T1,T2,T3…T n ,T n+1}; I={I1,I2,I3...I N*M};

[0145] Step 4.2: Apply the Softmax Normalization function to the similarity matrix S to obtain the final probability matrix P.

[0146] The similarity matrix S is normalized by the feature normalization exponential function to obtain the image-text probability matrix P, which describes the probability that each sub-image of the target image belongs to the corresponding category;

[0147] P={P1,P2,…,P N×M} T

[0148] in, i=1,2,...,N×M, that is

[0149]

[0150] in, τ is the temperature coefficient. is the element in the similarity matrix S that represents the similarity of subgraph i belonging to category k, P i j It represents the normalized probability value when the model predicts the i-th sub-graph as category j. The graph-text probability matrix P has a total of N×M×(n+1) elements.

[0151] In the above preprocessing, the target image is first cut into N×M sub-graphs. StringDB contains the names of n well-known brands (i.e., categories), plus the category of the URL domain name to be detected, so there are n+1 categories in total. Therefore, the "sub-graph-category" similarity probability prediction matrix P has a total of N×M×(n+1) elements. If we use the direct "P i j " is used to predict the category j to which subgraph i belongs, which is bound to bring a large time overhead. Therefore, it is necessary to select the subgraph category prediction probability that is useful for judging the nature of webpage phishing from the similarity probability prediction matrix P, and then provide a basis for judging whether the target website is a phishing website.

[0152] To improve efficiency, this embodiment applies the CLIP model's probabilistic inference results about the categories to which subgraphs belong to phishing detection based on the following assumption: if the key information in a subgraph of a webpage screenshot is highly correlated with the website category reflected by the domain name in the input URL, the website is considered legitimate. The detailed method of step 5 of this embodiment is as follows:

[0153] Step 5.1: Generate the subgraph category probability decision set DecisionSet, and use the decision set DecisionSet to provide a basis for judging the legal websites of unknown categories;

[0154] For a legitimate website, if its category is unknown, that is, the category represented by the domain name in the URL of the website is not stored in StringDB, then the probability of predicting subgraph i containing key information as the category to which the URL domain name belongs (i.e., n+1) will be significantly higher than other categories stored in StringDB. The method for generating the subgraph category probability decision set DecisionSet is:

[0155] Set the highest prediction probability set MaxProbSet to save the maximum probability value of each subgraph i belonging to different categories Among them, for subgraph i (i=1,2,…,N×M), the category number max to which the maximum probability value belongs is i (max i =1,2,…,n,n+1) is found as follows:

[0156] By traversing each row of the image-text probability matrix P through the above formula, the maximum value of the model's predicted probability of the corresponding sub-image belonging to each category is taken to complete the selection of the highest predicted probability value of the category to which the sub-image belongs;

[0157] The MaxProbSet obtained by the above process is expressed as:

[0158]

[0159] Based on the obtained highest prediction probability set MaxProbSet, a probability threshold ρ1 is set. By deleting the elements in MaxProbSet whose sub-graph category probability value is less than ρ1, the sub-graph category decision probability set DecisionSet is constructed. The content of step 5.1 can be expressed by Algorithm 1:

[0160] Algorithm 1: Generation of subgraph category probability decision set DecisionSet

[0161]

[0162] Step 5.2: Generate the subgraph category probability expansion set ExpandSet;

[0163] For a legitimate website, if its category is known, that is, the category represented by the domain name in the URL of the website has been stored in StringDB and the category is k, then the probability of predicting the subgraph i containing the key information of the website as category k is The probability that the model predicts it as the category to which the URL domain name belongs is apportioned Since the predicted probability of the known class k The partial probability value of the prediction of the unknown category n+1 is amortized, then and Will not exceed max i The threshold ρ1 is preset to find the maximum probability value;

[0164] The specific method of constructing the subgraph category probability decision expansion set ExpandSet is:

[0165] By setting the sub-image category probability threshold ρ2, we traverse each sub-image of the target image, that is, each row of the matrix P; during the traversal process, if the category of a sub-image i is predicted to be the probability of the existing category k in StringDB and If they are close and both are in the interval [ρ2,1], then the elements representing the probability information of the two categories are stored in the ExpandSet set;

[0166] The setting of ρ2 depends on the value of ρ1, that is, in general 2ρ2 should be close to ρ1.

[0167] The above process can be expressed by Algorithm 2:

[0168] Algorithm 2: Generation of the subgraph category probability expansion set ExpandSet.

[0169]

[0170] Step 5.3: Generate a phishing detection report;

[0171] The subgraph category probability decision set DecisionSet and the subgraph category probability expansion set ExpandSet are combined to generate a detection report reflecting whether the target website is a phishing website.

[0172] Furthermore, the phishing detection report in step 5.3 includes the following situations:

[0173] Case (1), DecisionSet is not empty and there is a subgraph prediction probability of category n+1, that is, there is Since the values ​​of these elements all exceed ρ1, we conclude that (a) the model's prediction of the category of the relevant subgraph is highly consistent with the category described by the input URL domain name; (b) the category of the input URL does not share probability with the website category in StringDB, and therefore it is not a website in StringDB;

[0174] At this point, the final detection report is: the target website is a legitimate website of unknown category;

[0175] Case (2), ExpandSet is not empty and there is a subgraph prediction probability of category n+1, that is, there is Since the values ​​of these elements all exceed ρ2, we can conclude that (a) the model's prediction of the category of the relevant subgraph is consistent with the category described by the input URL domain name; (b) the category of the input URL and the website category in StringDB share the probability, that is, the category of the URL already exists in StringDB;

[0176] At this point, the final detection report is: the target website is a legitimate website of a known category;

[0177] Case (3): At least one of the DecisionSet and ExpandSet is not empty, and there is no subgraph prediction probability of category n+1, indicating that the webpage has at least one key information and this key information is not related to the webpage domain name. Since the values ​​of these elements are all greater than ρ1 or ρ2, it can be judged that (a) the category they predict is not the category described by the input URL domain name; (b) the target website has at least one key information that is intended to imitate the key information of the legitimate website;

[0178] The final detection report at this time is: the target website is a phishing website that imitates the key information of the legitimate website;

[0179] In case (4), both DecisionSet and ExpandSet are empty, which indicates that there is no obvious logo or key text description in the webpage. For phishing attackers, webpages without key information will not serve the purpose of imitating phishing websites. That is, phishing attackers will not imitate such webpages without key information. Based on the above analysis, such webpages without key information are directly considered legitimate.

[0180] The final test report at this time is: a legal website without any key information.

Claims

1. A multimodal phishing website detection method based on a large model, characterized in that: The following steps are involved: Step 1: First extract the URL address of the target website that the user wants to visit and send it to the server for processing; The URL address includes the URL protocol, subdomain, domain name and domain suffix, port number, and resource path; Step 2: The server extracts the domain name of the URL address as the original text information and intercepts the screenshot of the web page corresponding to the URL address as the original image information. The original text information and the original image information are then preprocessed by their respective methods to obtain the text initial feature Token and the image initial feature Image; The method to obtain the initial features of the text is: A), construct an extensible brand category list StringDB; B) Preprocess the URL address text. The process is as follows: first, obtain the domain name string from the target website URL U and use it as the detection category of the target website; then, merge the detection category string of the target website with the StringDB category string to form an extended category string list L; then, combine the obtained extended category string list L with the Prompt prompt word P to generate a string list with prompt words; assuming that the Prompt prompt word used is P, the new string list with prompt words formed by combining L and P is L new ; Finally, for the string list L new Perform word segmentation processing and convert it into word segmentation form, that is, the initial feature token of the text; Step 3: For the obtained initial text feature Token and initial image feature Image, the server processes them through the Text Transformer layer and VIT layer of the neural network deep learning model CLIP to obtain text feature T and image feature I respectively; Step 4: The server calculates the similarity between the text feature T and the image feature I to obtain the image-text similarity matrix S. Then, the image-text similarity matrix is ​​processed using a normalized exponential function to obtain the image-text probability matrix P. Step 5: The graph-text probability matrix P is used to generate a decision set and an expansion set through the analysis selector. By analyzing the decision set and the expansion set, it is determined whether the target website is a phishing website; Step 6: Return the judgment result to the browser. If it is a phishing website, a warning dialog box will pop up to remind the user. If it is a legitimate website, no operation will be performed and the user can access it directly.

2. The large model-based multimodal phishing website detection method according to claim 1, characterized in that: The detailed method for obtaining the initial features of the text in step 2 is: A) Construct an extensible brand category list StringDB; after storing each brand category in StringDB, use the position of each brand category in StringDB as its label. The brand category list StringDB contains n strings; B) Preprocess the URL address text. The process is as follows: First, obtain the domain name string from the target website URL U and use it as the detection category of the target website ； Then, the detected category string of the target website is merged with the category string of StringDB to form an extended category string list L, that is, L={L1, L2, L3...L n ,L n+1 }; Among them, L i is the i-th category string stored in StringDB, where i = 1, 2, ..., n, L n+1 The target website domain name string obtained from the URL; Next, the obtained extended category string list L is combined with the prompt word P to generate a string list with prompt words; assuming that the prompt word used is P, the new string list with prompt words formed by combining L and P is L new for: L new ={[P]L1,[P]L2,…,[P]L n ,[P]L n+1 }; Finally, for the string list L new Perform word segmentation processing and convert it into word segmentation form, that is, the initial feature of the text Token: Token = {Token1, Token2, ..., Token n ,Token n+1 }; Among them, Token i For [P]L i ∈L new The result after word segmentation processing is i=1,2,…,n,n+1.

3. The large model-based multimodal phishing website detection method according to claim 1, characterized in that: The detailed method for processing the original image information to obtain the initial features of the image in step 2 is: segmenting, cropping, and padding the original image information in sequence, cutting the entire large image into several sub-images; assuming that N*M cutting is adopted, the final image is divided into N*M sub-images, and the final pre-processed initial features of the image are as follows: Image={Image1,Image2,Image3,Image4...Image n*M }。 4. The large model-based multimodal phishing website detection method according to claim 1, characterized in that: The specific method of calculating text features and image features in step 3 is: The initial feature Token text obtained in step 2 is encoded and feature extracted through the text encoding layer Text_Encoder, and then a text feature vector list T of the website brand category is generated to obtain the text features: T = Text_Encoder(Token) T={T1,T2,T3...T n ,T n+1 } The text encoding layer Text_Encoder adopts the Text Transformer architecture; The image encoding layer Image_Encoder encodes and extracts the initial features of the image obtained in step 2, and then maps the image to a higher-dimensional space to obtain image features; I = Image_Encoder(Image) I={I1,I2,I3...I N*M } The picture coding layer adopts the Vision Transformer architecture.

5. The large model-based multimodal phishing website detection method according to claim 4, characterized in that: The core of the Text Transformer architecture and the Vision Transformer architecture both use the self-attention mechanism. For a certain input original feature X, the calculation formula of multi-head self-attention is: First, the original feature X is linearly transformed to obtain the query (Query, Q), key (Key, K), and value (Value, V), where W Q ,W K ,W V is the weight matrix of the model; Q = XW Q ,K=XW K ,V=XW V ; Then, we calculate the dot product of Q and K to get the attention score and divide it by the scaling factor where d k is the dimension of the key K, and then the score is normalized by Softmax and multiplied by the value V to get the Attention score, where the Attention formula is as follows: Then, the multi-head attention replicates the self-attention mechanism multiple times, each head calculates the attention of a different representation subspace, and concatenates the results and passes another linear transformation to obtain the final output: MultiHead(Q,K,V)=Concat(head1,...,head h )W0 The expression of each head is: head i =Attention(QW i Q ,KW i K ,VW i V ); Finally, the feedforward network FFN and residual connection, as well as layer normalization LayerNorm are performed to obtain dimensional features: FFN(x)=LayerNorm(max(0,xW1+b1)W2+b2) The final output features of vector X are: X1=LayerNorm(X+MultiHead(X)) X2=LayerNorm(X1+FFN(X1)) That is, the original feature X is transformed into feature X2 after the transformer architecture.

6. The large model-based multimodal phishing website detection method according to claim 1, characterized in that: The detailed process of step 4 is as follows: Step 4.1: Each vector T in the text feature T j and each vector I in the image feature I i Perform dot product operations to obtain the similarity matrix S between each subgraph and the category prompt text: S={S1,S2,…,S N×M }; Among them, S i = {I i ● T1, I i ● T2, ……, I i ● T n , I i ● T n+1}, i = 1, 2, ……, N × M; j = 1, 2, ……, n + 1; T = {T1, T2, T3... T n , T n+1}; I = {I1, I2, I3... I N*M}; Step 4.2: Apply the Softmax Normalization function to the similarity matrix S to obtain the final probability matrix P. The similarity matrix S is normalized by the feature normalization exponential function to obtain the image-text probability matrix P, which describes the probability that each sub-image of the target image belongs to the corresponding category; P={P1,P2,…,P N×M } T in, Right now in, τ is the temperature coefficient, is the element in the similarity matrix S that represents the similarity of subgraph i belonging to category k, P i j It represents the normalized probability value when the model predicts the i-th sub-image as category j. The image-text probability matrix P has a total of N×M×(n+1) elements.

7. The large model-based multimodal phishing website detection method according to claim 1, characterized in that: The detailed method of step 5 is: Step 5.1: Generate the subgraph category probability decision set DecisionSet, and use the decision set DecisionSet to provide a basis for judging the legal websites of unknown categories; For a legitimate website, if its category is unknown, that is, the category represented by the domain name in the URL of the website is not stored in StringDB, then the probability of predicting the subgraph i containing key information as the category to which the URL domain name belongs will be significantly higher than other categories stored in StringDB. The method for generating the subgraph category probability decision set DecisionSet is: Set the highest prediction probability set MaxProbSet to save the maximum probability value of each subgraph i belonging to different categories Among them, for subgraph i, the category number max to which its maximum probability value belongs is i The search method is: By traversing each row of the image-text probability matrix P through the above formula, the maximum value of the model's predicted probability of the corresponding sub-image belonging to each category is taken to complete the selection of the highest predicted probability value of the category to which the sub-image belongs; The MaxProbSet obtained by the above process is expressed as: Based on the obtained highest prediction probability set MaxProbSet, a probability threshold ρ1 is set. By deleting the elements in MaxProbSet whose subgraph category probability value is less than ρ1, the subgraph category decision probability set DecisionSet is constructed. Step 5.2: Generate the subgraph category probability expansion set ExpandSet; For a legitimate website, if its category is known, that is, the category represented by the domain name in the URL of the website has been stored in StringDB and the category is k, then the probability of predicting the subgraph i containing the key information of the website as category k is The probability that the apportioned model predicts it as the category to which the URL domain belongs Since the predicted probability of the known class k The partial probability value of the prediction of the unknown category n+1 is amortized, then and Will not exceed max i The threshold ρ1 is preset to find the maximum probability value; The specific method of constructing the subgraph category probability decision expansion set ExpandSet is: By setting the sub-image category probability threshold ρ2, we traverse each sub-image of the target image, that is, each row of the matrix P; during the traversal process, if the category of a sub-image i is predicted to be the probability of the existing category k in StringDB and If they are close and both are in the interval [ρ2,1], then the elements representing the probability information of the two categories are stored in the ExpandSet set; The setting of ρ2 depends on the value of ρ1, that is, under normal circumstances, 2ρ2 should be close to ρ1; Step 5.3: Generate a phishing detection report; The subgraph category probability decision set DecisionSet and the subgraph category probability expansion set ExpandSet are combined to generate a detection report reflecting whether the target website is a phishing website.

8. The large model-based multimodal phishing website detection method according to claim 7, characterized in that: The phishing detection report in step 5.3 includes the following situations: Case (1), DecisionSet is not empty and there is a subgraph prediction probability of category n+1, that is, there is ...elements of this type; since the values ​​of these elements all exceed ρ1, we conclude that (a) the model's prediction of the category of the relevant subgraph is highly consistent with the category described by the input URL domain name; (b) the category of the input URL does not share probability with the website categories in StringDB, and therefore it is not a website in StringDB; At this point, the final detection report is: the target website is a legitimate website of unknown category; Case (2), ExpandSet is not empty and there is a subgraph prediction probability of category n+1, that is, there is ...such paired elements; since the values ​​of these elements all exceed ρ2, we can conclude that (a) the model's prediction of the category of the relevant subgraph is consistent with the category described by the input URL domain name; (b) the category of the input URL and the website category in StringDB share the same probability, that is, the category of the URL already exists in StringDB; At this point, the final detection report is: the target website is a legitimate website of a known category; Case (3): At least one of DecisionSet and ExpandSet is not empty, and there is no subgraph prediction probability of category n+1, indicating that the webpage has at least one key information and the key information is irrelevant to the webpage domain name; since the values ​​of these elements are all greater than ρ1 or ρ2, it can be judged that (a) the category they predict is not the category described by the input URL domain name; (b) the target website has at least one key information that is intended to imitate the key information of the legitimate website; The final detection report at this time is: the target website is a phishing website that imitates the key information of the legitimate website; In case (4), both DecisionSet and ExpandSet are empty, which indicates that there is no obvious logo or key text description in the webpage. For phishing attackers, webpages without key information will not serve the purpose of imitating phishing websites. That is, phishing attackers will not imitate such webpages without key information. Based on the above analysis, such webpages without key information are directly considered legitimate. The final test report at this time is: a legal website without any key information.

Images