Training method for domain-name generation model, phishing website detection method, and related apparatus

Abstract

Provided in the embodiments of the present application are a training method for a domain-name generation model, a phishing website detection method, and a related apparatus. The training method comprises: matching serial numbers for a top-level domain name of a phishing website domain name and for characters of a second-level domain name thereof, so as to generate domain-name vectors; extracting semantic features of the domain-name vectors; on the basis of the top-level domain name and the characters of the second-level domain name, performing clustering to obtain common features; and by means of a generative adversarial network, performing training on the basis of a target domain-name vector, domain-name semantic feature vectors and the common features, and by means of the common features, guiding a generator to generate a potential phishing website domain name on the basis of the target domain-name vector and the domain-name semantic feature vectors. Reliable a priori knowledge is provided by means of combining domain-name similarity and domain-name semantic feature vectors; and by means of clustering, common features covering the similarities of real phishing website domain names in terms of structure, grammar and semantics are obtained, and the common features are used to guide a generative adversarial network to generate a similar domain name. Therefore, the present application has the advantages of a wide detection range, high timeliness and high practicability.

Description

Domain name generation model training method, phishing website discovery method and related devices Technical Field

[0001] The embodiments of the present application relate to the field of data processing, and in particular to a training method for a domain name generation model, a phishing website discovery method, and related devices. Background Art

[0002] At present, the detection method of phishing website domain names is mainly based on the character feature patterns of existing known websites. For example, it can be achieved through the following methods: (1) using a detection method oriented to visual similarity to detect phishing websites based on factors such as text content, HTML elements and cascading style sheets; (2) detecting phishing websites by extracting multi-scale semantic information of invisible text such as URL, title, body and HTML tags; (3) detecting phishing websites by combining features such as domain name usage time, IP address change speed, TTL value, domain name search volume, etc.; (4) searching for domain name keywords through search engines and screening according to heuristic rules defined by character features.

[0003] However, the detection method based on the characteristics of the domain name itself and the domain name historical data can only detect known domain names, and it is difficult to identify phishing website domain names with a short survival time. The detection effect on newly generated phishing website domain names is weak, the processing is not fast enough, the detection range is small, the timeliness is poor, and it is difficult to accommodate and mine potential phishing domain names.

[0004] Summary of the Invention

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0006] The purpose of this application is to solve one of the technical problems existing in the relevant technology to at least a certain extent. The embodiments of this application provide a training method for a domain name generation model, a phishing website discovery method and related devices, which can generate potential domain names by using the common characteristics of real domain names, and have the advantages of a wide detection range and strong timeliness.

[0007] An embodiment of the first aspect of the present application provides a method for training a domain name generation model, comprising:

[0008] Obtain the domain name of the phishing website;

[0009] Matching the top-level domain name of the phishing website domain name and the characters obtained by segmenting the sub-domain name of the phishing website domain name to generate a domain name vector;

[0010] Performing semantic feature extraction on the domain name vector to obtain a domain name semantic feature vector;

[0011] Clustering the characters of the top-level domain and sub-domain of the phishing website domain to obtain common features;

[0012] Selecting a target domain name vector from the domain name vector using a generative adversarial network of the domain name generation model, performing generative adversarial training based on the target domain name vector, the domain name semantic feature vector, and the common features, to obtain a trained domain name generation model;

[0013] In the process of generative adversarial training, the common features are used to guide the generator of the generative adversarial network to generate potential phishing website domain names based on the target domain name vector and the domain name semantic feature vector.

[0014] An embodiment of the second aspect of the present application is a method for discovering phishing websites, comprising:

[0015] Get the domain name of the website to be tested;

[0016] Generate potential phishing website domain names through the trained website domain generation model;

[0017] Detecting a target phishing website domain name from the to-be-detected website domain names based on the potential phishing website domain name;

[0018] The website domain name generation model is trained according to the training method of the website domain name generation model described above.

[0019] According to an embodiment of the third aspect of the present application, a device for training a website domain name generation model includes:

[0020] Input module, used to obtain the domain name of the phishing website;

[0021] an encoding module for matching characters obtained by segmenting the top-level domain name of the phishing website domain name and the sub-domain name of the phishing website domain name to generate a domain name vector, and performing semantic feature extraction on the domain name vector to obtain a domain name semantic feature vector;

[0022] A clustering module, configured to cluster characters of the top-level domain name and the sub-domain name of the phishing website domain name to obtain common features;

[0023] a generative adversarial module, configured to select a target domain name vector from the domain name vectors using a generative adversarial network of the domain name generation model, and perform generative adversarial training based on the target domain name vector, the domain name semantic feature vector, and the common features to obtain a trained domain name generation model;

[0024] In the process of generative adversarial training, the common features are used to guide the generator of the generative adversarial network to generate potential phishing website domain names based on the target domain name vector and the domain name semantic feature vector.

[0025] An embodiment of the fourth aspect of the present application is an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the domain name generation model training method described above or the phishing website discovery method described above is implemented.

[0026] An embodiment of the fifth aspect of the present application is a computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the domain name generation model training method described above or the phishing website discovery method described above.

[0027] The above scheme has at least the following beneficial effects: combining domain name similarity and domain name semantic feature vectors to provide reliable and accurate prior knowledge for the generative adversarial network; by clustering real phishing website domain names, common features covering the structural, grammatical and semantic similarities of real phishing website domain names are obtained, and the common features are used to guide the generative adversarial network to generate similar potential phishing website domain names, which has the advantages of wide detection range, strong timeliness and strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are used to provide a further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0029] FIG1 is a diagram showing the steps of a method for training a domain name generation model;

[0030] FIG2 is a step diagram of step S200;

[0031] FIG3 is a step diagram of step S400;

[0032] FIG4 is a diagram showing the steps for determining the initial cluster centers;

[0033] FIG5 is a diagram showing the steps of clustering domain name feature vectors based on initial cluster centers to obtain common features;

[0034] FIG6 is a diagram showing the steps of selecting a target domain name vector from a domain name vector by a generator of a generative adversarial network, and generating a candidate domain name vector based on the target domain name vector, the domain name semantic feature vector, and the common features;

[0035] FIG7 is a diagram showing the steps of determining the authenticity of a candidate domain name vector through a discriminator and obtaining a discrimination result;

[0036] FIG8 is a diagram showing the steps of a method for discovering a phishing website;

[0037] FIG9 is a structural diagram of a domain name generation model;

[0038] Figure 10 is a schematic diagram of the principle of the encoding component of the Transformer model;

[0039] Figure 11 is a schematic diagram of the principle of generating an adversarial network;

[0040] FIG12 is a structural diagram of a training device for a domain name generation model;

[0041] FIG13 is a structural diagram of an electronic device. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0043] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and the like in the specification, claims, or accompanying drawings are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0044] The embodiments of the present application are further described below with reference to the accompanying drawings.

[0045] An embodiment of the present application provides a domain name generation model; referring to FIG9 , the domain name generation model includes an encoding network 10, a clustering network 20, and a generative adversarial network 30. This domain name generation model clusters and analyzes real phishing domain names, and generates potential phishing website domain names based on domain similarity and domain semantic feature vectors, facilitating the detection of hidden phishing websites.

[0046] An embodiment of the present application provides a method for training a domain name generation model, which is applied to training the above-mentioned domain name generation model.

[0047] 1 , the domain name generation model is trained according to the following training method.

[0048] Step S100, obtaining the domain name of the phishing website.

[0049] For step S100, while ensuring the security and reliability of data specifications, a packet capture tool is used to capture network packets to collect data sets, and the collected traffic data packets are analyzed to construct a data set consisting of real phishing website domain names.

[0050] Step S200 , generating a domain name vector by matching the top-level domain name of the phishing website domain name and the character numbers obtained by segmenting the sub-domain name of the phishing website domain name.

[0051] Step S200 is to train the encoding network 10. Referring to FIG2 , the top-level domain name of the phishing website domain name and the character matching number obtained by segmenting the sub-domain name of the phishing website domain name are used to generate a domain name vector, including but not limited to the following steps:

[0052] Step S210, matching numbers between the top-level domain name and the characters obtained by segmenting the sub-domain name;

[0053] Step S220: Concatenate the top-level domain name, the characters obtained by segmenting the sub-domain name, and the numbers accordingly to obtain an initial domain name vector;

[0054] Step S230 , supplementing the initial domain name vector with a null character so that the length of the initial domain name vector is unified to a preset length, thereby obtaining a domain name vector.

[0055] A domain name is composed of two or more groups of ASCII or language characters, separated by periods. The rightmost group is called the top-level domain or first-level domain, the second-to-last group is called the second-level domain, the third-to-last group is called the third-level domain, and so on. Sub-domains include second-level domains and third-level domains.

[0056] Because the top-level domain names of most phishing website domain names are unique and indivisible, the top-level domain name needs to be treated as a whole during encoding. The individual characters and space characters generated by segmenting the sub-domain names of the phishing website domain names in the dataset are combined with the top-level domain names of the phishing website domain names to form a character encoding table. The characters in the character encoding table are matched and numbered, and the characters and numbers in the character encoding table are concatenated one-to-one to obtain initial domain name vectors of varying lengths. The initial domain name vectors are supplemented with space characters to unify the length of the initial domain name vectors to a preset length, resulting in a domain name vector of uniform length composed of character codes.

[0057] Step S300: extract semantic features from the domain name vector to obtain a domain name semantic feature vector.

[0058] Step S300 is to train the encoding network 10. The domain name vector is re-encoded through the Transformer model to extract the semantic features of the domain name.

[0059] The Transformer model is an Encoder-Decoder architecture, which includes encoding components and decoding components.

[0060] The encoding component of the Transformer model consists of multiple encoders, each of which has the same structure but uses different weight parameters. Referring to Figure 10, each encoder contains a multi-head attention mechanism layer and a feedforward neural network, followed by a residual connection and a connection normalization module.

[0061] In other embodiments, the Transformer model may be a BERT (Bidirectional Encoder Representations from Transformers) network.

[0062] The encoder passes the domain vector to the multi-head attention mechanism layer, then to the feedforward neural network, and finally passes the output to the next encoder. The last encoder outputs a set of attention vectors Key and Value.

[0063] The domain vector passes through a multi-head attention mechanism layer, which allows the encoder to encode specific characters while also focusing on other characters in the domain vector. The multi-head attention mechanism layer maps the query, key, and value through different linear transformations, then concatenates the outputs of the different self-attention layers; finally, it performs another linear transformation.

[0064] The feedforward neural network is a fully connected feedforward neural network. The character at each position will pass through the feedforward neural network separately. It consists of two linear transformations, namely two fully connected layers.

[0065] Residual connections can better propagate gradients and help alleviate the problem of gradient vanishing; layer normalization of the normalization module can reduce internal covariate shift and speed up training.

[0066] The decoding component of the Transformer model consists of multiple decoders. The attention vector output by the encoding component is used in the Encoder-Decoder Attention layer of each decoder, which helps the decoder focus on the appropriate position of the input sequence.

[0067] The Encoder-Decoder Attention layer constructs the Query matrix using the output of the previous layer, while the Key and Value matrices come from the output of the encoder stack.

[0068] The decoder repeats the following process until it outputs a terminator, indicating that the decoder has completed its output. Positional encoding is added to the decoder input to indicate the position of each word. The output of each step is fed into the first decoder at the next time step, which then outputs the decoded result. The output of the final time step is used as the feature vector for the domain name.

[0069] This helps maintain semantic consistency with the generator while obtaining a global understanding of the entire domain name sequence and abstract high-level feature representation; this can provide the generator with more meaningful prior knowledge, help promote semantic consistency in the generation process and improve the quality of domain name generation.

[0070] A combination of character numbering and Transformer encoding is used to extract domain name features from two aspects: domain name similarity and domain name semantic information.

[0071] Step S400: Clustering is performed based on the characters of the top-level domain name and the sub-domain name of the phishing website domain name to obtain common features.

[0072] Step S400 is to train the clustering network 20. Referring to FIG3 , clustering is performed based on the characters of the top-level domains and sub-domains of the phishing website domain names to obtain common features, including but not limited to the following steps:

[0073] Step S410: extracting features from the top-level domain to obtain a top-level domain feature vector, extracting features from characters of the sub-level domain to obtain a sub-level domain feature vector, and combining the top-level domain feature vector and the sub-level domain feature vector to obtain a domain feature vector;

[0074] Step S420: clustering is performed based on the domain name feature vectors to obtain common features.

[0075] In step S410, features are extracted from the top-level domains to obtain a top-level domain feature vector. Specifically, all top-level domains are arranged sequentially to form a top-level domain sequence. A target top-level domain is selected from the top-level domains, and characters in the top-level domain sequence corresponding to the target top-level domain are converted to 1s, and characters in the top-level domain sequence not corresponding to the target top-level domain are converted to 0s, thereby obtaining a binary vector with a length equal to the total number of top-level domains. This binary vector is used as the top-level domain feature vector for the target top-level domain. All top-level domains are converted into top-level domain feature vectors in this manner.

[0076] A subdomain feature vector is obtained based on character extraction features of the subdomain, specifically by: counting the frequency of character substrings with a length of a preset character substring length in the subdomain character string; forming a substring list from character substrings with a frequency greater than a preset frequency threshold and a length of the preset character substring length; selecting a target subdomain from the subdomains, determining the position of the target subdomain at which the character substring in the substring list appears as the target subdomain position, and setting the value of the vector component corresponding to the target subdomain position to the frequency of the character substring in the substring list to construct a subdomain feature vector for the target subdomain; wherein the length of the subdomain feature vector is the total number of character substrings in the substring list. All subdomains are converted into subdomain feature vectors in this manner.

[0077] Among them, when the preset character substring length n is 3, the clustering effect is the best; of course, in other embodiments, the preset character substring length can also take other values.

[0078] The top-level domain name feature vector and the sub-domain name feature vector are combined to obtain the domain name feature vector.

[0079] Step S420, clustering the domain name feature vectors to obtain common features, includes: determining an initial cluster center, clustering the domain name feature vectors according to the initial cluster center, and obtaining common features.

[0080] 4 , determining the initial cluster center includes but is not limited to the following steps:

[0081] Step S4211, selecting a first cluster center from the domain name feature vector;

[0082] Step S4212: Calculate the shortest distance between the domain name feature vector and the first cluster center;

[0083] Step S4213, calculating the probability of the domain name feature vector based on the shortest distance;

[0084] Step S4214, constructing a set of candidate cluster centers based on the domain name feature vector according to probability;

[0085] Step S4215: obtaining a sampling weight of the domain name feature vector according to the probability of the domain name feature vector, the number of first cluster centers, and the total number of domain name feature vectors;

[0086] Step S4216: Select an initial cluster center from the set of candidate cluster centers according to the sampling weight.

[0087] A sample is randomly selected from the data set with the domain name feature vector as the sample as the first cluster center, that is, the first cluster center C.

[0088] For each sample i, the shortest distance D(i) between it and the first cluster center is calculated, where the shortest distance D(i) can be expressed by the following formula: D(i)=||iC||.

[0089] Replicate multiple samples of the cluster center. For each sample i, calculate the shortest distance D(i) between it and all replicated cluster centers. Calculate the probability p of the domain name feature vector based on the shortest distance. The probability p can be expressed as follows: Where α is a constant; is the total number of samples in the dataset.

[0090] Each sample i is added to the candidate cluster center set with probability p to construct the candidate cluster center set.

[0091] The sampling weight w(i) of the domain name feature vector is obtained according to the probability of the domain name feature vector, the number of the first cluster centers, and the total number of domain name feature vectors. The sampling weight w(i) of the i-th sample can be expressed by the following formula: In the formula, k represents the number of clusters, that is, the number of cluster centers.

[0092] P(i) represents the probability of sample i being selected as a candidate for the cluster center, that is, the probability of being selected as the cluster center in the initialization phase. w(i) represents the importance or sampling probability of sample i in the iteration phase.

[0093] Sampling is performed from the set of candidate cluster centers according to the sampling weights to select the initial cluster centers.

[0094] By configuring the weights, samples that are farther away can be more likely to be selected as new cluster centers during the iteration phase, thereby increasing the diversity and robustness of the algorithm.

[0095] Compared with the method of randomly initializing cluster centers, by introducing probability weights to select initial cluster centers, more representative and evenly distributed cluster centers can be selected to select the best initialization cluster centers, thereby reducing the situation where the algorithm converges to a local optimal solution.

[0096] 5 , clustering the domain name feature vectors according to the initial cluster centers to obtain common features includes but is not limited to the following steps:

[0097] Step S4221, calculating the first Euclidean distance between the domain name feature vector and the initial cluster center;

[0098] Step S4222, clustering the domain name feature vectors into clusters according to the first Euclidean distance;

[0099] Step S4223: Calculate the second Euclidean distance between the domain name feature vectors in the cluster, and determine the second cluster center of the cluster based on the second Euclidean distance;

[0100] Step S4224, determining the number of clusters according to the second Euclidean distance and the second cluster center;

[0101] Step S4225, determining a target cluster center from the second cluster centers according to the number of clusters;

[0102] Step S4226: Cluster the domain name feature vectors according to the target cluster center to obtain common features.

[0103] A small sample of phishing website domain names is randomly selected and clustered to obtain k clusters. For each phishing website domain name in the small sample, the first Euclidean distance between the domain name feature vector and the initial cluster center is calculated.

[0104] The Euclidean distance is calculated according to the following formula: Where, d ij It represents the Euclidean distance between the i-th domain name feature vector and the j-th domain name feature vector, and m is the dimension of the domain name feature vector.

[0105] The domain name feature vectors are divided into the clusters with the closest distance according to the first Euclidean distance.

[0106] For each domain name feature vector in the cluster, the average second Euclidean distance between the domain name feature vector and other domain name feature vectors is calculated, and the cluster center of the cluster is updated according to the average second Euclidean distance to obtain the second cluster center.

[0107] The optimal number of clusters K is determined in the following way. When K is smaller than the actual number of clusters, as the number of clusters K increases, the sample division will become more refined, and the degree of aggregation of each cluster will also be higher, resulting in the sum of squares of the distances between the sample and the nearest cluster center I (the sum of squared errors) gradually becoming smaller. When the decrease in I tends to be slow, the K value obtained at this time is the optimal K value. The sum of squared errors is calculated according to the following formula: Where I is the sum of the squares of the distances between the sample and the nearest cluster center, n is the total number of clustered samples, and X i is the feature vector of the i-th sample, μ j is the cluster center of cluster C.

[0108] The above operations are iterated until the clustering result is stable or the maximum number of iterations is reached, and the optimal number of clusters is obtained.

[0109] The target cluster center is determined from the second cluster centers according to the number of clusters.

[0110] All domain name feature vectors are re-clustered according to the target cluster center and the final clustering result is obtained.

[0111] It is important to note that this clustering algorithm requires parameter tuning and optimization based on the specific dataset and problem to achieve better clustering results. At the same time, for large datasets, you can consider using distributed computing and parallel algorithms to accelerate the clustering process.

[0112] Step S500: Select a target domain name vector from the domain name vector through the generative adversarial network of the domain name generation model, perform generative adversarial training based on the target domain name vector, the domain name semantic feature vector and the common features, and obtain a trained domain name generation model.

[0113] Step S500 is the training of the generative adversarial network 30. Referring to FIG11 , specifically, the discriminator of the generative adversarial network 30 is trained using real phishing website domain names. The generator selects a target domain name vector from the domain name vectors, and generates candidate domain name vectors based on the target domain name vector, the domain name semantic feature vector, and common features. The trained discriminator determines the authenticity of the candidate domain name vectors to obtain a determination result. The parameters of the generator are adjusted based on the determination result. The above steps are repeated until the maximum number of training times is reached or the training results converge, resulting in a trained discriminator and a trained generator, and thus a trained domain name generation model.

[0114] 6 , a target domain name vector is selected from the domain name vectors by a generator of a generative adversarial network, and a candidate domain name vector is generated based on the target domain name vector, the domain name semantic feature vector, and the common features, including but not limited to the following steps:

[0115] Step S511: Select a target domain name vector from the domain name vector according to the number corresponding to the random seed, convert the target domain name vector into an embedding vector, and concatenate the embedding vector and the domain name semantic feature vector to obtain a feature sequence;

[0116] Step S512: performing weighted summation on the feature sequence and the common features based on the self-attention mechanism to generate a first context vector, and obtaining a hidden state based on the first context vector;

[0117] Step S513, generating an initial candidate domain name vector according to the hidden state;

[0118] Step S514: obtaining a logarithmic probability distribution based on the initial candidate domain name vector, and determining a candidate domain name vector from the initial candidate domain name vector based on the logarithmic probability distribution.

[0119] The number corresponding to the random seed and the domain name semantic feature vector are used as input to the generator. The random seed is actually a pseudo-random seed. A pseudo-random seed is an initial random vector used to introduce randomness, thereby increasing the diversity of the generated text. In the generator, the role of the pseudo-random seed is to ensure that the generator produces different outputs each time, rather than relying strictly on the domain name semantic feature vector. The goal of the generator is to generate text sequences similar to real phishing website domain names, with the output dimension being the size of the domain name character list. In this process, the pseudo-random seed plays a key role in guiding the generation process, providing the model with some initial uncertainty and variation, making it difficult for the discriminator to distinguish between the authenticity and fiction of the generated domain name vectors, thereby improving the robustness and adversarial resistance of the generator.

[0120] The generator's task is to generate sequences that are as similar as possible to real phishing website domain vectors, making it difficult for the discriminator to distinguish between the authenticity and fictitiousness of the input domain vectors. During generative adversarial training, common features guide the generator of the generative adversarial network 30 to generate potential phishing website domain names based on the target domain vector and the domain semantic feature vector. The information obtained from the clustering covers the common features of real domain names, including structural, syntactic, and semantic similarities. This ensures that the domain names generated by the generator are somewhat similar to real phishing website domains in a specific cluster.

[0121] The generator includes an embedding layer, a GRU layer, a fully connected layer, and a LogSoftMax layer.

[0122] In the generator's embedding layer, the domain name vector contains the number, the top-level domain name corresponding to the number, and the characters separated from the sub-domain name. The number is selected based on a random seed, and a target domain name vector is selected from the domain name vector. The target domain name vector is converted into an embedding vector, providing the generator with a meaningful representation of the initial characters. This process aims to introduce semantic information into the generated domain name sequence, ensuring a certain degree of semantic coherence. This embedding vector provides a good starting point for the generator, allowing it to gradually adjust parameters during training to generate more reasonable domain names. The embedding vector and the domain name semantic feature vector are concatenated to generate a feature sequence, achieving a fixed-size and meaningful feature sequence output. Each feature sequence is a vector containing constant real values ​​that describes the domain name from different perspectives. Concatenating the embedding vector and the domain name semantic feature vector helps capture long-range dependencies and integrate these dependencies into the generated domain name sequence, improving the overall coherence of the generated domain names.

[0123] In the GRU layer of the generator, the feature sequence and common features are weighted and summed based on the self-attention mechanism to generate the first context vector, and the hidden state is obtained according to the first context vector.

[0124] The GRU layer contains multiple network units. For each network unit, the update gate of time step t is calculated according to the following formula: t =σ(W (z) x t +U (z) h t-1 ).x t is the characteristic sequence and common characteristics, when x t is input to the update gate, x t will be affected by its own weight W (z) Multiplied by. h t-1 Keep the information of the previous t-1 network units. t-1 is input to the update gate, h t-1 will be affected by its own weight U (z) The two products are added together and a sigmoid activation function is applied to compress the result to between 0 and 1. The update gate helps the network unit determine how much information from past time steps needs to be passed to the future time steps, allowing the network unit to decide how to copy all the information from past time steps and eliminate the risk of the gradient vanishing problem.

[0125] The reset gate is used to determine how much information of the past time steps is forgotten. The reset gate of the time step t is calculated according to the following formula: t =σ(W (r) x t +U (r) h t-1 ). W (r) is x t The weight of U (r) h t-1 The weight of x t and h t-1 The product of the corresponding weights is added and the sigmoid activation function is applied to compress the result to between 0 and 1.

[0126] The reset gate is used to store relevant information from the past through the memory content of the current time step, which is calculated as follows: t =tanh(Wx t +r t ⊙Uh t-1 ). t Multiply it by the weight W and h t-1 Multiply by weight U. Calculate reset gate r t With Uh t-1 This determines the information removed from the previous time step. The two products are added together and the nonlinear activation function tanh is applied to the result of the addition.

[0127] The input memory content of the current time step is used to decide to save the information of the current network unit, that is, the hidden state, and pass it to the next network unit, which decides to start from the current memory content h′ t Information collected from h t-1 The information collected in is calculated as follows: t =z t ⊙h t-1 +(1-z t )⊙h′ t .

[0128] The self-attention mechanism introduced between network units in the GRU layer effectively addresses the long-distance dependencies that are difficult for RNNs to handle. This self-attention mechanism focuses on key features within feature sequences and common features, performs a weighted sum of these features, and generates a context vector. This context vector is then used to update the hidden state for the current time step. The hidden state contains predictions for the next batch of character numbers, thus guiding the generation of subsequent domain name characters. This self-attention mechanism helps ensure that generated domain names are more contextually reasonable and better mimic the structure of real domain names.

[0129] In the fully connected layer of the generator, the context vector and the prediction information of the next batch of character numbers are converted into a vector with a dimension equal to the size of the domain name character table to generate the initial candidate domain name vector.

[0130] In the generator's LogSoftMax layer, a logarithmic probability distribution is generated based on the initial candidate domain name vectors. Initial candidate domain name vectors with large probabilities in the logarithmic probability distribution are selected as candidate domain name vectors. This ensures that the domain name vectors output by the generator have a reasonable probability distribution, making the generated domain names more naturally resemble real domain names.

[0131] The generator repeats the above steps to generate a batch of domain name vectors. By continuously learning and adjusting parameters, the generator gradually improves the quality of the generated domain names, making them closer to the real domain names in a specific cluster in terms of structure, syntax, and semantics.

[0132] 7 , the discriminator determines the authenticity of the candidate domain name vector to obtain a discrimination result, including but not limited to the following steps:

[0133] Step S521, converting the candidate domain name vector into an embedding tensor;

[0134] Step S522: extracting domain name character features from the embedded tensor to obtain a feature tensor;

[0135] Step S523: Based on the attention mechanism, an attention score tensor and a hidden state tensor with sequence information are obtained from the feature tensor, the hidden state tensor at the last moment is extracted from the hidden state tensor, a dot product operation is performed on the attention score tensor and the hidden state tensor at the last moment to obtain an attention weight tensor, and a weighted sum of the attention weight tensor and the hidden state tensor is performed to obtain a second context vector;

[0136] Step S524: obtaining the probability of the candidate domain name vector being true or false according to the second context vector, and obtaining a discrimination result according to the probability of the candidate domain name being true or false.

[0137] The task of the discriminator is to distinguish the true and false domain name vectors generated by the generator.

[0138] The discriminator consists of an embedding layer, a convolutional pooling layer, a terminal convolutional layer, a highway network layer, and a fully connected layer.

[0139] In the embedding layer of the discriminator, the candidate domain vector is converted into an embedding tensor with the dimensions of batch size × maximum domain length × convolution layer. The embedding tensor is transposed to fit the format of the convolution operation, and the size of the transposed embedding tensor is batch size × convolution layer × maximum domain length.

[0140] The discriminator's convolutional pooling layer, consisting of 30 2×2 filters and 15 3×3 filters, extracts domain name character features from the embedding tensor to produce a feature tensor. Each filter performs convolution, ReLU activation, and max pooling to extract key character features from the domain name. This process generates a tensor of size: batch size × convolutional layer × 1. By squeezing the last dimension, a feature vector of size: batch size × convolutional layer is obtained. This feature vector contains the key features of the domain name characters extracted by the convolutional pooling layer, providing the discriminator with the basis for distinguishing between real and generated domain name vectors.

[0141] In the terminal convolutional layer of the discriminator, the attention mechanism is implemented through a custom attention_3d_block function, converting the feature tensor, which originally lacks hidden states, into a hidden state tensor with sequence information, and calculating the attention score tensor. A custom lambda function is used to extract the hidden state tensor at the last moment from the hidden state tensor. The dot function is used to perform a dot product operation on the attention score tensor and the hidden state tensor at the last moment in the specific dimension [2,1] to obtain the attention weight tensor. To further focus on the key features of the domain name vector, the attention weight tensor is normalized through the activation module and converted into a probability distribution through the softmax function, thus allocating the importance of the input features. The dot function is used to perform a weighted sum of the attention weight tensor and the hidden state tensor in the specific dimension [1,1] to obtain a second context vector that integrates the importance of different positions, providing a more comprehensive semantic understanding for the discriminator.

[0142] The terminal convolutional layer further enhances the discriminator's feature extraction capability. The double-layer convolutional neural network consisting of the convolution pooling layer and the terminal convolutional layer enables the discriminator to mine the key features of the candidate domain name vector.

[0143] In the highway network layer of the discriminator, there are two layers; by adaptively selecting the amount and path of information transmission, the problem of performance degradation caused by gradient vanishing or gradient exploding as the depth of the model increases is avoided.

[0144] In the fully connected layer of the discriminator, the probability of the candidate domain name vector being true or false is determined based on the second context vector, and the discriminant result is obtained based on the probability of true or false. When the probability of true or false is greater than 0.5, the candidate domain name vector is judged to be a real phishing website domain; when the probability of true or false is less than or equal to 0.5, the candidate domain name vector is judged to be a fake phishing website domain.

[0145] Whether the training of the generative adversarial network 30 is completed is determined by judging whether the probability of true or false is stable or whether the maximum number of iterations has been reached; when the probability of true or false is not stable or has not reached the maximum number of iterations, the training is not completed, and the judgment result is fed back to the generator as a feedback signal; when the probability of true or false is stable or has reached the maximum number of iterations, the training is completed, and the trained generative adversarial network 30 is obtained.

[0146] By combining domain name similarity and domain name semantic feature vectors, reliable and accurate prior knowledge is provided to the generative adversarial network 30; by clustering real phishing website domain names, common features covering the structural, grammatical and semantic similarities of real phishing website domain names are obtained, and the common features are used to guide the generative adversarial network 30 to generate similar potential phishing website domain names, so that the generated potential phishing website domain names have the advantages of a wide detection range, strong timeliness and strong practicality.

[0147] An embodiment of the present application provides a method for discovering phishing websites.

[0148] 8 , the method for discovering a phishing website includes but is not limited to the following steps:

[0149] Step S10, obtaining the domain name of the website to be detected;

[0150] Step S20, generating potential phishing website domain names using the trained website domain name generation model;

[0151] Step S30: Detect and obtain the target phishing website domain name from the website domain names to be detected based on the potential phishing website domain name.

[0152] The website domain name generation model is trained according to the above website domain name generation model training method.

[0153] A large number of potential phishing website domain names similar to real phishing website domain names are generated through the website domain name generation model, which is conducive to quickly and accurately discovering hidden phishing websites from the massive number of website domain names to be detected on the Internet.

[0154] It is understandable that the above phishing website discovery method is also applicable to detecting and discovering domain names of bad websites, etc.

[0155] An embodiment of the present application provides a training device for a website domain name generation model.

[0156] 12 , the training apparatus includes an input module 110 , an encoding module 120 , a clustering module 130 , and a generative adversarial module 140 .

[0157] Among them, the input module 110 is used to obtain the domain name of the phishing website; the encoding module 120 is used to match the characters of the top-level domain name of the phishing website and the sub-domain name of the phishing website to generate a domain name vector, and perform semantic feature extraction on the domain name vector to obtain a domain name semantic feature vector; the clustering module 130 is used to cluster the characters of the top-level domain name and sub-domain name of the phishing website to obtain common features; the generative adversarial module 140 is used to select a target domain name vector from the domain name vector through the generative adversarial network of the domain name generation model, and perform generative adversarial training based on the target domain name vector, the domain name semantic feature vector and the common features to obtain a trained domain name generation model; in the process of generative adversarial training, the common features are used to guide the generator of the generative adversarial network to generate potential phishing website domain names based on the target domain name vector and the domain name semantic feature vector.

[0158] An embodiment of the present application provides an electronic device. Referring to FIG. 13 , the electronic device includes a memory 220, a processor 210, and a computer program stored in the memory 220 and executable on the processor 210. When the processor 210 executes the computer program, it implements the domain name generation model training method or phishing website discovery method described above.

[0159] The electronic device may be any intelligent terminal including a computer.

[0160] In general, for the hardware structure of the electronic device, the processor 210 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0161] The memory 220 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 220 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 220 and is called by the processor 210 to execute the methods of the embodiments of this application.

[0162] The input / output interface is used to realize information input and output.

[0163] The communication interface is used to realize the communication interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0164] The bus 230 transmits information between the various components of the device (such as the processor 210, memory 220, input / output interface, and communication interface). The processor 210, memory 220, input / output interface, and communication interface are connected to each other within the device through the bus 230.

[0165] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for executing the above-mentioned domain name generation model training method or phishing website discovery method.

[0166] Those skilled in the art will appreciate that all or some of the steps and systems in the method disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those skilled in the art, the term computer storage media is included in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data) and is volatile and non-volatile, removable, and non-removable. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage, or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium. In the above description of this specification, the reference terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" and the like are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0167] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0168] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0169] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0170] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0171] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. Although the embodiments of the present application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and purpose of the present application, and the scope of the present application is defined by the claims and their equivalents.

[0172] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present application, and these equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for training a domain name generation model, characterized in that: include: Obtain the domain name of the phishing website; Matching the top-level domain name of the phishing website domain name and the characters obtained by segmenting the sub-domain name of the phishing website domain name to generate a domain name vector; Performing semantic feature extraction on the domain name vector to obtain a domain name semantic feature vector; Clustering the characters of the top-level domain and sub-domain of the phishing website domain to obtain common features; Selecting a target domain name vector from the domain name vector using a generative adversarial network of the domain name generation model, performing generative adversarial training based on the target domain name vector, the domain name semantic feature vector, and the common features, to obtain a trained domain name generation model; Wherein, during the generative adversarial training process, the common features are used to guide the generator of the generative adversarial network to generate a potential phishing website domain name according to the target domain name vector and the domain name semantic feature vector; The clustering is performed based on the characters of the top-level domain name and the sub-domain name of the phishing website domain name to obtain common features, including: Extracting features from the top-level domain to obtain a top-level domain feature vector, extracting features from characters of the sub-level domain to obtain a sub-level domain feature vector, and combining the top-level domain feature vector and the sub-level domain feature vector to obtain a domain feature vector; Clustering is performed based on the domain name feature vectors to obtain common features; Extracting features from the top-level domain to obtain a top-level domain feature vector includes: Arrange the top-level domain names in sequence to form a top-level domain name sequence; Selecting a target top-level domain from the top-level domains, converting characters in the top-level domain sequence corresponding to the target top-level domain to 1, and converting characters in the top-level domain sequence not corresponding to the target top-level domain to 0, to obtain a binary vector with a length equal to the total number of top-level domains, and using the binary vector as a top-level domain feature vector for the target top-level domain; Extracting features from the characters of the sub-domain name to obtain a sub-domain name feature vector includes: Obtaining a preset character substring length and a preset frequency threshold; Counting the frequency of character substrings with a length equal to a preset character substring length in the character string of the secondary domain name; The character substrings whose frequency is greater than the preset frequency threshold and whose length is the preset character substring length form a substring list; Selecting a target sub-domain name from the sub-domain names, determining a position in the target sub-domain name where a character sub-string in the sub-string list appears as the target sub-domain name position, and setting a value of a vector component corresponding to the target sub-domain name position to the frequency of the character sub-string in the sub-string list to construct a sub-domain name feature vector for the target sub-domain name, wherein the length of the sub-domain name feature vector is the total number of character sub-strings in the sub-string list; The clustering is performed based on the domain name feature vector to obtain common features, including: Selecting a first cluster center from the domain name feature vector; Calculating the shortest distance between the domain name feature vector and the first cluster center; Calculating the probability of the domain name feature vector according to the shortest distance; Constructing a set of candidate cluster centers based on the domain name feature vector according to the probability; Obtaining a sampling weight of the domain name feature vector according to the probability of the domain name feature vector, the number of the first cluster centers, and the total number of the domain name feature vectors; Selecting an initial cluster center from the set of candidate cluster centers according to the sampling weight; The domain name feature vectors are clustered according to the initial cluster centers to obtain common features.

2. The method according to claim 1, characterized in that Generating a domain name vector by matching characters of the top-level domain name of the phishing website domain name and the sub-domain name of the phishing website domain name, includes: Matching numbers between the top-level domain name and the characters obtained by dividing the sub-domain name; Concatenate the top-level domain name, the characters obtained by segmenting the sub-domain name, and the numbers accordingly to obtain an initial domain name vector; The initial domain name vector is supplemented with a null character so that the length of the initial domain name vector is unified to a preset length, thereby obtaining a domain name vector.

3. The method according to claim 1, characterized in that Clustering the domain name feature vectors according to the initial cluster centers to obtain common features includes: Calculating a first Euclidean distance between the domain name feature vector and the initial cluster center; clustering the domain name feature vectors into clusters according to the first Euclidean distance; Calculating a second Euclidean distance between domain name feature vectors in the cluster, and determining a second cluster center of the cluster according to the second Euclidean distance; determining the number of clusters according to the second Euclidean distance and the second cluster center; determining a target cluster center from the second cluster centers according to the number of the clusters; The domain name feature vectors are clustered according to the target cluster center to obtain common features.

4. The method according to claim 1, wherein The generative adversarial network of the domain name generation model selects a target domain name vector from the domain name vector, performs generative adversarial training based on the target domain name vector, the domain name semantic feature vector, and the common features, and obtains a trained domain name generation model, including: Training the discriminator of the generative adversarial network using the phishing website domain name to obtain a trained discriminator; Selecting a target domain name vector from the domain name vectors by the generator, and generating a candidate domain name vector based on the target domain name vector, the domain name semantic feature vector and the common features; Using a trained discriminator to determine the authenticity of the candidate domain name vector, and obtaining a discrimination result; The parameters of the generator are adjusted according to the discrimination result to obtain a trained generator, and then a trained domain name generation model is obtained.

5. The method according to claim 4, characterized in that The generator of the generative adversarial network selects a target domain name vector from the domain name vector, and generates a candidate domain name vector according to the target domain name vector, the domain name semantic feature vector, and the common feature, including: Selecting a target domain name vector from the domain name vector according to the number corresponding to the random seed, converting the target domain name vector into an embedding vector, and concatenating the embedding vector and the domain name semantic feature vector to obtain a feature sequence; Performing a weighted summation of the feature sequence and the common feature based on a self-attention mechanism to generate a first context vector, and obtaining a hidden state according to the first context vector; generating an initial candidate domain name vector according to the hidden state; A logarithmic probability distribution is obtained according to the initial candidate domain name vector, and a candidate domain name vector is determined from the initial candidate domain name vector according to the logarithmic probability distribution.

6. The method according to claim 4, characterized in that The authenticity of the candidate domain name vector is determined by the trained discriminator to obtain a discrimination result, including: Convert the candidate domain name vector into an embedding tensor; Extracting domain name character features from the embedded tensor to obtain a feature tensor; Based on the attention mechanism, an attention score tensor and a hidden state tensor with sequence information are obtained from the feature tensor, the hidden state tensor at the last moment is extracted from the hidden state tensor, a dot product operation is performed on the attention score tensor and the hidden state tensor at the last moment to obtain an attention weight tensor, and a weighted sum of the attention weight tensor and the hidden state tensor is performed to obtain a second context vector; A probability of true or false of the candidate domain name vector is obtained according to the second context vector, and a discrimination result is obtained according to the true or false probability.

7. A method for discovering phishing websites, characterized in that: include: Get the domain name of the website to be tested; Generate potential phishing website domain names through the trained website domain generation model; Detecting a target phishing website domain name from the to-be-detected website domain names based on the potential phishing website domain name; The website domain name generation model is trained according to the website domain name generation model training method according to any one of claims 1 to 6.

8. A training device for a website domain name generation model, characterized in that: include: Input module, used to obtain the domain name of the phishing website; an encoding module for matching characters obtained by segmenting the top-level domain name of the phishing website domain name and the sub-domain name of the phishing website domain name to generate a domain name vector, and performing semantic feature extraction on the domain name vector to obtain a domain name semantic feature vector; A clustering module, configured to cluster characters of the top-level domain name and the sub-domain name of the phishing website domain name to obtain common features; Generate an adversarial module, configured to select a domain name from the domain name vector through a generative adversarial network of the domain name generation model. a target domain name vector, and performing generative adversarial training based on the target domain name vector, the domain name semantic feature vector, and the common features to obtain a trained domain name generation model; Wherein, during the generative adversarial training process, the common features are used to guide the generator of the generative adversarial network to generate a potential phishing website domain name according to the target domain name vector and the domain name semantic feature vector; The clustering is performed based on the characters of the top-level domain name and the sub-domain name of the phishing website domain name to obtain common features, including: Extracting features from the top-level domain to obtain a top-level domain feature vector, extracting features from characters of the sub-level domain to obtain a sub-level domain feature vector, and combining the top-level domain feature vector and the sub-level domain feature vector to obtain a domain feature vector; Clustering is performed based on the domain name feature vectors to obtain common features; Extracting features from the top-level domain to obtain a top-level domain feature vector includes: Arrange the top-level domain names in sequence to form a top-level domain name sequence; Selecting a target top-level domain from the top-level domains, converting characters in the top-level domain sequence corresponding to the target top-level domain to 1, and converting characters in the top-level domain sequence not corresponding to the target top-level domain to 0, to obtain a binary vector with a length equal to the total number of top-level domains, and using the binary vector as a top-level domain feature vector for the target top-level domain; Extracting features from the characters of the sub-domain name to obtain a sub-domain name feature vector includes: Obtaining a preset character substring length and a preset frequency threshold; Counting the frequency of character substrings with a length equal to a preset character substring length in the character string of the secondary domain name; The character substrings whose frequency is greater than the preset frequency threshold and whose length is the preset character substring length form a substring list; Selecting a target sub-domain name from the sub-domain names, determining a position in the target sub-domain name where a character sub-string in the sub-string list appears as the target sub-domain name position, and setting a value of a vector component corresponding to the target sub-domain name position to the frequency of the character sub-string in the sub-string list to construct a sub-domain name feature vector for the target sub-domain name, wherein the length of the sub-domain name feature vector is the total number of character sub-strings in the sub-string list; The clustering is performed based on the domain name feature vector to obtain common features, including: Selecting a first cluster center from the domain name feature vector; Calculating the shortest distance between the domain name feature vector and the first cluster center; Calculating the probability of the domain name feature vector according to the shortest distance; Constructing a set of candidate cluster centers based on the domain name feature vector according to the probability; Obtaining a sampling weight of the domain name feature vector according to the probability of the domain name feature vector, the number of the first cluster centers, and the total number of the domain name feature vectors; Selecting an initial cluster center from the set of candidate cluster centers according to the sampling weight; The domain name feature vectors are clustered according to the initial cluster centers to obtain common features.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for training a domain name generation model according to any one of claims 1 to 6 or the method for discovering a phishing website according to claim 7 is implemented.

10. A computer storage medium, characterized in that Computer-executable instructions are stored, and the computer-executable instructions are used to execute the domain name generation model training method according to any one of claims 1 to 6 or the phishing website discovery method according to claim 7.

Images