Detection device, detection method, and detection program

The detection device uses clustering and machine learning to generate matching rules for identifying malicious domain names, overcoming the limitations of conventional methods by accurately detecting complex phishing domains, thereby preventing phishing attempts.

WO2024248154A1PCT designated stage expired Publication Date: 2024-12-05NT T INC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/020115
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-06-01
Filing Date
2024-05-31
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Conventional methods fail to accurately detect domain names used for phishing, especially those generated by combining multiple domain squatting techniques, which can evade existing detection technologies by having different structures or fewer common characters.

Method used

A detection device that includes a clustering unit to classify domain names based on feature amounts, generating matching rules for each cluster, and using a determination unit to identify malicious domain names by calculating similarity with known phishing domains, employing a machine learning model like BERT fine-tuned with Triplet Loss for feature extraction and clustering, and determining domain names with similarity exceeding a threshold as malicious.

Benefits of technology

The solution enables accurate detection of complex phishing domain names created by combining multiple domain squatting methods, allowing for early identification and prevention of phishing attempts, effectively capturing linguistic similarities and meta-information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024020115_05122024_PF_FP_ABST
    Figure JP2024020115_05122024_PF_FP_ABST
Patent Text Reader

Abstract

A detection device (10) includes: a clustering unit (153); a rule generation unit (154), and a determination unit (156). The clustering unit (153) classifies a plurality of domain names that are known to be malicious into a plurality of clusters on the basis of the feature amounts of the plurality of domain names. The rule generation unit (154) generates a matching rule for each of the plurality of clusters. The determination unit (156) determines whether a detection target domain name is malicious by using the matching rule generated for, from among the plurality of clusters, a cluster that includes a domain name for which the degree of similarity with the detection target domain name is at least a threshold.
Need to check novelty before this filing date? Find Prior Art

Description

Detection device, detection method, and detection program

[0001] The present invention relates to a detection device, a detection method, and a detection program.

[0002] A type of attack known as phishing has been known in the past. Attackers use phishing sites to trick victims into believing they are on legitimate websites, and then ask them to enter personal information or make monetary payments.

[0003] When setting up phishing sites, attackers often acquire domain names that are similar to those of legitimate websites, a method of creating such domain names known as domain squatting.

[0004] Attackers use domain names obtained through domain squatting as the domain names of phishing sites or as URLs in email messages. Domain squatting involves substituting similar characters, such as "o" for "0" or "i" for "l," or substituting nearby characters on the keyboard that are likely to be typos, to obtain similar domain names.

[0005] Attackers may also acquire domain names that include the brand name of a legitimate service (e.g., BRAND.test) (e.g., BRAND-login.test) or that have the domain name of a legitimate website set as a subdomain (e.g., BRAND.test.example.com).

[0006] One existing method for detecting domain names obtained using domain squatting is to use the domain name of a legitimate website as input and generate similar domain names using pre-registered character substitution patterns (see, for example, Non-Patent Document 1).

[0007] There is also a method for detecting domain names that include the brand names of legitimate services (see, for example, Non-Patent Document 2). Both methods are methods for detecting domain names that are similar to the domain names of legitimate websites.

[0008] Agten P, Joosen W, Piessens F, Nikiforakis N (2015) Seven Months' Worth of Mistakes: A Longitudinal Study of Typosquatting Abuse. In: 22nd Annual Network and Distributed System Security Symposium, NDSS 2015, San Diego, California, USA, February 8-11, 2015. The Internet SocietyKintis P, Miramirkhani N, Lever C, Chen Y, Gomez RR, Pitropakis N, Nikiforakis N, Antonakakis M (2017) Hiding in Plain Sight: A Longitudinal Study of Combosquatting Abuse. In: Thuraisingham B, Evans D, Malkin T, Xu D (eds) Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS 2017, Dallas, TX, USA, October 30 - November 03, 2017. ACM, pp 569-586

[0009] However, conventional methods may not be able to accurately detect domain names used in phishing.

[0010] In recent years, domain names have been generated by combining multiple domain squatting methods in order to increase the efficiency of creating large numbers of domain names and to evade existing detection techniques.

[0011] For example, methods are known for constructing domain names based on specific naming rules, such as setting a subdomain as a string that is a modified version of a regular domain name, keeping the first few characters of a regular domain name and changing the remaining characters, or combining other words.

[0012] Domain names generated in this way may bear a strong resemblance to legitimate domain names, but may have a different structure or fewer characters in common with legitimate domain names.

[0013] In order to solve the above-mentioned problems and achieve the object, the detection device is characterized by having a clustering unit that classifies a plurality of domain names known to be malicious into a plurality of clusters based on feature quantities of the domain names; a rule generation unit that generates a matching rule for each of the plurality of clusters; and a determination unit that determines whether the domain name to be detected is malicious or not, using the matching rule generated for a cluster among the plurality of clusters that includes a domain name whose similarity with the domain name to be detected is equal to or greater than a threshold.

[0014] According to the present invention, domain names used in phishing can be detected with high accuracy.

[0015] FIG. 1 is a diagram illustrating an example of the configuration of a detection device according to a first embodiment. FIG. 2 is a diagram illustrating processing by the detection device. FIG. 3 is a diagram illustrating fine tuning. FIG. 4 is a diagram illustrating an example of triplet data. FIG. 5 is a diagram illustrating feature extraction processing. FIG. 6 is a diagram illustrating an example of feature quantities. FIG. 7 is a diagram illustrating an example of matching rules. FIG. 8 is a diagram illustrating an example of matching rules. FIG. 9 is a diagram illustrating an example of similarity. FIG. 10 is a flowchart illustrating the flow of processing for generating matching rules. FIG. 11 is a flowchart illustrating the flow of processing for detecting malicious domain names. FIG. 12 is a diagram illustrating an example of a computer that executes a program.

[0016] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of a detection device, a detection method, and a detection program according to the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention is not limited to the following embodiments.

[0017] [Configuration of First Embodiment] The configuration of a detection device according to a first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of a detection device according to the first embodiment.

[0018] As shown in FIG. 1 , the detection device 10 includes a communication unit 11 , an input unit 12 , an output unit 13 , a storage unit 14 , and a control unit 15 .

[0019] The communication unit 11 performs data communication with other devices via a network. For example, the communication unit 11 is a network interface card (NIC).

[0020] The input unit 12 receives input of data. For example, the input unit 12 is an interface connected to input devices such as a mouse and a keyboard.

[0021] The output unit 13 outputs data. The output unit 13 is an interface connected to an output device such as a display and a speaker.

[0022] The storage unit 14 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), an optical disk, etc. Note that the storage unit 14 may also be a data-rewritable semiconductor memory such as a random access memory (RAM), a flash memory, or a non-volatile static random access memory (NVSRAM). The storage unit 14 stores an operating system (OS) and various programs executed by the detection device 10.

[0023] The storage unit 14 stores model information 141 and rule information 142. The model information 141 is parameters for constructing a machine learning model. For example, the model information 141 is parameters such as weights and biases of a deep neural network. The rule information 142 is a set of matching rules, which will be described later.

[0024] The control unit 15 controls the entire detection device 10. The control unit 15 is, for example, an electronic circuit such as a central processing unit (CPU), a micro processing unit (MPU), or a graphics processing unit (GPU), or an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0025] The control unit 15 also has an internal memory for storing programs that define various processing procedures and control data, and executes each process using the internal memory. The control unit 15 also functions as various processing units when various programs are run. For example, the control unit 15 functions as a feature extraction unit 151, an adjustment unit 152, a clustering unit 153, a rule generation unit 154, a similarity calculation unit 155, and a determination unit 156.

[0026] The feature extraction unit 151 extracts features from domain names using a machine learning model constructed based on the model information 141. The adjustment unit 152 tunes the machine learning model. The clustering unit 153 classifies multiple domain names into multiple clusters based on the features. The rule generation unit 154 generates a matching rule for each cluster. The similarity calculation unit 155 calculates the similarity between the domain name to be detected and the domain names included in the cluster. The determination unit 156 determines whether the domain name to be detected is malicious using a matching rule generated for a cluster, out of the multiple clusters, that includes a domain name whose similarity with the domain name to be detected is equal to or greater than a threshold.

[0027] The processing of the detection device 10 will be described with reference to Fig. 2. Fig. 2 is a diagram illustrating the processing of the detection device. As shown in Fig. 2, the detection device 10 performs a first processing and a second processing. The first processing is a processing for generating matching rules. The second processing is a processing for detecting malicious domain names (for example, domain names used for phishing) using the matching rules.

[0028] 2, in the first process, phishing domain names are input as known malicious data to the detection device 10. The phishing domain names are a list (blacklist) of domain names known to have been used for phishing.

[0029] The feature extraction unit 151 extracts features of the input phishing domain name using a machine learning model (hereinafter simply referred to as a model) constructed from the model information 141. The feature extraction unit 151 can extract features using a model that converts character strings into vector data. For example, the feature extraction unit 151 extracts features using BERT (Bidirectional Encoder Representations from Transformers), a deep learning method.

[0030] A model such as BERT may have been trained using a large corpus to analyze the linguistic features of general sentences and conversations. Therefore, the tuning unit 152 fine-tunes the model such as BERT to optimize it for analyzing domain names, which are short character strings.

[0031] Fine-tuning is a method of retraining a part of a trained model (for example, only the output layer) using a prepared dataset so that it can be adapted to a specific task.

[0032] In fine tuning, learning is performed using a data set that matches a set loss function. For example, the loss function is Triplet Loss. As shown in FIG. 3, the adjustment unit 152 adjusts the trained model based on triplet data and outputs the adjusted trained model. FIG. 3 is a diagram illustrating fine tuning.

[0033] As shown in Fig. 4, triplet data is a set of data consisting of a sentence that is an anchor, a sentence that is similar to the anchor (positive), and a sentence that is not similar (negative). Fig. 4 is a diagram showing an example of triplet data.

[0034] Triplet Loss is a loss function used for learning to reduce the distance between similar data (Anchor and Positive) and increase the distance between dissimilar data (Anchor and Negative). Note that the distance here refers to the distance between vectors representing features (cosine similarity, Euclidean distance, etc.).

[0035] For example, an adjusted BERT fine-tuned using the triplet data shown in Fig. 4 will output similar features for the Anchor domain name "exaaample.test" and the Positive domain name "exaaamplee.test". On the other hand, the adjusted BERT will output dissimilar features for the Anchor domain name "exaaample.test" and the Negative domain name "negative1.test".

[0036] As shown in Fig. 5, the feature extraction unit 151 extracts features from multiple domain names known to be malicious, using a BERT adapted to the domain name through fine tuning using Triplet Loss. Fig. 5 is a diagram illustrating the feature extraction process.

[0037] The feature amount is expressed as vector data as shown in Fig. 6. Fig. 6 is a diagram showing an example of the feature amount.

[0038] The clustering unit 153 classifies multiple domain names into multiple clusters based on the feature quantities of multiple domain names known to be malicious. The clustering unit 153 performs clustering based on the feature quantities, which are vector data. The clustering unit 153 may perform clustering using an algorithm such as DBSCAN (Density-based Spatial Clustering of Applications with Noise), which does not require the number of clusters to be set.

[0039] Furthermore, the clustering unit 153 excludes domain names that do not belong to any cluster as a result of the clustering, so that the remaining domain names belong to one of the clusters.

[0040] 2 , the rule generation unit 154 generates a matching rule for each of the plurality of clusters. For example, the rule generation unit 154 generates a matching rule for a character string at any level included in each of the plurality of domain names and the number of characters in each of the plurality of domain names.

[0041] 7 and 8 are diagrams showing examples of matching rules. Cluster A and Cluster B are clusters classified by the clustering unit 153.

[0042] As shown in Figure 7, the multiple domain names classified into Cluster A have in common the top-level domain "test." The multiple domain names classified into Cluster A also have in common the number of characters, which is 14. On the other hand, the multiple domain names classified into Cluster A have in common the second-level domains, which include "exaaample," "exaiample," etc.

[0043] Therefore, the rule generation unit 154 generates a matching rule A for cluster A, which states that the top level domain is "test" and the number of characters is "14."

[0044] As shown in Figure 8, the multiple domain names classified in Cluster B have in common the top-level domain "test." Also, the multiple domain names classified in Cluster B have in common the second-level domain "example." However, the multiple domain names classified in Cluster B do not have in common the number of characters.

[0045] Therefore, the rule generation unit 154 generates a matching rule B for cluster B, which specifies that the top-level domain is "test" and the second-level domain is "example."

[0046] [Second Process] Next, the second process will be described with reference to Fig. 2. As shown in Fig. 2, in the second process, domain names acquired from a passive Domain Name System (DNS), a Certificate Transparency (CT) log, and a zone file are input to the detection device 10 as detection target data.

[0047] The domain names of the detection target data are newly registered or observed domain names, and it may be unknown whether they are being used for phishing.

[0048] Passive DNS is data obtained by observing DNS cache servers. CT logs are newly acquired domain names for which certificate authorities have published certificates. Zone files are files set in authoritative DNS servers that associate domain names with IP addresses.

[0049] The feature extraction unit 151 extracts the feature amount of the domain name included in the detection target data in the same way as in the first process.

[0050] The similarity calculation unit 155 calculates the similarity (cosine similarity, Euclidean distance, etc.) between the feature amount of each domain name included in the detection target data and the feature amount of each clustered domain name included in the known malicious data.

[0051] 9 is a diagram showing an example of similarity. "phishing001.test" is a domain name included in known phishing domain names and has been clustered in the first process. "phishing011.test" and "phishing002.test" are domain names included in the detection target data.

[0052] The similarity calculation unit 155 calculates the similarity between "phishing001.test" and "phishing011.test" to be "0.9." The similarity calculation unit 155 also calculates the similarity between "phishing001.test" and "phishing002.test" to be "0.8."

[0053] Then, the determination unit 156 determines whether the domain name to be detected is malicious by using a matching rule generated for a cluster among the multiple clusters that includes a domain name whose similarity with the domain name to be detected is equal to or greater than a threshold.

[0054] 9, if the threshold is 0.85, the determination unit 156 uses the matching rule to determine whether "phishing011.test", whose similarity exceeds the threshold, is malicious. On the other hand, the determination unit 156 does not use the matching rule to determine whether "phishing002.test", whose similarity is equal to or less than the threshold.

[0055] In this way, the determining unit 156 determines whether the domain name to be detected obtained from the CT log, passive DNS, or zone file is malicious or not.

[0056] For example, assume that "www.example.co.test" is known as a canonical domain name. In this case, the following domain names can be detected: www.exampppliie.exampppllleaee.co.test.abcdeg.test www.examppplss.examplllaaeeee.co.test.bcdegh.test www.examppaaie.examppaaeelee.co.test.cdeghi.test www.examppeeie.examppaaleeee.co.test.deghij.test

[0057] [Processing Flow of First Embodiment] The first processing, that is, the processing flow for generating matching rules, will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the processing flow for generating matching rules.

[0058] 10 , first, the detection device 10 performs fine tuning of the trained model (step S101). For example, the detection device 10 performs fine tuning of the BERT using Triplet Loss.

[0059] Next, the detection device 10 uses the fine-tuned model to extract feature quantities from known malicious domain names (step S102). The detection device 10 extracts the feature quantities as vector data.

[0060] Next, the detection device 10 performs clustering on the extracted domain names and classifies the domain names into clusters (step S103). The detection device 10 may exclude domain names that have not been classified into any cluster.

[0061] Then, the detection apparatus 10 generates a matching rule common to each cluster (step S104). For example, the detection apparatus 10 generates the matching rule based on the number of characters in the top-level domain, the second-level domain, or the domain name.

[0062] The second process, that is, the process flow for detecting malicious domain names, will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the process flow for detecting malicious domain names.

[0063] 11, first, the detection device 10 extracts features from the domain name to be detected using a model (step S201). At this time, the detection device 10 uses the same model as in the first process.

[0064] Next, the detection device 10 calculates the similarity between the extracted feature amount and the feature amount of a known malicious domain name (step S202). For example, the detection device 10 calculates the distance between vectors as the similarity.

[0065] The detection device 10 then uses the matching rule to detect malicious domain names from among the domain names to be detected whose similarity exceeds the threshold (step S203). The detection device 10 outputs, from among the domain names whose similarity exceeds the threshold, domain names that match the matching rule as domain names that may be used for phishing.

[0066] [Effects of the First Embodiment] As described above, the clustering unit 153 classifies multiple domain names into multiple clusters based on feature quantities of multiple domain names known to be malicious. The rule generation unit 154 generates a matching rule for each of the multiple clusters. The determination unit 156 determines whether the domain name to be detected is malicious by using the matching rule generated for a cluster, among the multiple clusters, that includes a domain name whose similarity with the domain name to be detected is equal to or greater than a threshold.

[0067] In this way, the detection device 10 detects target domain names that are similar to known phishing domain names using matching rules that are suited to the phishing domain names. As a result, the detection device 10 can accurately detect even complex phishing domain names that are created by combining multiple domain squatting methods.

[0068] Furthermore, the detection device 10 can detect domain names acquired by attackers before they are used as phishing sites or in the early stages of their use, and by using these domain names, it is possible to prevent users from accessing phishing sites.

[0069] The feature extraction unit 151 extracts features from multiple domain names known to be malicious using BERT, which is adapted to the domain names through fine tuning using Triplet Loss. The clustering unit 153 classifies the multiple domain names into multiple clusters based on the features extracted by the feature extraction unit 151. Phishing domain names may be created according to a common naming rule. The detection device 10 can capture linguistic similarities between domain names by using a machine learning model such as BERT.

[0070] The rule generation unit 154 generates matching rules for character strings at any level included in each of the multiple domain names and the number of characters in each of the multiple domain names, thereby enabling the detection device 10 to perform detection using both linguistic information and meta-information such as the number of characters.

[0071] The determination unit 156 determines whether the domain name to be detected obtained from the CT log, passive DNS, or zone file is malicious, thereby enabling efficient acquisition of the domain name to be detected.

[0072] [System Configuration, etc.] The components of each device shown in the figures are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of the devices can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0073] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0074] [Program] In one embodiment, the detection device 10 can be implemented by installing a program that executes the above-described learning process as package software or online software on a desired computer. For example, by executing the above-described program on an information processing device, the information processing device can function as the detection device 10. The information processing device referred to here includes desktop and notebook personal computers. Other information processing devices also include mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as slate terminals such as PDAs (Personal Digital Assistants).

[0075] The detection device 10 may also be implemented as a server device that provides services related to the above processing to a client terminal device used by a user. For example, the server device may be implemented as a server device that receives a domain name to be detected as an input and outputs a determination result as to whether the domain name is malicious. In this case, the server device may be implemented as a web server or as a cloud that provides services related to the above processing through outsourcing.

[0076] 12 is a diagram showing an example of a computer that executes a program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0077] The memory 1010 includes a read-only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores a boot program such as a basic input / output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0078] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the detection device 10 is implemented as a program module 1093 in which computer-executable code is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing processes similar to those of the functional configuration of the detection device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0079] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.

[0080] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0081] REFERENCE SIGNS LIST 10 Detection device 11 Communication unit 12 Input unit 13 Output unit 14 Storage unit 15 Control unit 141 Model information 142 Rule information 151 Feature extraction unit 152 Adjustment unit 153 Clustering unit 154 Rule generation unit 155 Similarity calculation unit 156 Determination unit

Claims

1. A detection device comprising: a clustering unit that classifies a plurality of domain names known to be malicious into a plurality of clusters based on the features of the domain names; a rule generation unit that generates a matching rule for each of the plurality of clusters; and a determination unit that determines whether the domain name to be detected is malicious or not using the matching rule generated for a cluster among the plurality of clusters that includes a domain name whose similarity with the domain name to be detected is equal to or greater than a threshold.

2. The detection device according to claim 1, further comprising a feature extraction unit that extracts features from the multiple domain names known to be malicious using a BERT adapted to the domain names by fine-tuning using Triplet Loss, and the clustering unit classifies the multiple domain names into multiple clusters based on the features extracted by the feature extraction unit.

3. The detection device according to claim 1, characterized in that the rule generation unit generates matching rules regarding character strings of any level contained in each of the multiple domain names and the number of characters in each of the multiple domain names.

4. The detection device according to claim 1, characterized in that the judgment unit judges whether the domain name to be detected obtained from a CT log, a passive DNS, or a zone file is malicious or not.

5. A detection method executed by a detection device, comprising: a clustering step of classifying a plurality of domain names known to be malicious into a plurality of clusters based on the features of the domain names; a rule generation step of generating a matching rule for each of the plurality of clusters; and a determination step of determining whether the domain name to be detected is malicious or not, using the matching rule generated for a cluster among the plurality of clusters that includes a domain name whose similarity with the domain name to be detected is equal to or greater than a threshold.

6. A detection program that causes a computer to execute the following steps: a clustering step of classifying a plurality of domain names known to be malicious into a plurality of clusters based on the features of the domain names; a rule generation step of generating a matching rule for each of the plurality of clusters; and a determination step of determining whether the domain name to be detected is malicious or not, using the matching rule generated for a cluster among the plurality of clusters that includes a domain name whose similarity with the domain name to be detected is equal to or greater than a threshold.

Citation Information

Patent Citations

  • System and methods of detecting malicious elements of web pages

    JP2018063694A

  • System and method for detecting phishing-domains in a set of domain name system (DNS) records

    US20220377107A1