A threat perception system based on active domain name generation and real-time detection of malicious domain names

Abstract

The application discloses a threat perception system based on active domain name generation and real-time detection of malicious domain names, and belongs to the technical field of network security detection. The system comprises a historical threat perception module, a real-time threat perception module and a malicious detection module. In the historical threat perception module, a domain name generation module uses a self-recurrence model based on a Transformer to construct a similar domain name generation model, generates a similar domain name list of a target enterprise, and a risk assessment module detects whether the similar domain names have been registered; if yes, the similar domain names can be accessed and added to a detection list. The real-time threat perception module monitors newly registered domain names in real time, and adds domain names with high similarity to the target enterprise domain names to a suspicious domain name list. The malicious detection module detects the detection list and the suspicious domain name list, and identifies phishing websites. The domain names generated by the system can be highly similar to the real domain names of an enterprise in terms of vision and semantics, and can rapidly identify and evaluate potential domain name abuse risks.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of network security detection, and particularly relates to a threat perception system based on active domain name generation and real-time detection of malicious domain names. BACKGROUND

[0002] In the field of Internet security, attackers often use domain names similar to target domain names to implement phishing or other malicious activities.

[0003] In the field of Internet security, attackers register domain names similar to target domain names to implement phishing, data theft and other malicious activities. This attack method is called domain name impersonation attack. Attackers will choose domain names that are extremely similar in characters to the domain names of legitimate websites, take advantage of user negligence or spelling errors when entering the website address, and lure users to visit malicious websites, thereby stealing users' sensitive information.

[0004] The current domain name grabbing generation method is fixed, and the main methods are as follows:

[0005] Domain name deformation: Attackers create a seemingly similar domain name by making minor modifications to the spelling of the target website domain name, such as letter replacement, deletion, insertion or exchange of characters. For example, attackers may register domain names similar to g00gle.com or amaz0n.com. This method relies on user negligence or spelling errors when entering the domain name. Usually, attackers will choose domain names that are similar in characters and spelling to well-known brand domain names.

[0006] Similar character utilization: Some attackers will take advantage of the similarity between different character sets, such as using Latin letters and Cyrillic letters, for example, using "а" instead of "a", to create an impersonated domain name. This method can avoid detection by character-level similarity detection tools, making the impersonated domain name more hidden.

[0007] Brand name pinyin / phoneme change: Some attackers will take advantage of phoneme similarity to register domain names that sound similar to the target brand or application name. This method uses similar sounding words to induce users to enter the impersonated website.

[0008] Use different top-level domain names (TLDs): Attackers may register domain names similar to target domain names, but change the top-level domain name (such as.com,.net,.org, etc.). For example, attackers may register facebook.net instead of facebook.com, or use some rare TLDs (such as.xyz,.club) instead.

[0009] Problems with existing methods:

[0010] 1. Strong dependence on rules makes it ineffective against unknown attack patterns: Rule-based domain generation methods usually rely on manually written or statically defined rules. This means that the system can only recognize new types of domain names if the rule base is updated and new misspellings or common variants are added. If a new attack method is not included in the rule base, the detection system can miss related malicious domain names. Attackers only need to constantly change their strategies and methods, and rule-based methods often cannot cope with this evolving attack. For example, attackers can create new domain name variants that do not conform to preset rules, which may not be detected.

[0011] 2. Low adaptability to complex attack methods: Low diversity of character sets and languages: Existing rules usually assume that the target domain name is based on a standard character set (such as ASCII), ignoring various character set variants that attackers may use (e.g. Latin characters, non-Latin characters, emoticons, etc.). When attackers use different character sets (such as Cyrillic letters instead of Latin letters) to generate fake domain names, rule-based methods often cannot identify these domain names.

[0012] 3. Lack of context and semantic understanding: Rule-based domain name detection methods usually only focus on the similarity of domain name characters, ignoring the semantic information that may exist behind the domain name. For example, some malicious domain names may be similar to well-known brand names by replacing characters, but semantically they are still unrelated domain names. Without semantic understanding, rule-based methods cannot identify these cases.

[0013] 4. Ignoring brand context: Many rules only consider character-level changes, but do not consider the context information of the target brand (such as the product type and business category of the brand). Therefore, there may be unreasonable matching of some legal domain names with the target brand. SUMMARY

[0014] The present invention aims to solve the above problems, and proposes a threat perception system based on active domain name generation and real-time detection of malicious domain names, which actively generates a list of similar domain names for the target domain name through an autoregressive model, and then uses it for phishing page risk assessment and detection. At the same time, by monitoring newly registered domain names in real time, high similarity domain names are detected for phishing.

[0015] The threat perception system based on active domain name generation and real-time detection of malicious domain names provided by the present invention comprises a historical threat perception module, a real-time threat perception module and a malicious detection module. The domain name generation module and the risk assessment module are set in the historical threat perception module.

[0016] The domain name generation module uses a Transformer-based autoregressive model to construct a similar domain name generation model, collects the identification information of the target enterprise and all registered and used subdomain names, identifies the keywords and phrases in the subdomain names, and generates similar domain names of the target enterprise from the keywords or phrases of the subdomain names, the identification of the target enterprise, and the input of the top-level domain name generation model, and stores the similar domain names in a similar domain name list.

[0017] The risk assessment module determines whether the domain names in the list have been registered through WHOIS query and DNS resolution, and adds the registered and accessible domain names to a to-be-detected list.

[0018] The real-time threat perception module monitors newly registered domain names in real time, detects whether the monitored domain names have a similarity to the target enterprise domain names exceeding a set similarity threshold, and if so, adds the monitored domain names to a suspicious domain name list.

[0019] The malicious detection module detects the domain names in the to-be-detected list and the suspicious domain name list, and identifies phishing websites.

[0020] The domain name generation module generates the input of the autoregressive model for the target enterprise, and realizes:

[0021] (1) Collect the identification of the target enterprise, including the formal name, pinyin, English name, and abbreviation of the enterprise;

[0022] (2) Collect all registered and used subdomain names of the target enterprise;

[0023] (3) Perform semantic analysis and splitting on the collected identification and subdomain names, extract keywords and phrases, perform clustering and redundancy removal, and then generate the input of the autoregressive model according to the set mode;

[0024] The mode of generating the input of the autoregressive model is: brand phrase constraint + subdomain phrase constraint + top-level domain name constraint; wherein the top-level domain name constraint is to select any top-level domain name from a top-level domain name set, and the top-level domain name set includes.com and.cn; the brand phrase constraint is to select any phrase from a brand cluster, and the formal name, pinyin, English name, and abbreviation of the enterprise are all added to the brand cluster as a phrase; the subdomain phrase constraint is to select any phrase from at least one subdomain cluster set, split and cluster the subdomain names, divide them into different clusters, and the keywords or phrases in each cluster are all approximate terms.

[0025] The domain name generation module is a similar domain name generation model constructed based on a Transformer autoregressive model, and includes an embedding processing module, K decoder blocks and a linear projection and output module, K being an integer greater than 1; the embedding processing module splits the input of the model into a character sequence, obtains character embedding representation by table lookup, obtains character position coding according to the sequence, adds the character embedding representation and the character position coding element by element to form an initial representation X0, processes X0 through the K decoder blocks in turn, gradually fuses context information, and finally the linear projection and output module projects and maps the output of the last decoder block to the vocabulary dimension and obtains the next character probability through a Softmax activation function, the output character is fed back to the input end through an autoregressive cycle, and iteration is generated until a terminal symbol is encountered, to generate a similar domain name. The similar domain name generation model generates similar domain names based on beam search, and all keywords or fields in the initial input are hit in the output.

[0026] The advantages and positive effects of the present application are that:

[0027] (1) The system can quickly identify and assess potential domain name abuse risks, and in actual application, can be connected with more security analysis and intelligence platforms, take blocking measures or notify relevant parties in the first time after discovering high-risk domain names, and finally effectively reduce phishing, brand abuse and other domain name-based security threats.

[0028] (2) Compared with the rule-based comparison method, the similar domain name generation model of the system can learn the potential laws of domain names from tens of thousands of domain names in advance, can generate a large number of similar domain names of enterprise domain names at one time, and can detect them to determine whether there is a phishing website, so that efficient real-time detection in a large-scale network environment can be realized. At the same time, the similar domain name generation model is based on the autoregressive characteristics of the transformer model, so that it can learn from the latest data in real time without manual rule updating.

[0029] (3) The similar domain name generation model of the system learns the statistical characteristics of legal domain names and malicious domain names based on historical domain names, and when generating, can add specific keywords learned to generate an impersonated website; at the same time, the similar domain name generation model considers brand identification, existing subdomain name features and other information, so as to ensure that the generated domain name can maintain high similarity with the real domain name of the enterprise in vision and semantics, so as to more effectively simulate potential phishing attacks. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is a composition block diagram of the threat perception system based on active domain name generation and real-time detection of malicious domain names of the present application;

[0031] Figure 2is a schematic diagram of the system of the present application for historical threat perception of target enterprise domain name;

[0032] Figure 3 is an implementation framework diagram of the present application for generating similar domain names;

[0033] Figure 4 is a flowchart of the threat detection of the malicious detection module of the present application. DETAILED DESCRIPTION

[0034] The present application will be further described in detail below in combination with the drawings and embodiments.

[0035] The threat perception system of the present application based on active domain name generation and real-time detection of malicious domain names includes historical threat and real-time threat perception of target enterprise domain name. The historical threat perception mainly adopts the autoregressive domain name generation algorithm to automatically generate possible threat domain names. The real-time threat perception adopts the transparency log to realize real-time monitoring of the domain names just registered, and performs phishing detection on the domain names with high similarity to the target domain name through the similarity algorithm.

[0036] Specifically, as shown in Figure 1 The threat perception system of the present application includes a historical threat perception module, a real-time threat perception module and a malicious detection module. The historical threat perception module performs historical danger perception of the target enterprise domain name. The implementation of the historical threat perception of the present application relies on the domain name generation algorithm. First, the autoregressive model is used to generate and simulate the threat domain names similar to the target enterprise that may be used by the attacker. The model is pre-trained on one million benign and malicious domain names to learn the statistical characteristics about the brand domain name, and automatically constructs a list of domain names that may be used for phishing or malicious distribution. Then, the phishing page risk assessment is performed on the domain names that have been registered and can be accessed, as shown in Figure 2 The real-time threat perception module monitors the domain names just registered in real time. If the similarity to the target domain name is high, the phishing page risk assessment is performed.

[0037] The historical threat perception module of the embodiment of the present application is provided with a domain name generation module and a risk assessment module. In order to effectively evaluate the potential threat degree faced by the target domain name, the domain name generation module of the present application first generates a list of domain names similar to the target domain name through the autoregressive model, and then the risk assessment module judges whether the generated similar domain names have been registered through WHOIS query and DNS resolution. The domain names that have been registered and can be accessed are stored in the detection list to perform the phishing page risk assessment.

[0038] The domain name generation module uses a multiple domain name generation algorithm to construct the domain name list for a single enterprise, as follows:

[0039] Step 11. Collecting enterprise identification information: The system collects as much identification information of the target enterprise as possible, including the company's official name, common abbreviation, pinyin, English name, etc. This step is to ensure that the generated domain name can be highly similar to the real domain name of the enterprise in terms of vision and semantics, so as to more effectively simulate potential phishing attacks.

[0040] Step 12. Subdomain information collection: The system collects all subdomains registered and used by the enterprise through web crawlers and enterprise IT asset management records. For example, for Baidu company, subdomains such as baidu.com, hao123.com, etc. may be collected. This step not only includes public main domain names, but also may include uncommon or internal use subdomains.

[0041] Step 13. Semantic analysis of collected subdomains, extraction of keywords or phrases, and clustering and redundancy removal of all identification and subdomain keywords or phrases, respectively, to generate model input according to the set mode.

[0042] In the embodiment of the present application, the subdomain is split by the separator, and the keywords or phrases are extracted, for example, "hao123" is extracted from "hao123.com". The official name, English name, pinyin, abbreviation, etc. of the enterprise are all taken as a keyword or phrase.

[0043] In the embodiment of the present application, the phrases of all identifications and subdomains of the target enterprise collected are clustered and de-redundant, including: processing of keywords or phrases of subdomains: first, using length threshold and global appearance frequency to eliminate excessively long (greater than 15 characters) and rarely appearing subdomain keywords, second, using edit distance or Jaro-Winkler distance to cluster and merge approximate items of keywords extracted from subdomains, forming different clusters, and the keywords in each cluster are all approximate items, and then selecting 2 or 3 keywords in each cluster according to appearance frequency or TF-IDF (term frequency-inverse document frequency) to retain, so as to screen out redundant homograph words in subdomains. For identification, the official name, pinyin, English, and abbreviation of the enterprise are all taken as phrases and grouped into a synonym cluster, i.e. a brand cluster.

[0044] On the basis of the obtained subdomain cluster set and brand cluster, an input of an autoregressive model is constructed. Embodiments of the present application use Disjunctive Constraint to represent each constraint, and Disjunctive Constraint represents "any one of the phrases in the cluster must appear", such as {baidu|bd}. In embodiments of the present application, a top-level domain name (TLD) set is also set, which includes.com and.cn. Each time the input is generated, a phrase is randomly selected from the brand cluster as a brand phrase constraint, a domain name is randomly selected from the TLD set as a TLD constraint, and at least one phrase is randomly selected from the subdomain cluster set as a subdomain phrase constraint, and then the "brand phrase constraint + subdomain phrase constraint + TLD constraint" are combined to form the input.

[0045] For example, after deleting the redundancy of the subdomain name cluster of a certain target enterprise, different clusters are obtained, and the retained phrases belonging to different clusters are, for example, zhidao, image, hao123, cloud, and pan. When generating the input, if single-subdomain batch generation is performed, for example, the brand phrase constraint is bd, the subdomain phrase constraint is hao123, and the TLD constraint is.com, then bd-hao123.com is formed as the input. If multi-subdomain same-round generation is performed, for example, the brand phrase constraint is bd, the subdomain phrase constraint is zhidao and image, and the TLD constraint is.com, then bd-zhidao-image.com is formed as the input.

[0046] The autoregressive model of the present application generates similar domain names based on CBS (Constrained Beam Search). Constrained Beam Search forces the generated text to contain the pre-specified words or phrases while ensuring that the sequence with the highest language model probability is selected. That is, the autoregressive model of the present application must contain the input fields when generating the dead domain name. However, the input is generated in the above-mentioned manner, and even if there are hundreds of subdomain names and brand information, the number of hard constraints of each round of CBS can be compressed, and the search space explosion can be avoided.

[0047] The autoregressive model generates similar domain names based on CBS. When traversing the path with the highest probability, the input is forced to hit all the positions, and the remaining positions are freely spliced by the autoregressive model to maintain the natural insertion of hyphen and number, which ensures that the model can generate a domain name containing the target subdomain name and brand information.

[0048] Step 14. The input is generated into the autoregressive model based on the Transformer architecture in the manner provided in step 13, and the autoregressive model outputs similar domain names.

[0049] For example, Figure 3As shown, the embodiment of the application implements a similar domain name generation model based on a Transformer architecture autoregressive model, which includes an embedding processing module, a plurality of decoder blocks, and a linear projection and output module. The embedding processing module separates the input containing the subdomain name and brand information into a character sequence, and looks up the Token embedding for each character, while looking up the corresponding position encoding according to the sequence position, and adding the two elements to form the initial representation X0. Then X0 flows through a plurality of decoder blocks in turn, gradually refining and fusing context information. Each decoder block internally includes a multi-head self-attention with a lower triangular mask and a feedforward fully connected network, and is equipped with a residual connection and a normalization layer. For specific implementation, refer to the decoder in the Transformer architecture-based autoregressive model, which will not be described here. The linear projection and output module projects the output of the last decoder block to the vocabulary dimension and obtains the next character probability through the Softmax activation function. The output character is fed back to the input end through the autoregressive loop, and is iteratively generated until the termination symbol is encountered, generating a similar domain name.

[0050] In the generation phase, the system will first perform "constraint pool compression" on the massive subdomain fragments and brand aliases in step 13, and only keep less than 8 phrase constraints per round to generate input, and package it into several "brand x subdomain" batches. For each input, use CBS decoding: dynamically track the "constraint completion degree" when beam-search is expanded, force the candidate sequence to hit all these keywords, and let the language model freely splice other positions to maintain the natural hyphen, number insertion, and other "domain name flavors". Finally, the model output is projected to the vocabulary dimension through a linear projection and gets the next character probability through Softmax. The sampled character is fed back to the input end through the autoregressive loop, and is iteratively generated until the termination symbol is encountered to generate a complete similar domain name. After all batches are processed, regular filtering, WHOIS registrability verification, and similarity sorting are performed to obtain a wide coverage and highly simulated similar domain name list.

[0051] When training the autoregressive model, one million existing domain names including normal web and phishing web domain names are collected. The model learns the statistical characteristics of domain names through these domain names, such as attackers often adding inducible phrases such as login, community, support, account, etc. in the imitation domain name. These phrases are usually closely related to brand-related login pages, social functions, or user accounts, etc. These phrases are learned during model training and added to the vocabulary. The model learns how to generate phishing websites that induce users by adding specific keywords such as login, community through training.

[0052] The Transformer-based autoregressive model can learn the contextual information of domain name structure from existing domain name data. By training on a large number of legitimate domain names, the model can understand common patterns, spelling rules, and brand features of domain names. The Transformer model can automatically learn and identify new domain name variants from data. The autoregressive model of the Transformer architecture can accurately capture multi-level similarity features: learn the deep-level dependencies of characters, words, and their context in domain names. This allows the model to more accurately identify subtle differences between legitimate domain names and counterfeit domain names, rather than just simple character-level matching. In this way, the model can reduce false negatives, especially when dealing with different languages or non-standard character sets.

[0053] The Transformer can handle input of different character sets and capture potential similarities between characters. It can not only identify spelling errors, but also understand similarities in different languages and character sets (such as using Russian letters instead of Latin letters). This allows the identification of domain name variants composed of different character sets and languages.

[0054] In addition to character similarity, the Transformer model can also capture semantic relevance of domain names through semantic embedding. For example, amazoon.com may have subtle differences in characters from amazon.com, but semantically they still point to the same brand. The Transformer-based autoregressive model can understand brand semantics through training and better identify this type of counterfeit. Semantic learning of brand features: by incorporating brand-related background information (such as company name, product name, trademark, etc.) into the training process of the model, the Transformer can better understand the uniqueness of a particular brand. For example, goog1e.com is not only similar to google.com in characters, but also belongs to the variant of google.com semantically.

[0055] When analyzing domain name squatting threats, a specific domain name structure, namely the construction of a second-level domain name like baidu.com.example.com, is used abnormally frequently in building malicious pages. This domain name pattern is often used by network attackers to mislead users into thinking they are accessing a legitimate subdomain. This type of second-level domain construction is difficult to obtain through simple domain construction methods, mainly because the owner of the top-level domain example.com has not registered all possible combinations of subdomains. Therefore, malicious attackers can register seemingly legitimate second-level domains under it, although this is technically completely legal, it may pose a security threat to users.

[0056] Due to the special structure and construction difficulty of such domain names, simple historical data analysis and pre-generated domain name methods cannot effectively cover all potential risk domain names. Therefore, the system integrates the similar domain name detection module into the real-time threat perception module.

[0057] After obtaining the list of similar domain names of the target domain name, the risk assessment module judges whether the generated similar domain names have been registered through DNS resolution and WHOIS query. For the registered and accessible domain names, phishing page risk assessment is performed, combining URL features, page text similarity, certificate information, suspicious redirection behavior, and security vendor intelligence database, etc. to comprehensively score or judge the detected pages, automatically identify potential phishing risks. The risk assessment module also performs risk priority division on the registered and accessible domain names, combines the registration situation, webpage content malicious degree, and approximation degree with the target domain name / website, generates a risk grading report, helps users to prioritize high-risk domain names to reduce potential harm. Supervisory authorities or enterprises can also conduct correlation investigation on the attacker organization behind the malicious domain name based on the similarity of registration information and server IP address according to the risk grading report, and improve the perception ability of continuous attack activities.

[0058] The real-time threat perception module of the embodiment of the application monitors newly registered domain names in real time by using transparency logs, and performs real-time threat perception on the target enterprise domain name. The transparency log provides a public and auditable system that records all issued SSL / TLS certificates, including those used for HTTPS websites. By analyzing these logs, the system can capture newly registered and certificate-issued domain names in real time, especially focusing on new domain names similar to the target enterprise domain name. SSL is the abbreviation of Secure Sockets Layer, and TLS is the abbreviation of Transport Layer Security.

[0059] The real-time threat perception module of the system uses CertStream API to monitor reports to the Certificate Transparency Log (CTL), which can identify suspicious TLS certificate issuance in near real time. These certificates are usually associated with phishing activities. Once a new domain name similar to a known enterprise domain name obtains a TLS certificate, this "suspicious" issuance activity is identified as its domain name score exceeds a certain threshold based on the configuration file. CertStream API is a real-time Certificate Transparency Log update stream that provides real-time updates from the Certificate Transparency Log network.

[0060] Embodiments of the present invention use the similarity detection algorithm Jaro-Winkler distance to compare the similarity between new domain names and the target enterprise domain names. Jaro-Winkler distance is specifically optimized to compare the similarity of short text strings and is more robust to spelling differences and common typos. By integrating the similarity algorithm into real-time data stream analysis, the present invention can effectively filter and flag suspicious domain names that only have minor changes or common typos from the official domain names of the enterprise. This mechanism enables the enterprise to quickly discover potential malicious domain names at the TLS certificate issuance stage.

[0061] In addition to the malicious domain names constructed from the secondary domain names of the target enterprise proposed for historical threat awareness, the real-time threat awareness module detects the solution including the following steps:

[0062] Step 21. Domain name segmentation: Process the captured complete domain name Y, using the dot "." as a delimiter. For example, the captured domain name Y is baidu.com.example.com, and the Y word segmentation is performed to obtain the word array ['baidu', 'com', 'example', 'com']. This step is to extract individual components from the composite domain name for further analysis and processing.

[0063] Step 22. Reorganization: Combine the words in the word array obtained after segmentation two by two. This step is mainly to simulate the domain name variant strategy that the attacker may adopt, such as recombining the original subdomain name and the top-level domain name to form a new domain name. For example, from the above array ['baidu', 'com', 'example', 'com'], a variety of possible domain names such as baidu.com, example.com, com.example.com can be combined.

[0064] Step 23. Domain name matching: Match and detect each new domain name after reorganization with the official domain name of the known target enterprise. The purpose of this step is to identify domain names that look similar to the official domain name but are actually potential malicious behavior. For example, if the official domain name of the enterprise is baidu.com, and baidu.com is obtained through the above combination, the two match, which indicates that this domain name may be a malicious domain name trying to imitate the official website.

[0065] Step 24. Suspicious domain name classification: If the combined domain name is found to be the same or extremely similar to the domain name of the target enterprise, it is classified as very suspicious and added to the suspicious domain name list. These domain names will be subject to further security review and verification to determine their authenticity and security.

[0066] The history threat perception module of the system outputs a list to be detected, the real-time threat perception module outputs a list of suspicious domain names, and the malicious detection module executes a phishing threat detection method to perform phishing page risk assessment on the domain names in the two lists. To more finely describe the risks that these domain names may bring, the malicious detection module uses a fuzzy judgment method to divide the threat levels of similar domain names into four levels: "benign", "partially suspicious", "suspicious", and "very suspicious". The entire process is shown in Figure 4 As shown in the figure, first, the domain name to be detected is redirected. If it is redirected to an official enterprise page, it can be determined that the domain name may be an official domain name grabbing behavior, and it is directly regarded as benign. The domain name is removed from the list to be detected. Then, the lists of generated and real-time acquired domain names are queried to determine whether they exist in the blacklist of phishing websites detected by major security manufacturers or community maintained blacklists such as Google Safe Browsing, PhishTank, VirusTotal, openphish, etc. If the domain name has been included in the blacklist and marked as a phishing or malicious website, it can be quickly identified and blocked, and at the same time, the domain name is moved to the suspicious domain name list. For domain names that are not in the blacklist and cannot be correctly redirected, the suspicious features in the URL are used to assess the potential phishing risk. A classification model trained in advance using public phishing data is used to determine the URL features. The similarity between similar domain names and known phishing domain names is analyzed from multiple dimensions such as length, spelling, keywords, and registration information, and the corresponding risk classification is output. For example, the URL length is abnormal, contains too many subdomains or suspicious parameters, and does not match the target brand name but contains keywords such as "login", "verify", "secure", etc. These abnormal features are often confused configurations made by phishing attackers to imitate regular websites. In addition, the detection method for malicious domain names constructed using the secondary domain name mentioned in the foregoing is used to determine the abnormality of the domain name, and then the abnormal domain name is moved to the suspicious domain name list. At the same time, the digital certificate information used by the suspicious website is checked, including the certificate authority, validity period, and domain name issued to the certificate. If the certificate does not match the actual domain name, the domain name is moved to the suspicious domain name list. For domain names that are not marked as phishing or malicious websites and pass the detection of the secondary domain name segmentation and reorganization by the real-time threat perception module and the detection of the URL features, the domain name is considered to be a benign domain name and is removed from the list to be detected.

[0067] To further reduce false positives and identify counterfeit pages, the malicious detection module detects the domain names in the suspicious domain name list using the web content detection module. The system automatically takes screenshots of suspicious websites and evaluates their page structure, text and image content, and similarity to regular websites using computer vision and OCR methods. The functions and workflow of the web content detection module will be described in detail below, and its role in phishing website identification will be illustrated through case analysis.

[0068] The web content detection module is mainly used to identify phishing features in suspicious website pages, and to determine whether they are impersonating trusted sites and implementing attacks. Specifically, the module focuses on the following aspects:

[0069] Logo impersonation check: Detect whether the suspicious page contains the logo or brand identity of the target website. If the logo or other identity of the target website is found in the web content or screenshot of the suspicious domain name, it indicates that the website may be impersonating the target site and has phishing suspicion.

[0070] Sensitive information request: Detect whether the page contains a request for the user to input credentials, such as login username, password, bank account number, etc. If the web page contains a login form, password input box, etc. component, attention should be paid, because phishing websites often lure users to input sensitive information.

[0071] Suspicious file download: Check whether the page tries to guide the user to download unknown software or compressed files, etc. Phishing websites often lure users to download malicious software by pretending to be an update program or security patch, so if a download link is detected and the file source is suspicious, it needs to be focused on.

[0072] Through the above judgment, the system can preliminarily determine whether a suspicious website has a phishing threat. For example, if a fake bank website homepage appears the official logo of the bank and requires the input of login credentials, or provides a download of a "security upgrade" software, these are obvious danger signals.

[0073] Specifically, the web content detection module of the embodiment of the present application performs the following series of steps on each suspicious domain name to comprehensively analyze whether the web content of the suspicious domain name has phishing features. The entire process is as follows:

[0074] Data acquisition: First, the system crawls the HTML source code of the homepage of the suspicious domain name, and generates a screenshot of the page. The HTML source code provides the text and structural information of the page, and the screenshot facilitates subsequent image analysis, such as logo identification.

[0075] Phishing feature identification: Based on the extracted page elements and screenshots, the system further checks for phishing features:

[0076] Logo matching: Analyze the page screenshot through image recognition technology to determine whether it contains the logo or name identity of the target website. If the logo of a well-known brand / website is found on a page of an unrelated domain name, it is usually abnormal, indicating that the website may be impersonating the appearance of the brand.

[0077] Sensitive information input: Check whether the page DOM element contains a login form or similar component that requires the user to input credentials, such as account, password input box, submit button, etc.

[0078] Download link scanning: Scan links, button text, etc. on the page to see if there is any content prompting you to download software or attachments. For example, look for buttons or links with words like "Download Update" or "Security Patch". If found, extract the file information they point to (filename, type) and determine whether the file source is trustworthy.

[0079] Hidden Links and Redirect Detection: To combat more sophisticated phishing techniques, the system uses a headless browser to simulate user actions and detect multiple redirects. Some phishing websites may not display malicious content directly on their homepage, but instead hide links to steal information or download malicious files within buttons or page interactions. Clicking these links triggers multiple URL redirects before revealing the true destination. For example, clicking a seemingly normal button might redirect the user several times, ultimately leading to a page requiring a password or a download dialog box. The detection module automatically simulates clicking suspicious buttons or links on the page, tracing the redirect chain. If the final redirection is found to an address that deviates from the expected path—for example, redirecting from the official website to a suspicious domain or download site instead—it can be determined that the website is engaging in malicious redirection and is phishing in nature.

[0080] Result Judgment and Recording: Based on the above inspection results, the system determines whether the suspicious domain name poses a phishing threat. If multiple detection points indicate problems, such as the discovery of a counterfeit logo and a password input box, or abnormal redirects, the domain name is marked as a suspicious phishing website. Subsequently, the system saves the complete HTML source code and screenshots of this suspicious page as evidence for subsequent manual review and further analysis by security personnel. All normal domains without problems are also recorded, completing the final content scanning process. Researchers can quickly identify truly threatening phishing sites and take timely blocking measures or alert relevant departments. This maximizes the use of fuzzy discrimination and machine learning models' automated processing capabilities while also using manual review to prevent missed or false alarms during high-risk periods, effectively improving the overall detection accuracy and stability. This enhances the system's automated detection capabilities while maintaining interpretability and accuracy in complex scenarios, thereby maximizing the protection of enterprises and users from phishing attacks.

[0081] In general, the various example embodiments of the present disclosure can be implemented in hardware or special-purpose circuits, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while other aspects can be implemented in firmware or software which can be executed by a controller, microprocessor or other computing device, Although the various example embodiments of the present disclosure can be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that blocks, apparatus, systems, techniques or methods described herein can be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof.

[0082] In addition to the technical features described in the specification, they are known to those skilled in the art. The present application omits the description of known components and known technologies to avoid redundancy and unnecessary limitation of the present application. The embodiments described in the above embodiments do not represent all embodiments consistent with the present application. Various modifications or variations made by those skilled in the art on the basis of the technical solutions of the present application without creative labor are still within the protection scope of the present application.

Claims

1. A threat perception system for generating and real-time detecting malicious domain names based on active domain names, characterized in that, The historical threat perception module, the real-time threat perception module and the malicious detection module are included; The domain name generation module and the risk assessment module are arranged in the historical threat perception module; The domain name generation module uses a Transformer-based autoregressive model to construct a similar domain name generation model, collects the identification information of a target enterprise and all registered and used subdomain names, identifies keywords and phrases in the subdomain names, generates inputs of the similar domain name generation model from the keywords or phrases of the subdomain names, the identification of the target enterprise and the top-level domain name, and the similar domain name generation model generates similar domain names of the target enterprise and stores them in a similar domain name list; The similar domain name generation model comprises an embedding processing module, K decoder blocks and a linear projection and output module, K being an integer greater than 1; the embedding processing module splits the input of the similar domain name generation model into a character sequence, obtains a character embedding representation by table lookup, obtains a character position code according to the character sequence, adds the character embedding representation and the character position code element by element to form an initial representation , and The K decoder blocks are processed in sequence, context information is gradually fused, the output of the last decoder block is projected and mapped to a vocabulary dimension by the linear projection and output module, and a next character probability is obtained by a Softmax activation function, the output character is fed back to the input end by an autoregressive cycle, iteration is performed until a terminal symbol is encountered, and a similar domain name is generated; the similar domain name generation model generates a similar domain name based on beam search, and all keywords or fields in the initial input are hit and output. The risk assessment module determines whether the domain names in the similar domain name list have been registered through WHOIS query and DNS resolution, and adds the registered and accessible domain names to a detection list; The real-time threat perception module monitors newly registered domain names in real time, detects whether the monitored domain names have a similarity to the target enterprise domain names exceeding a set similarity threshold, and adds the monitored domain names to a suspicious domain name list if the similarity exceeds the threshold; The malicious detection module detects the domain names in the detection list and the suspicious domain name list, identifies phishing websites, and performs page risk assessment.

2. The system of claim 1, wherein, The domain name generation module generates inputs of the similar domain name generation model for the target enterprise, and the inputs include: (11) collecting the identification of the target enterprise, including the formal name, pinyin, English name and abbreviation of the enterprise; (12) collecting all registered and used subdomain names of the target enterprise; (13) performing semantic analysis and splitting on the collected identification and subdomain names, extracting keywords and phrases, clustering and removing redundancies, and then generating inputs of the similar domain name generation model according to a set mode; The mode of generating the inputs of the similar domain name generation model is: brand phrase constraint + subdomain phrase constraint + top-level domain name constraint; wherein the top-level domain name constraint is to select any top-level domain name from a top-level domain name set, and the top-level domain name set includes.com and.cn; the brand phrase constraint is to select any phrase from a brand cluster, and the formal name, pinyin, English name and abbreviation of the enterprise are all added to the brand cluster as a phrase; the subdomain phrase constraint is to select any phrase from at least one subdomain cluster set, split and cluster the subdomain names, and divide them into different clusters, and the keywords or phrases in each cluster are all approximate terms.

3. The system of claim 2, wherein, The domain name generation module clusters and removes redundancies of the extracted keywords and phrases of all registered and used subdomain names of the target enterprise, including: first, using a length threshold and a global appearance frequency to remove keywords and phrases that do not meet the threshold length and are less than the global appearance frequency; second, using an edit distance or Jaro-Winkler distance to cluster the remaining keywords and phrases, forming different clusters, and the keywords in each cluster are all approximate terms; and third, retaining keywords with high appearance frequency or TF-IDF in each cluster, and TF-IDF represents term frequency-inverse document frequency.

4. The system of claim 1 or 2, wherein, The domain name generation module pre-trains the similar domain name generation model, collects existing domain names including legal domain names and malicious website domain names, learns statistical features of the domain names, and adds corresponding feature phrases to a word table.

5. The system of claim 1, wherein, The real-time threat perception module also detects the target enterprise secondary domain name, judges whether it is a malicious domain name, and the detection includes: (21) The captured complete domain name Y is split with "." as the separator to obtain a word array; (22) The words in the word array are combined to form a new domain name; (23) Each new domain name obtained by recombination is matched with the official domain name of the target enterprise for detection. If the domain names match or have high similarity, it is considered that the domain name Y is a malicious domain name that imitates the official domain name of the target enterprise, and is added to the suspicious domain name list.

6. The system of claim 1 or 5, wherein, The malicious detection module detects, including: (31) Redirect the domain name in the detection list. If it is redirected to the official page of the enterprise, it is determined that the domain name is an official domain name, and the domain name is removed from the detection list. For domain names that cannot be redirected, query whether it is in the blacklist. If the domain name has been marked as phishing or malicious website, identify the domain name as malicious website and block it, and add the domain name to the suspicious domain name list. If the domain name has not been marked as phishing or malicious website, perform word segmentation and word recombination on the domain name, calculate the matching degree of the recombined domain name and the domain name of the target enterprise, if the matching degree is high, add the domain name to the suspicious domain name list, otherwise, continue to detect the URL features of the domain name, judge whether it is abnormal, if it is abnormal, add it to the suspicious domain name list, otherwise, consider it as a benign domain name, and remove it from the detection list; (32) The domain name in the suspicious domain name list is detected by webpage content, including: grabbing the homepage HTML source code of the domain name, and generating a screenshot of the page; parse the page elements and extract the page composition elements; detect whether the following phishing features exist from the page screenshot and page composition elements: The page contains the logo or brand identification of the target enterprise website; There is a component in the page composition elements that requires the user to input credentials; The page has a link or button prompting to download software or files; Simulate user operation to detect multiple jumps and jump to an unexpected address; According to the existing phishing features, divide the domain name threat level.

Images