BERT and multi-scale deep learning fused DGA domain name detection method and system

Abstract

The invention belongs to the technical field of domain name detection, discloses a DGA domain name detection method fusing BERT and multi-scale deep learning, and provides a brand new DGA domain name detection thought by combining semantic modeling ability in natural language processing and a multi-scale feature extraction mechanism in deep learning. According to the method, global semantic information and local structure features of domain name characters are extracted in parallel through a double-branch structure, and a cross-attention mechanism (Cross-Attention) is introduced in a fusion stage to realize dynamic interaction of the two types of features, so that the aspects of detection precision, robustness and generalization performance are remarkably improved. According to the method, a detection framework of semantic-structure double-branch joint modeling and attention fusion is put forward for the first time, and hidden or unknown DGA domain names which are difficult to find by a traditional algorithm can be effectively recognized. The method is particularly suitable for intelligent security detection of enterprise DNS traffic, real-time analysis of malicious domain names and automatic alarm tasks of a threat intelligence system.

Description

Technical Field

[0001] This invention belongs to the field of domain name detection technology, and in particular relates to a DGA domain name detection method and system that integrates BERT and multi-scale deep learning. Background Technology

[0002] With the rapid development of the internet, the cybersecurity situation has become increasingly severe. Botnets, as a typical cyberattack vector, have become a key target of global cybersecurity governance. A botnet consists of a large number of terminal devices infected with malware. Under the remote control of attackers, these devices can launch distributed denial-of-service (DDoS) attacks, spread malware, steal private information, and conduct phishing or spam distribution activities. Traditional botnets rely on fixed command and control (C2) servers for communication. While this centralized architecture is convenient to manage, once the C2 server is discovered and blocked by security agencies, the entire botnet loses control. Therefore, attackers have gradually introduced Domain Generation Algorithms (DGAs) to dynamically generate domain names for communication with C2 servers, greatly enhancing the stealth and survivability of botnets. A domain generation algorithm is an algorithm that automatically generates a large number of candidate domain names based on random seeds (such as timestamps, system states, or specific keywords). The compromised host and the attacker can use the same algorithm to generate tens of thousands of candidate domain names simultaneously. The attacker only needs to register a very small number of these to establish communication with the infected host, thereby gaining control and issuing commands. This mechanism not only avoids the risk of fixed C2 domain names being blocked but also poses a significant challenge to network security defense systems. On the one hand, a large number of randomly generated domain names differ significantly from normal domain names in character distribution, but their generation patterns are highly diverse and random. On the other hand, the DGA algorithm can be changed or embedded with different variants at any time, making detection methods based on blacklists or static rules ineffective.

[0003] In the early stages of DGA detection research, mainstream methods were primarily based on manual feature extraction and traditional machine learning algorithms. Researchers typically designed feature sets based on statistical characteristics such as domain name length, character entropy value, vowel-to-consonant ratio, digit ratio, and triple repetition rate, and then used algorithms such as Random Forest (RF), Support Vector Machine (SVM), or Logistic Regression for classification. However, these methods have significant drawbacks: feature design relies on human experience, resulting in poor generality and scalability; when the DGA algorithm is updated or the obfuscation rules change, the model needs to re-extract features and retrain; it is difficult to capture the implicit semantic patterns and generation logic in the domain name character sequences; and the detection capability drops sharply for newly emerging or unknown DGA families.

[0004] In recent years, the development of deep learning technology has provided new solutions for DGA detection. Through automatic feature learning mechanisms, models can directly extract discriminative features from the original character sequences, reducing manual intervention. Typical deep learning architectures include Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs) and their variants, Long Short-Term Memory Networks (LSTMs) and Gated Recurrent Units (GRUs). While CNNs can extract local N-gram features, their ability to model long-distance dependencies is weak; LSTMs can handle time-series information, but their model complexity is high, training time is long, and they are prone to the vanishing gradient problem; single-layer deep networks cannot simultaneously take into account local character patterns and global semantic information, thus limiting further improvements in detection performance.

[0005] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0006] (1) Insufficient feature representation: Most models only utilize character-level local features and lack understanding of global semantics, resulting in insufficient DGA detection accuracy for complex generation patterns or multi-layered nested rules.

[0007] (2) Weak feature fusion capability: Dual-structure or multi-structure models usually adopt simple splicing or weighted fusion at the feature level, which fails to establish dynamic correlation between features of different modalities and results in low information utilization.

[0008] (3) Insufficient generalization ability: For unknown DGA family or variants that have not appeared before, the model lacks effective transfer and adaptation ability, often resulting in a significant drop in detection rate.

[0009] (4) High training cost: Some models rely on large-scale labeled data, which takes a long time to train and has complex parameters, making them unsuitable for real-time deployment in actual network environments.

[0010] Traditional rule-based or handcrafted feature-based DGA detection methods struggle to cope with dynamically changing malicious domain name generation methods; while single-structure deep learning models, although highly automated, still have shortcomings in global modeling, local structure extraction, and cross-feature fusion.

[0011] The approximate prior art found is: US 11777969 B2 (System and method for detecting a DGA domain-generation algorithm).

[0012] This patent discloses a method for detecting Domain Generation Algorithm (DGA) in a computer communication network, which mainly includes: forming a tuple of the requested domain name and the requesting client identifier for each DNS request; combining these tuples into similar partitions according to the selected aggregation technique; and then inferring the clients using the same DGA based on each partition.

[0013] While this existing technology has indeed contributed to the field of DGA detection, it still has several significant technical problems, including:

[0014] 1. Limited Feature Dimensions and Lack of Semantic-Structure Interaction: This patent primarily detects DGAs based on request behavior (domain request + client identifier tuple aggregation). In other words, it focuses on the behavioral patterns of "who is requesting" and "what is being requested," without delving into the synergistic effect of the domain's semantic information (e.g., domain composition words, contextual semantics) and structural information (character sequence patterns, local structural features). In other words, its features are mainly biased towards behavioral aggregation analysis, without constructing a parallel model of domain semantics / character structure.

[0015] 2. Inadequate Feature Fusion Method and Improvement Potential: While this method achieves analysis based on client behavior aggregation, it does not employ a cross-attention mechanism to deeply fuse features from different dimensions (such as domain name characteristics and request behavior characteristics). In other words, the correlation between the two types of features cannot be adaptively learned and weighted, resulting in a relatively static fusion method that may limit its ability to identify complex, evolving DGA families.

[0016] 3. Insufficient generalization ability to unknown DGAs: Because this scheme relies on tuple clustering and client behavior aggregation, it depends more on known client groups or behavioral habits. Its effectiveness may decline when dealing with entirely new or highly mutated DGA families (especially novel DGAs constructed through dictionary manipulation, character perturbation, or structural confusion). In other words, when handling unknown DGAs or those employing detection evasion strategies, its detection accuracy and recall may be inferior to model-based feature learning methods.

[0017] In summary, while the existing technology possesses certain industrial application value, it still exhibits significant technical deficiencies in areas such as the depth of domain name feature modeling, the design of the fusion mechanism, and the ability to generalize and identify unknown DGAs. These are precisely the technical weaknesses that this invention addresses and overcomes. Summary of the Invention

[0018] To address the problems existing in the prior art, this invention provides a DGA domain name detection method that integrates BERT and multi-scale deep learning.

[0019] This invention is implemented as follows: A DGA domain name detection method integrating BERT and multi-scale deep learning includes:

[0020] Step 1: Input and Preprocessing;

[0021] Input: DNS traffic or a collection of domain name strings D={d1,d2,...,dn} collected from the dataset; the dataset is divided into 70% training set, 15% validation set, and 15% test set to ensure a balanced data distribution.

[0022] Output: Character index matrix and attention mask;

[0023] Step 2: BERT semantic feature extraction module, used to extract global contextual semantic features of the domain name;

[0024] Step 3: Multi-scale CNN-BiLSTM feature extraction module, used to capture local structural features and long-range dependencies at the domain name character level;

[0025] Step 4: The cross-attention fusion module enables deep interaction between BERT semantic features and CNN-BiLSTM structural features;

[0026] Step 5: The classification and output module maps the fused features to the category space and outputs the detection results.

[0027] Furthermore, the input and preprocessing:

[0028] Input: DNS traffic or a collection of domain name strings D={d1,d2,...,dn} collected from the dataset; the dataset is divided into 70% training set, 15% validation set, and 15% test set to ensure a balanced data distribution.

[0029] Output: Character index matrix and attention mask;

[0030] (1) Remove the top-level domain suffix (such as ".com" or ".net") and keep the main part;

[0031] (2) Establish a character dictionary V, including letters a–z, numbers 0–9, the symbol “-”, and special identifiers “[CLS]”, “[SEP]”, and “[PAD]”;

[0032] (3) Convert each domain name into a fixed-length character index sequence, and fill any insufficient parts with [PAD];

[0033] (4) Construct two inputs:

[0034] BERT branch: Generating token sequences and attention masks;

[0035] CNN-BiLSTM branch: Generates a character index matrix and feeds it into the embedding layer.

[0036] Furthermore, the BERT semantic feature extraction module:

[0037] (1) The encoding structure of the BERT model is adopted, and the contextual semantic modeling of the input sequence is performed using a multi-layer self-attention mechanism;

[0038] (2) The input of BERT consists of tokenembedding, positionembedding and segmentembedding;

[0039] (3) Obtain the semantic vector for each character position through a multi-layer Transformer encoder:

[0040]

[0041] (4) Take the output vector corresponding to the [CLS] label. As a semantic feature representation of the entire domain name;

[0042] (5) The vector has a dimension of 768 and contains global semantic information of the domain name and potential pattern dependencies.

[0043] Furthermore, the multi-scale CNN-BiLSTM feature extraction module:

[0044] (1) The character index sequence is first input into the embedding layer and mapped to an embedding vector matrix of dimension 128;

[0045] (2) Set up three sets of parallel convolution kernels with sizes of 3, 4 and 5 respectively to extract features at different n-gram scales:

[0046]

[0047] (3) Perform ReLU activation and BatchNormalization on the convolution output;

[0048] (4) Extract key information through max pooling and concatenate the three sets of convolution results to form a local feature matrix:

[0049]

[0050] (5) Feed the convolutional features into a bidirectional LSTM network to learn the forward and backward dependencies of the character sequence:

[0051]

[0052] (6) Perform average pooling on the output sequence to obtain the comprehensive structural feature vector. The dimension is 512.

[0053] Furthermore, the cross-attention fusion module:

[0054] (1) Define the input: ,

[0055] (2) Calculate the correlation weights between the two types of features:

[0056]

[0057] (3) The structural features are weighted and summed according to their weights to obtain the fused features. ;

[0058] (4) Enhance model stability through residual connections and layer normalization operations:

[0059]

[0060] (5) Output characteristics It contains both semantic information and preserves local structural features, providing high-quality feature input for the final classification.

[0061] Furthermore, the classification and output module:

[0062] (1) Input the fused features into the fully connected layer and map them to the classification space:

[0063]

[0064] (2) Output binary classification results: 0 represents normal domain name, 1 represents DGA domain name;

[0065] (3) Employ the cross-entropy loss function:

[0066]

[0067] (4) Use the Adam optimizer to update parameters; set the learning rate of the BERT part to 1e-5 and the rest to 1e-4, the number of training rounds to 20, and the Dropout ratio to 0.3 to prevent overfitting.

[0068] Another objective of this invention is to provide a DGA domain name detection system that integrates BERT and multi-scale deep learning, comprising:

[0069] The input and data preprocessing module is responsible for converting the raw domain name string into a character index sequence that the model can process.

[0070] The BERT semantic feature extraction module is used to capture the global contextual semantics of domain name sequences through a pre-trained language model;

[0071] A multi-scale CNN-BiLSTM feature extraction module is used to extract the structural features and temporal dependencies of domain names from local to global perspectives.

[0072] The cross-attention fusion module is used to achieve dynamic information fusion of semantic features and structural features;

[0073] The classification and output module is used to map the fused features to the category space and output the detection results.

[0074] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the DGA domain name detection method that integrates BERT and multi-scale deep learning.

[0075] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the DGA domain name detection method that integrates BERT and multi-scale deep learning.

[0076] Another objective of this invention is to provide an information data processing terminal for implementing the DGA domain name detection system that integrates BERT and multi-scale deep learning.

[0077] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0078] This invention proposes a dual-branch DGA domain name detection method that integrates BERT and multi-scale deep learning. This method fully leverages BERT's advantages in global semantic understanding and contextual dependency modeling, while combining CNN and bidirectional LSTM structures to capture domain name generation patterns from both local character pattern and long-distance dependency levels. By introducing a cross-attention mechanism, dynamic complementarity and fusion of global and local features are achieved, significantly improving the accuracy, generalization, and robustness of DGA detection. It can effectively identify unknown or variant families of malicious domain names, addressing the problems of insufficient feature fusion, limited detection accuracy, and poor generalization ability in existing technologies. This provides an efficient and reliable intelligent detection method for network security defense systems.

[0079] This invention discloses a DGA domain name detection method that integrates BERT and multi-scale deep learning. This method combines the semantic modeling capabilities of natural language processing with the multi-scale feature extraction mechanism of deep learning, proposing a novel DGA domain name detection approach. The method extracts global semantic information and local structural features of domain name characters in parallel through a dual-branch structure, and introduces a cross-attention mechanism in the fusion stage to achieve dynamic interaction between the two types of features, thereby achieving significant improvements in detection accuracy, robustness, and generalization performance.

[0080] Unlike existing detection algorithms that rely on single features or model structures, this invention proposes for the first time a detection framework of "semantic-structural dual-branch joint modeling + attention fusion," which can effectively identify hidden or unknown DGA domains that are difficult for traditional algorithms to detect. This method is particularly suitable for intelligent security detection of enterprise DNS traffic, real-time analysis of malicious domains, and automatic alerting tasks in threat intelligence systems.

[0081] This invention is based on the observation that although domain names generated by DGA appear to be random combinations of characters on the surface, their generation logic still exhibits potential regularity at the character level and in terms of statistical patterns. For example, different families of DGAs differ in character combination length, the ratio of letters and numbers, local repetition, and substring distribution. Traditional methods, which only utilize character distribution statistics or extract features using fixed convolutional kernels, struggle to fully capture this complex structure.

[0082] Therefore, this invention proposes a detection model that integrates semantic and multi-scale structural features. The BERT branch utilizes the Transformer's self-attention mechanism to capture global contextual semantics, while the CNN-BiLSTM branch models local patterns and inter-character dependencies through multi-scale convolution and sequence learning structures. Subsequently, the features from the two branches undergo deep interaction through a cross-attention fusion module to generate a high-dimensional representation that combines global and local features, achieving accurate domain name classification.

[0083] This technical solution enables the joint learning and dynamic fusion of multi-scale features during DGA detection. Compared to traditional single deep structure models, this invention offers significant advantages in feature representation, discriminative ability, and model generalization. This solution is suitable for malicious domain name identification in large-scale DNS traffic environments, and can output detection results in real time, providing efficient and intelligent technical support for network security protection systems.

[0084] Compared with existing DGA detection algorithms based on a single deep learning structure, this invention has significant innovation and practical value in its overall design concept and implementation mechanism.

[0085] (1) More comprehensive feature expression: This invention proposes for the first time a dual-branch architecture that integrates BERT semantic modeling and multi-scale deep structural feature extraction. It can capture the global semantic dependency between domain name characters and extract multi-scale local structural features, thereby realizing a comprehensive modeling of the domain name generation rules and significantly improving the feature expression capability.

[0086] (2) Novel feature fusion mechanism: A cross-attention mechanism is introduced to establish a dynamic interaction relationship between semantic features and structural features. This mechanism strengthens the correlation between the two types of features through adaptive weight allocation, enabling the model to automatically focus on the feature regions that contribute most to the discrimination, thereby improving detection accuracy and stability.

[0087] (3) Significantly improved detection performance: On the 25-DGA and UMUDGA public datasets, the model achieved binary classification accuracy of 97.4% and 97.6% respectively, multi-class classification accuracy of 93.5% and 86.3% respectively, and the average accuracy for detecting unknown family DGA exceeded 94%, which is better than existing similar methods.

[0088] (4) Strong generalization and robustness: Thanks to the context modeling ability of BERT and the temporal feature learning of CNN-BiLSTM, this invention can still maintain high detection accuracy when facing domain names of different families, different lengths and noisy ones, and has good cross-dataset generalization ability.

[0089] As further supporting evidence of the inventiveness of the claims of this invention, the invention demonstrates significant advantages in terms of expected benefits, commercial value, and technological breakthroughs, fully reflecting the non-obviousness and substantial progress of the technical solution. Firstly, based on a semantic-structural dual-branch parallel modeling and cross-attention deep fusion mechanism, this invention can significantly improve the accuracy, recall rate, and generalized detection capability of unknown DGA families in malicious domain name identification. Because traditional methods have limited identification capabilities when facing diverse and constantly evolving DGA domain names, enterprises often face high potential security risk costs in DNS security incidents. This invention, through end-to-end automatic feature learning and deep model adaptive expression, achieves a higher correct identification rate and a lower false positive / false negative rate, thereby significantly reducing the economic losses caused by security incidents such as ransomware, botnet propagation, and C2 control, and significantly reducing the overall network security operation cost.

[0090] Secondly, the network architecture of this invention possesses strong engineering deployability, enabling real-time discrimination in large-scale DNS environments. It is suitable for DNS security auditing scenarios in carrier backbone networks, cloud platform security systems, and government and financial sectors, and can also serve as a core module for enterprise-level security gateways and threat intelligence systems. Its high concurrency processing capabilities and stable inference performance allow this invention to directly support the online detection needs of high-traffic DNS data, demonstrating promising commercial application prospects. Simultaneously, thanks to the model's automatic learning capabilities, this invention reduces reliance on manually constructed domain name rules, subsequent manual analysis, and model retraining, significantly reducing the analytical burden on security analysts, improving threat discovery efficiency, and saving enterprises substantial security operation and human resource costs.

[0091] This invention fills a gap in domestic and international research on related technologies in terms of theoretical framework and methodology. Existing DGA detection techniques mostly employ single-path feature extraction methods, such as relying solely on CNN or RNN structures, making it difficult to simultaneously capture the global semantic pattern and local character structure information of a domain name. This invention proposes for the first time a "semantic-structural" dual-branch parallel modeling framework, independently modeling BERT semantic representation and multi-scale CNN-BiLSTM structural representation before deep fusion, achieving a dual-modal collaborative feature learning system that has not yet been established in the industry. Furthermore, this invention introduces a cross-attention mechanism in DGA detection for the first time, solving the problem that traditional feature fusion relies solely on concatenation or fixed weighting, making it difficult to dynamically focus on features based on different domain name distributions. This technical solution achieves an adaptive deep fusion strategy through learnable computation of the correlation between semantic and structural features, constructing a high-quality fusion mechanism never before realized in the field, significantly improving model expressive power and detection performance in complex scenarios. None of these groundbreaking designs have been disclosed or inspired by existing technologies, fully demonstrating that this invention possesses outstanding substantive features and significant technological advancements. Attached Figure Description

[0092] Figure 1 This is a flowchart of the DGA domain name detection method that integrates BERT and multi-scale deep learning provided in the embodiments of the present invention.

[0093] Figure 2 This is a block diagram of the DGA domain name detection system that integrates BERT and multi-scale deep learning, provided in an embodiment of the present invention.

[0094] Figure 3 This is a detailed structural diagram of the DGA domain name detection system that integrates BERT and multi-scale deep learning, provided in an embodiment of the present invention.

[0095] Figure 4 This is a word embedding structure diagram of the BERT model provided in the embodiments of the present invention.

[0096] Figure 5 This is a multi-scale convolutional graph provided in the embodiments of the present invention.

[0097] Figure 6 This is a diagram of a bidirectional LSTM structure provided in an embodiment of the present invention.

[0098] Figure 7 This is an Attention structure diagram provided in an embodiment of the present invention.

[0099] Figure 8 This is a comparison of the accuracy of different models provided in this embodiment of the invention on the 25-DGA dataset; a) bar chart, b) line chart.

[0100] Figure 9 This is an accuracy comparison on the UMUDGA dataset provided in this embodiment of the invention; a) bar chart, b) line chart.

[0101] Figure 10 This is a comparison of the binary classification accuracy of the 25-DGA dataset provided in this embodiment of the invention; a) bar chart, b) line chart.

[0102] Figure 11 This is a comparison of the binary classification accuracy of the UMUDGA dataset provided in this embodiment of the invention; a) bar chart, b) line chart.

[0103] Figure 12 The embodiments of this invention provide the following domain name prediction rates for each DGA family: a) accuracy, b) F1 score, c) precision vs. recall for each DGA family, and d) precision vs. recall for each DGA family.

[0104] Figure 13 This is a comparison of the multi-class classification accuracy of the 25-DGA dataset provided in this embodiment of the invention; a) bar chart, b) line chart.

[0105] Figure 14 This is a comparison of the multi-class classification accuracy of the UMUDGA dataset provided in this embodiment of the invention; a) bar chart, b) line chart. Detailed Implementation

[0106] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0107] like Figure 1 As shown, the DGA domain name detection method integrating BERT (Bidirectional Encoder Representations from Transformers) and multi-scale deep learning provided by this embodiment of the invention includes the following steps:

[0108] S101: Input and Preprocessing;

[0109] Input: DNS (Domain Name System) traffic or a collection of domain name strings D={d1,d2,...,dn} collected from the dataset; the dataset is divided into 70% training set, 15% validation set, and 15% test set to ensure a balanced data distribution.

[0110] Output: Character index matrix and attention mask;

[0111] S102: BERT semantic feature extraction module, used to extract global contextual semantic features of a domain name;

[0112] S103: Multi-scale CNN-BiLSTM feature extraction module, used to capture local structural features and long-range dependencies at the domain name character level. Here, CNN stands for Convolutional Neural Network; BiLSTM stands for Bidirectional Long Short-Term Memory.

[0113] S104: The cross-attention fusion module enables deep interaction between BERT semantic features and CNN-BiLSTM structural features;

[0114] S105: The classification and output module maps the fused features to the category space and outputs the detection results.

[0115] Input and preprocessing provided in this embodiment of the invention:

[0116] Input: DNS traffic or a collection of domain name strings D={d1,d2,...,dn} collected from the dataset; the dataset is divided into 70% training set, 15% validation set, and 15% test set to ensure a balanced data distribution.

[0117] Output: Character index matrix and attention mask;

[0118] (1) Remove the top-level domain suffix (such as ".com" or ".net") and keep the main part;

[0119] (2) Establish a character dictionary V, including letters a–z, numbers 0–9, the symbol “-”, and the identifiers in the BERT model “[CLS]” (used to indicate the beginning of a sequence), “[SEP]” (used to separate two sentences), and “[PAD]” (used to fill a sequence to a fixed length);

[0120] (3) Convert each domain name into a fixed-length character index sequence, and fill any insufficient parts with [PAD];

[0121] (4) Construct two inputs:

[0122] BERT branch: Generate token sequences (word segmentation sequences) and attention masks;

[0123] CNN-BiLSTM branch: Generates a character index matrix and feeds it into the embedding layer.

[0124] The BERT semantic feature extraction module provided in this embodiment of the invention:

[0125] (1) The encoding structure of the BERT model is adopted, and the contextual semantic modeling of the input sequence is performed using a multi-layer self-attention mechanism;

[0126] (2) The input of BERT consists of token embedding, position embedding and segment embedding;

[0127] (3) Obtain the semantic vector for each character position through a multi-layer Transformer encoder:

[0128]

[0129] (4) Take the output vector corresponding to the [CLS] label. As a semantic feature representation of the entire domain name;

[0130] (5) The vector has a dimension of 768 and contains global semantic information of the domain name and potential pattern dependencies.

[0131] The multi-scale CNN-BiLSTM feature extraction module provided in this embodiment of the invention:

[0132] (1) The character index sequence is first input into the embedding layer and mapped to an embedding vector matrix of dimension 128;

[0133] (2) Set up three sets of parallel convolution kernels with sizes of 3, 4 and 5 respectively to extract features at different n-gram scales:

[0134]

[0135] (3) Perform ReLU activation and BatchNormalization on the convolution output;

[0136] (4) Extract key information through max pooling and concatenate the three sets of convolution results to form a local feature matrix. :

[0137]

[0138] (5) The convolutional features are fed into a bidirectional LSTM network to learn the forward and backward dependencies of the character sequence, where The hidden state of the forward LSTM. The hidden state for the backward LSTM:

[0139]

[0140] (6) Perform average pooling on the output sequence to obtain the comprehensive structural feature vector. The dimension is 512.

[0141] The cross-attention fusion module provided in this embodiment of the invention:

[0142] (1) Define the input: query vector key vector value vector

[0143] (2) Calculate the correlation weights between the two types of features, where The dimension of the key vector:

[0144]

[0145] (3) The structural features are weighted and summed according to their weights to obtain the fused features. ;

[0146] (4) Enhance model stability and output features through residual connection and layer normalization operations. It contains both semantic-level information and preserves local structural features, providing high-quality feature input for the final classification:

[0147]

[0148] The classification and output module provided in this embodiment of the invention:

[0149] (1) Input the fused features into the fully connected layer and map them to the classification space, where This is the weight matrix. For bias terms:

[0150]

[0151] (2) Output binary classification results: 0 represents normal domain name, 1 represents DGA domain name;

[0152] (3) The cross-entropy loss function is adopted, where For real labels, For predicting probabilities:

[0153]

[0154] (4) Use the Adam optimizer to update parameters; set the learning rate of the BERT part to 1e-5 and the rest to 1e-4, the number of training rounds to 20, and the Dropout ratio to 0.3 to prevent overfitting.

[0155] like Figure 2 , Figure 3 As shown, an embodiment of the present invention provides a DGA domain name detection system that integrates BERT and multi-scale deep learning, comprising:

[0156] The input and data preprocessing module is responsible for converting the raw domain name string into a character index sequence that the model can process.

[0157] The BERT semantic feature extraction module is used to capture the global contextual semantics of domain name sequences through a pre-trained language model;

[0158] A multi-scale CNN-BiLSTM feature extraction module is used to extract the structural features and temporal dependencies of domain names from local to global perspectives.

[0159] The cross-attention fusion module is used to achieve dynamic information fusion of semantic features and structural features;

[0160] The classification and output module is used to map the fused features to the category space and output the detection results.

[0161] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the DGA domain name detection method that integrates BERT and multi-scale deep learning.

[0162] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the DGA domain name detection method that integrates BERT and multi-scale deep learning.

[0163] Another objective of this invention is to provide an information data processing terminal for implementing the DGA domain name detection system that integrates BERT and multi-scale deep learning.

[0164] Core technical features

[0165] (1) Dual-branch parallel architecture:

[0166] By designing the BERT semantic model in parallel with the multi-scale CNN-BiLSTM structural model, it is possible to learn the semantic and structural information of domain names simultaneously.

[0167] (2) Multi-scale feature extraction mechanism:

[0168] Convolutional layers use kernels of different sizes to capture character patterns of different granularities, thereby enhancing feature diversity.

[0169] (3) Cross-attention fusion mechanism:

[0170] We innovatively introduce the Cross-Attention mechanism to establish a dynamic association between semantic and structural features, thereby achieving adaptive fusion at the feature level.

[0171] (4) End-to-end feature learning and detection:

[0172] The entire detection process does not rely on manual feature extraction. Instead, it automatically learns domain name features through deep networks, thereby improving generalization performance and adaptability.

[0173] Technical Principle Explanation

[0174] This invention is based on the principle of deep representation learning and achieves automatic identification of the generation patterns of DGA domain names through joint modeling of semantic and structural features.

[0175] The BERT branch excels at modeling long-term dependencies between characters and extracting global contextual semantic information; the CNN-BiLSTM branch effectively captures local patterns and temporal features; the cross-attention module establishes an information mapping between the two, enabling the model to adaptively focus on the feature regions that contribute most to classification.

[0176] Through this multi-layer fusion mechanism, the model can accurately detect multi-family, hybrid, and unknown DGA domain names, overcoming the shortcomings of traditional methods such as single features and insufficient robustness.

[0177] Specific implementation of the present invention:

[0178] (I) System Hardware and Software Environment

[0179] The method described in this invention is implemented in a deep learning framework environment, and the running platform includes the following configuration:

[0180] Hardware setup: NVIDIA RTX 3090 graphics card with 24GB VRAM; Intel Xeon CPU; 32GB RAM.

[0181] Software environment: Operating system is Ubuntu 22.04, Python 3.10, deep learning framework is PyTorch 2.0+;

[0182] Dependencies include: transformers, numpy, scikit-learn, pandas, matplotlib, etc.

[0183] Data sources: publicly available 25-DGA and UMUDGA datasets.

[0184] This system adopts a modular design, and its overall structure includes: a data input module, a semantic feature extraction module, a structural feature extraction module, a cross-attention fusion module, and a classification decision module.

[0185] (II) Implementation Steps of the Technical Solution

[0186] The DGA domain name detection method of the present invention can be implemented according to the following steps:

[0187] Step 1: Input and Preprocessing

[0188] Input: A collection of domain name strings gathered from DNS traffic or data sets. A 70% training set, 15% validation set, and 15% test set partitioning method is adopted to ensure a balanced data distribution.

[0189] Output: Character index matrix and attention mask.

[0190] (1) Remove the top-level domain suffix (such as ".com" or ".net") and keep the main part;

[0191] (2) Establish a character dictionary V, including letters a–z, numbers 0–9, the symbol “-”, and special identifiers “[CLS]”, “[SEP]”, and “[PAD]”;

[0192] (3) Convert each domain name into a fixed-length character index sequence, and fill any insufficient parts with [PAD];

[0193] (4) Construct two inputs:

[0194] BERT branch: Generating token sequences and attention masks;

[0195] CNN-BiLSTM branch: Generates a character index matrix and feeds it into the embedding layer.

[0196] Step 2: BERT Semantic Feature Extraction Module

[0197] This module is used to extract the global contextual semantic features of a domain name.

[0198] (1) This invention adopts the encoding structure of BERT (Bidirectional Encoder Representations from Transformers) model and uses a multi-layer self-attention mechanism to perform contextual semantic modeling on the input sequence.

[0199] (2) The input of BERT consists of token embedding, position embedding and segment embedding.

[0200] (3) Obtain the semantic vector for each character position through a multi-layer Transformer encoder:

[0201]

[0202] (4) Take the output vector corresponding to the [CLS] label. As a semantic feature representation of the entire domain name.

[0203] (5) The vector has a dimension of 768 and contains global semantic information of the domain name and potential pattern dependencies.

[0204] Step 3: Multi-scale CNN-BiLSTM feature extraction module

[0205] This module is used to capture local structural features and long-distance dependencies at the domain name character level.

[0206] (1) The character index sequence is first input into the embedding layer and mapped to an embedding vector matrix with a dimension of 128.

[0207] (2) Set up three sets of parallel convolution kernels with sizes of 3, 4 and 5 respectively to extract features at different n-gram scales:

[0208]

[0209] (3) Perform ReLU activation and Batch Normalization on the convolution output;

[0210] (4) Extract key information through max pooling and concatenate the three sets of convolution results to form a local feature matrix:

[0211]

[0212] (5) Feed the convolutional features into a bidirectional LSTM network to learn the forward and backward dependencies of the character sequence:

[0213]

[0214] (6) Perform average pooling on the output sequence to obtain the comprehensive structural feature vector. The dimension is 512.

[0215] Step 4: Cross-attention fusion module

[0216] To achieve deep interaction between BERT semantic features and CNN-BiLSTM structural features, this invention designs a cross-attention mechanism.

[0217] (1) Define the input: ,

[0218] (2) Calculate the correlation weights between the two types of features:

[0219]

[0220] (3) The structural features are weighted and summed according to their weights to obtain the fused features. .

[0221] (4) Enhance model stability through residual connections and layer normalization operations:

[0222]

[0223] (5) Output characteristics It contains both semantic information and preserves local structural features, providing high-quality feature input for the final classification.

[0224] Step 5: Classification and Output Module

[0225] (1) Input the fused features into the fully connected layer and map them to the classification space:

[0226]

[0227] (2) Output binary classification results: 0 represents normal domain name, 1 represents DGA domain name.

[0228] (3) Employ the cross-entropy loss function:

[0229]

[0230] (4) Use the Adam optimizer to update parameters. Set the learning rate of the BERT part to 1e-5 and the rest to 1e-4, the number of training rounds to 20, and the Dropout ratio to 0.3 to prevent overfitting.

[0231] (III) Parameter Settings

[0232] Table 1. Parameter Settings for DGA Domain Name Detection Method

[0233]

[0234] (iv) Model training and optimization

[0235] The model training process is as follows:

[0236] (1) Feature input: Domain name character sequence is input in parallel to the BERT branch and the CNN-BiLSTM branch;

[0237] (2) Semantic and structural extraction: The two branches calculate the semantic feature vector and the structural feature matrix respectively;

[0238] (3) Fusion and attention weighting: The cross-attention module dynamically integrates semantic and structural information according to the weight distribution;

[0239] (4) Classification and backpropagation of error: The fused features are passed through a fully connected layer and a Softmax layer to output the classification probability, and the parameters are updated by backpropagation through cross-entropy loss;

[0240] (5) Training iteration and model convergence: The accuracy and F1 score are monitored by the validation set. When the performance improvement is less than 0.1% for three consecutive epochs, the early stopping mechanism is triggered.

[0241] (V) Experimental Verification

[0242] To verify the effectiveness of this invention, performance tests were conducted on the 25-DGA and UMUDGA datasets. Simultaneously, experiments were performed comparing the BERT module, the CNN-LSTM module, and the module without cross-attention.

[0243] Table 2 Experimental results on the 25-DGA dataset

[0244]

[0245] Table 3 Experimental results on the UMUDGA dataset

[0246]

[0247] Figure 8 This is a comparison of the accuracy of different models provided in this embodiment of the invention on the 25-DGA dataset; a) bar chart, b) line chart.

[0248] Figure 9 This is an accuracy comparison on the UMUDGA dataset provided in this embodiment of the invention; a) bar chart, b) line chart.

[0249] Figure 10 This is a comparison of the binary classification accuracy of the 25-DGA dataset provided in this embodiment of the invention; a) bar chart, b) line chart.

[0250] Figure 11This is a comparison of the binary classification accuracy of the UMUDGA dataset provided in this embodiment of the invention; a) bar chart, b) line chart.

[0251] Figure 12 The embodiments of this invention provide the following domain name prediction rates for each DGA family: a) accuracy, b) F1 score, c) precision vs. recall for each DGA family, and d) precision vs. recall for each DGA family.

[0252] Figure 13 This is a comparison of the multi-class classification accuracy of the 25-DGA dataset provided in this embodiment of the invention; a) bar chart, b) line chart.

[0253] Figure 14 This is a comparison of the multi-class classification accuracy of the UMUDGA dataset provided in this embodiment of the invention; a) bar chart, b) line chart.

[0254] I. Specific application areas or related products of this invention.

[0255] The DGA domain name detection method and system proposed in this invention, which integrates BERT and multi-scale deep learning, features high accuracy, high robustness, and engineerable deployment. It can be widely applied in various fields such as network security, telecommunications operations, cloud computing platforms, and government and enterprise-level information security infrastructure. The following scenarios illustrate the application areas and related products of this invention to further demonstrate its inventiveness and technical value.

[0256] Various platforms can be equipped with DGA domain name detection modules. The system uses a built-in DGA detection model to analyze domain names in DNS data and aggregates all suspected domain names to this interface, forming a pool of pending work orders. This provides a foundation for subsequent manual assessment and event generation. The interface mainly displays the following fields: detected suspected domain names and their status (including suspected and assessed), domain name domestic / international attributes, domain name location, and detection time, providing analysts with a reference for decision-making priorities.

[0257] II. Evidence related to the technical effects obtained by the embodiments of the present invention.

[0258] Table 4. Binary classification results on the 25-DGA dataset.

[0259]

[0260] Table 5. Binary classification results on the UMUDGA dataset.

[0261]

[0262] Table 6. Prediction results of unknown DGA families on the 25-DGA dataset.

[0263]

[0264] Table 7. Multiclass classification results on the 25-DGA dataset

[0265]

[0266] Table 8. Multiclass classification results on the UMUDGA dataset.

[0267]

[0268] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0269] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A DGA domain name detection method integrating BERT and multi-scale deep learning, characterized in that, Includes the following steps: Step 1, Input and Preprocessing: Input the DNS traffic or the set of domain name strings collected in the dataset, and divide them into training set, validation set and test set in a ratio of 70:15:15; Step 2: Use the BERT semantic feature extraction module to extract the contextual semantic information of the domain name; Step 3: Extract the local structure and temporal dependency features of domain names using multi-scale convolutional neural networks and bidirectional long short-term memory networks; Step 4: Deep fusion of semantic features and structural features is achieved through the cross-attention fusion module; Step 5: Use the classification and output module to map the fused features and output the detection results.

2. The method as described in claim 1, characterized in that, The input and preprocessing include: Remove the top-level suffix from the domain name, create a character dictionary containing letters, numbers, symbols, and special identifiers, and convert the domain name into a fixed-length character index sequence, padding any insufficient parts with padding characters; Generate the label sequence and attention mask for the BERT branch, and simultaneously generate the character index matrix for the convolutional network branch to input the embedding layer.

3. The method as described in claim 1, characterized in that, The BERT semantic feature extraction module includes: Context modeling of the input sequence is performed using a multi-layer self-attention mechanism; The word embeddings, positional embeddings, and segmented embeddings are summed to form the input matrix; The semantic vector of each character is obtained through a multi-layer Transformer encoder, and the output corresponding to the classification label is extracted as the global semantic feature representation. The feature vector has a dimension of 768 and represents the overall semantic information of the domain name.

4. The method as described in claim 1, characterized in that, The feature extraction module of the multi-scale convolutional neural network and bidirectional long short-term memory network includes: The character index sequence is input into the embedding layer and mapped to a 128-dimensional embedding matrix; Three sets of convolutional kernels were set up with kernel sizes of 3, 4, and 5 to extract string features at different scales. After the convolution results are activated by rectified linear units and batch normalized, max pooling is performed and the results are concatenated to form a local feature matrix. The local features are input into a bidirectional long short-term memory network to extract the positive and negative dependencies, and the output feature vector has a dimension of 512.

5. The method as described in claim 1, characterized in that, The cross-attention fusion module includes: Using semantic feature vectors as query vectors and structural feature matrices as keys and values, calculate the relevance weights between the two. The structural features are weighted and summed using normalized weights to obtain the fused features; The model stability is improved by using residual connections and layer normalization operations, and the output fused feature vector is used as the classification input.

6. The method as described in claim 1, characterized in that, The classification and output module includes: The fused features are input into the fully connected layer and mapped to the binary classification space, outputting the prediction results of normal domain names or DGA domain names; The model is trained using the cross-entropy loss function. The parameters are updated using an adaptive momentum optimization algorithm, with a BERT partial learning rate of 1×10⁻⁶. -5 The learning rate for the remaining networks is 1×10. -4 The training rounds were 20, and the dropout rate was 0.3%.

7. A DGA domain name detection system based on the method described in any one of claims 1 to 6, which integrates BERT and multi-scale deep learning, characterized in that, The system includes: The input and data preprocessing module is used to convert the original domain name string into model input; The BERT semantic feature extraction module is used to extract global semantic information from domain name sequences; A feature extraction module using multi-scale convolutional neural networks and bidirectional long short-term memory networks is used to extract local structure and temporal dependencies; The cross-attention fusion module is used to fuse semantic and structural features; The classification and output module is used to output the detection results.

8. The system as described in claim 7, characterized in that, The input and data preprocessing module is configured to generate two inputs through character mapping and masking mechanisms, which are used by the BERT branch and the convolutional loop branch, respectively.

9. The system as described in claim 7, characterized in that, The cross-attention fusion module achieves dynamic interaction between semantic and structural features through an attention weight matrix and employs a layer normalization mechanism to enhance fusion stability.

10. A computer-readable storage medium for implementing the method as described in any one of claims 1 to 6, characterized in that, It stores a computer program that, when executed by a processor, causes the computer to perform DGA domain name detection steps that combine BERT and multi-scale deep learning, including input and preprocessing, semantic feature extraction, structural feature extraction, cross-attention fusion, and classification output.

Images