Incremental learning-based technique and tactics classification method

Abstract

The invention relates to the field of network security and artificial intelligence, and discloses a technique and tactics classification method based on incremental learning, which comprises the following steps: firstly, carrying out regularization preprocessing on an original threat intelligence text, and carrying out semantic enhancement by utilizing a large language model to relieve sample scarcity and category imbalance; then, a classification model capable of being dynamically expanded is constructed, the model comprises a technique and tactics encoder, a label attention mechanism and a lightweight classifier, and in combination with a low-rank self-adaptive fine tuning strategy, rapid adaptation of newly-added categories is achieved; and through combined optimization of weighted classification loss and two types of knowledge distillation loss, and by adopting reservoir sampling, a playback memory library with a fixed capacity is maintained, so that disastrous forgetting in an incremental learning process is reduced. According to the method, efficient and stable automatic technology and tactical classification can be realized in a scene of continuously updating the threat tactics.

Description

Technical Field

[0001] This invention relates to the fields of cybersecurity and artificial intelligence, and in particular to a tactical classification method based on incremental learning. Background Technology

[0002] Cyber ​​threat intelligence aims to collect, analyze, and share information about potential current cyber threats and threat actors, providing organizations with a deep understanding of cyberattack risks and enabling them to effectively prepare for and respond to security incidents. Among cyber threat intelligence, the most valuable and difficult-to-obtain information is the attacker's TTPs (Tactics, Techniques, and Procedures). TTPs reveal the attacker's intent, behavioral patterns, and attack paths, exhibiting strong stability and helping security defenders gain a deeper understanding of the attack lifecycle and improve proactive defense capabilities. The MITRE ATT&CK (Adversarial Tactics, Techniques, and Common Knowledge) framework provides a comprehensive and continuously updated TTP classification system and has become a widely adopted knowledge standard in the industry.

[0003] In recent years, many studies have attempted to combine machine learning or deep learning methods with the MITRE ATT&CK knowledge base to automatically extract and classify TTPs (Threat Transaction Processing) information from massive amounts of unstructured threat intelligence text. Despite some progress, the ATT&CK knowledge base is frequently updated to cover new attack behaviors as cyberattack methods evolve. This dynamic evolutionary nature poses significant challenges to existing TTP classification models.

[0004] Untimely model updates: Existing models are typically trained on a fixed version of the ATT&CK knowledge base. When new tactics or techniques are added to the knowledge base or the classification structure is adjusted, traditional methods require merging historical and new data to retrain the entire model, resulting in high training costs and severe response delays, making it difficult to meet the real-time identification requirements of new attack tactics and techniques.

[0005] Imbalanced category distribution: The ATT&CK framework uses a hierarchical tree structure to organize tactics and techniques, resulting in significant differences in the frequency of different categories in threat intelligence. Some common tactics and techniques are abundant, while new or highly concealed tactics and techniques are scarce, causing a severe category imbalance problem. This, in turn, affects the model's ability to learn and identify a few categories accurately. Summary of the Invention

[0006] To address the aforementioned problems, the purpose of this invention is to provide a tactical / technical classification method based on incremental learning, enhancing the ability to rapidly adapt to new TTPs and achieving a highly efficient, lightweight threat behavior classification method with continuous learning capabilities. The technical solution is as follows:

[0007] A technique and tactic classification method based on incremental learning includes the following steps:

[0008] Step 1: Data Collection: In the t-th round of incremental learning, acquire the incremental update data of the MITRE ATT&CK knowledge base; the incremental update data is the TTPs information added or updated in the t-th round; where t is an integer greater than or equal to 1; perform data preprocessing on the incremental update data.

[0009] Step 2: Construct a basic training dataset based on the incrementally updated data, and then perform semantic enhancement on the threat intelligence text in the basic training dataset using a large language model to obtain the expanded training dataset. ;

[0010] Step 3: Construct a low-rank adaptive (LoRA) fine-tuned SecureBERT model as the semantic understanding foundation for a tactics and techniques classification model. This model extracts semantic features from threat intelligence text and tactical label description text, and classifies attack tactics and techniques. The tactical classification model includes a tactical encoder, a label attention mechanism, and a label classifier, and uses the expanded training dataset. The replay memory M, which employs a reservoir sampling strategy (the number of samples in the replay memory is empty when t=1), is used as input.

[0011] Step 4: When t=1, based on the expanded training dataset, the low-rank adaptive technique is used to train the tactical classification model to obtain the initial tactical classification model;

[0012] When t>1, based on the expanded training dataset and the training samples selected by the reservoir sampling in the first t-1 rounds in the replay memory M, the low-rank adaptive technique is used to incrementally train the t-1 round training of the t-t t-t classification model to obtain the t-t round t-t classification model; and the t-t round t-t classification model is used to output the classification results of the t-t skills and tactics in the threat intelligence text.

[0013] The incremental learning process is constrained by the incremental learning loss function. After each round of incremental learning, the reservoir sampling algorithm is used to incrementally add the training samples of the current round to the playback memory to update the playback memory.

[0014] The beneficial effects of this invention are:

[0015] The method of this invention is based on a data replay and knowledge distillation framework, which integrates the text generation capabilities of a large language model with the efficient fine-tuning mechanism of low-rank adaptive (LoRA) to improve the ability to quickly adapt to new TTPs.

[0016] In scenarios with imbalanced data distribution, the method of this invention generates enhanced samples through a large language model and supplements them with positive sample weight adjustment to effectively alleviate class bias. During the continuous evolution of the model, historical knowledge is retained by experience replay, while knowledge distillation is used to control parameter growth and maintain model compactness, thereby achieving an efficient, lightweight threat behavior classification method with continuous learning capabilities. Attached Figure Description

[0017] Figure 1 This is a conceptual diagram of the incremental learning-based tactical classification method of the present invention.

[0018] Figure 2 This is a diagram illustrating the model structure and incremental learning process of the present invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0020] This invention proposes a tactical and technical classification method based on incremental learning. For example... Figure 1 As shown, this invention mainly includes four core steps: training data preprocessing and expansion, model construction (including a technical and tactical encoder, a label attention mechanism, and a label classifier), incremental learning loss function optimization, and replay memory update. It takes the training data of round t and the replay memory samples as input and outputs the classification results of technical and tactical skills (TTPs) in threat intelligence texts under the corresponding version.

[0021] 1. Training data preprocessing and augmentation:

[0022] First, data collection is performed: In the t-th round of incremental learning, the t-th version of the MITRE ATT&CK knowledge base is obtained; where t is an integer greater than or equal to 1, and the data includes tactical, technical and procedural (TTP) information added or updated in the t-th version.

[0023] Then, an initial training dataset is constructed based on the t-th version data. The initial training dataset It contains several threat intelligence texts and corresponding tactical tags in multi-hot encoded formats. Subsequently, this initial training dataset... For each category of threat intelligence text, pre-trained large language models are used for data augmentation: On the one hand, pre-trained models based on Masked Language Modeling (MLM) (such as BERT) can be used to randomly mask some words in the same category of threat intelligence text, and the model can predict the masked content to generate semantically preserved fill-in-the-blank samples; on the other hand, pre-trained language models based on autoregressive mechanisms (such as Decoder-only architecture models like GPT and Qwen) can be used to generate rewritten samples with the original threat intelligence text as input prompts, keeping the original meaning unchanged but with different expressions. Through the above methods, fill-in-the-blank or rewritten threat intelligence texts of the same category are processed to obtain the expanded training dataset.

[0024] Raw threat intelligence text typically contains various types of Indicator of Compromise (IOC) substrings, such as file paths, IP addresses, domain names, and file hashes. While these specific instances have intelligence value, they may introduce noise or lead to overfitting during model training, thereby interfering with the model's understanding of the semantic essence. To address this issue, this invention employs a regular expression-based pattern matching method to identify and generalize the replacement of various IOC substrings in the raw text.

[0025] First, the raw threat intelligence text is preprocessed, replacing various vulnerability indicators with predefined, generic placeholders representing their types. For example, in the original sentence: "For Windows Vista or higher, the path would appear as C:\Users\Public", "C:\Users\Public" is a typical file path IOC. According to predefined rules, the system uses regular expressions to identify and uniformly replace it with the generic placeholder "FilePath", thus transforming the original sentence into: "For Windows Vista or higher, the path would appear as File Path". Similarly, IP addresses (such as "192.168.1.10") can be replaced with "IP Address", domain names (such as "malicious.example.com") with "Domain", and SHA-256 hash values ​​with "Hash", etc. This preprocessing step effectively removes specific instance information while preserving the contextual semantic structure, laying the foundation for subsequent text augmentation and model training.

[0026] Then, text generation techniques are used to rewrite the replaced text to generate new samples with similar semantics but different expressions, thereby expanding the data and alleviating the problem of class imbalance affecting the model's learning ability. Text generation techniques can include: BERT-based Masked Language Modeling (MLM) tasks, which generate new sentences by masking parts of the text and using the BERT model to predict and fill in the missing parts; or using the text generation capabilities of generative language models (such as the GPT series) for synonym rewriting.

[0027] To address the issue of sparse technical and tactical label samples in the MITRE ATT&CK framework, semantic enhancement is performed on the preprocessed text. In one implementation, the SecureBERT language model, pre-trained on a cybersecurity corpus, performs a masked language modeling task: randomly masking several words in the input text, using the model to predict the masked content, and generating new legitimate sentences. In another implementation, a generative language model is invoked to paraphrase the original text, generating diverse training samples. Specifically, this invention ensures that each label contains at least 18 associated threat intelligence texts through the aforementioned text enhancement.

[0028] 2. Model Construction:

[0029] In the specific implementation of this invention, the core objective of constructing the tactics and techniques classification model is to support incremental learning capabilities in scenarios where the MITRE ATT&CK knowledge base is continuously and dynamically evolving, rather than simply implementing TTPs classification in a static version. This model natively supports category expansion and knowledge retention at the architectural level, ensuring that it is not necessary to retrain all historical data when introducing new tactics and techniques, while effectively mitigating catastrophic forgetting.

[0030] Figure 2 The diagram illustrates the overall structure of the model of this invention, primarily composed of upper and lower halves. The upper half represents the model after the (t-1)th training round, and the lower half represents the current model (i.e., the model undergoing the tth training round). This represents the set of descriptive texts about technical and tactical labels in the MITRE ATT&CK framework during the t-th round of incremental training. This refers to the technical and tactical encoder. Let be the high-dimensional vector representation of the nth token. This indicates the label attention mechanism. This indicates that the threat intelligence text is related to the first... A vector representation of each label. This is the classification loss function. Let be the first knowledge distillation loss function. This is the second knowledge distillation loss term. To indicate the first The version model applies to X based on its learned set of technical and tactical labels (total). The logits vector of (each category), For the current model, for the same input X, in the above-mentioned set of the same technical and tactical labels (i.e., the previous... The logits vector output on the category learned in the stage. The current model applies the same input X to all sets of technical and tactical labels (total). There are 10 categories, among which Greater than or equal to The logits vector output on ).

[0031] This model is used to extract semantic features from threat intelligence texts and classify attack techniques and tactics (TTPs). Its architecture supports dynamic expansion of the output layer to adapt to new categories, meeting the need for continuous model updates during incremental learning. Figure 2 As shown, the technical and tactical classification model consists of three parts in sequence: a technical and tactical encoder, a label attention mechanism, and a label classifier (the blue and red parts represent the label classifier, where the red part represents the label classifier dynamically expanding to cover newly added technical and tactical labels). The overall architecture supports dynamic expansion of the output layer to adapt to newly added technical and tactical categories.

[0032] The Tactical Encoder is a BERT-based model that only accepts two types of text information as input: threat intelligence text and label description. It uses the SecureBERT model pre-trained on cybersecurity corpora as the semantic understanding backbone and introduces a Low-Rank Adaptation (LoRA) fine-tuning strategy to efficiently adapt to TTPs classification tasks while maintaining the model's general semantic capabilities, and to improve the learning speed of the Tactical Encoder. Specifically, the LoRA configuration parameters are shown in Table 1. The input of the Tactical Encoder is the preprocessed threat intelligence text, and the output is its context vector representation (see Formula (2)).

[0033] During the training phase At that time, the BERT model used was the SecureBERT model pre-trained on a cybersecurity corpus; during the training phase At that time, the initial parameters of the technical and tactical encoder are taken from the technical and tactical encoder parameters in stage t-1; the purpose of introducing the low-rank adaptive fine-tuning strategy is to maintain the general semantic ability of the model while efficiently adapting to TTPs classification tasks.

[0034] Low-rank adaptive techniques typically add a low-rank matrix to the weight matrix of a pre-trained model, enabling the model to learn task-specific information more effectively. During training, the low-rank adaptive layer can be initialized with random values ​​and updated during fine-tuning to learn task-specific information.

[0035] Table 1 LoRA Configuration .

[0036] The tag attention mechanism mainly consists of a matrix representing technical and tactical tags, which has learning capabilities.

[0037] The label classifier part then transforms the input from the previous stage into... This indicates that it is used to classify threatening behaviors, and the classifier is essentially composed of a set of learnable weight vectors (or a weight matrix). Both the label attention mechanism and the label classifier support the addition of new tactical labels, and achieve near-linear parameter growth through lightweight design, minimizing the number of new parameters.

[0038] When t=1, based on the expanded training dataset, the classification model based on the SecureBERT model is trained using the low-rank adaptive (LoRA) technique to obtain the initial tactical classification model. During the training process, only the low-rank adaptation parameters introduced by LoRA and the classification-related parameters are updated, while the original parameters of the SecureBERT base model are frozen.

[0039] When t>1, based on the expanded training dataset and the training samples selected by the reservoir sampling in the first t-1 rounds in the replay memory, the classification model obtained in the t-1 round of training is incrementally trained using LoRA technology to obtain the t-th round of tactical classification model; during the incremental training process, only the low-rank adaptation parameters and classification-related parameters introduced by the LoRA module are updated, while the SecureBERT base parameters in the t-1 round of classification model remain unchanged.

[0040] Training or incremental training using LoRA technology includes: introducing a low-rank adaptation module next to a small number of attention layers in the SecureBERT base model. The module consists of a pair of low-rank decomposition matrices, and the output of the module is residually superimposed with the output of the original attention layer.

[0041] 3. Incremental learning loss function:

[0042] The learning process of the model is as follows Figure 2 As shown, its incremental learning process is constrained by the incremental learning loss function, which is a weighted combination of the following three terms:

[0043] 1) Classification Loss: Based on the samples in the replay memory (if any) and the expanded training dataset of the current round, calculate the weighted binary cross-entropy loss. Specifically, for each sample, if the label is positive, assign a weight. The negative label weight remains at 1. This weighting strategy is used to alleviate the class imbalance problem. The classification loss function is shown in formula (1) in the invention content and is calculated in combination with formulas (2) to (6).

[0044] The binary cross-entropy loss is expressed by the following formula:

[0045] ;

[0046] In the formula, Represents a threat intelligence text. The technical and tactical tag represents the multi-hot encoded form of the threat intelligence text in version t. The total number of technical and tactical tags. Representing the Each label corresponds to a one-hot encoding. Corresponding model for the first The predicted value of each technical and tactical tag. It is calculated using the following formula:

[0047] ;

[0048] ;

[0049] ;

[0050] ;

[0051] ;

[0052] In formula (2), TTPEnc represents the technical and tactical encoder. This represents the high-dimensional vector representation of threat intelligence text after it has been encoded by a tactical encoder. Its dimension is... ,in This represents the number of tokens obtained after word segmentation preprocessing for each threat intelligence text. This represents the high-dimensional vector dimension of each token after encoding. Formula (3) represents the label attention mechanism process of the model, where... Indicates about the first The first tag vector has learning capabilities. Formula (4) represents the threat intelligence text for the first tag vector. The label vector is a vector representation of each label. Equations (5) and (6) illustrate the working mechanism of the label classifier, which calculates the probability distribution of the threat intelligence text with respect to each technical and tactical label. To further alleviate the label distribution imbalance problem, Equation (1) introduces a positive sample weight parameter. The traditional binary cross-entropy loss formula is modified.

[0053] 2) First knowledge distillation loss term: using the first knowledge term... The classification model obtained from the first training round serves as the teacher model, generating soft labels from the samples in the playback memory and the training data of the current round. The student model then compares the labels output by the student model in the current round with the guidance labels generated by the teacher model by introducing a temperature hyperparameter. The scaling mechanism calculates the temperature-based Kullback-Leibler (KL) divergence between the student model output and the soft label as the knowledge distillation loss.

[0054] The distillation loss term is expressed by the following formula:

[0055] ;

[0056] In the formula, Indicates the first The number of tactical and technical labels learned by the model; and They represent the first Version of the model and the current model In the The predicted probability of technical and tactical tags. The probability calculation formula is expressed as follows:

[0057] ;

[0058] In the formula, and When respectively representing the first The version of the model and the current model for X in the th... The logit value of the tactical tag, where T represents the scaling temperature.

[0059] In a preferred embodiment, the temperature parameter It is set to 2, which helps the current model better maintain its memory of old knowledge while learning new tactical labels, thus achieving a balance between new and old knowledge.

[0060] 3) Second knowledge distillation loss term: Based on the semantic embedding information of technical and tactical labels, constrain the consistency between the classification head parameters and the semantic representation of their corresponding labels.

[0061] Its constraint loss function consists of the following formula:

[0062] ;

[0063] In the formula, Indicates the first The semantics of each technical and tactical label are obtained by encoding the corresponding label description information in the model.

[0064] As shown in formula (10), Indicates for the first An embedded representation of a tag. For the t-th round of incremental training, the MITRE ATT&CK knowledge base contains the knowledge of the t-th... Each tactical / technical tag description text; express For a dimension Real vector, where It is a positive integer. Specifically, this embedding vector, the label embeddings, are derived solely from BERT. vector.

[0065] ;

[0066] 4. Replay memory bank update:

[0067] After each round of incremental learning, the reservoir sampling algorithm is used to incrementally add the training samples of the current round to the replay memory to update the replay memory.

[0068] In the actual deployment of this invention, the replay memory is used to store threat intelligence text samples from historical rounds to support knowledge retention during incremental learning. To achieve unbiased sampling of historical data with limited storage resources, this embodiment employs a reservoir sampling algorithm to dynamically update the replay memory, specifically including the following steps:

[0069] S1: Initialize a playback memory M with a capacity of K;

[0070] S2: For the i-th new training sample received, i.e. threat intelligence text, if i<=K, then directly add it to M;

[0071] S3: If i>K, determine whether to put it into the playback memory bank with probability K / i;

[0072] S4: If a decision is made to add, then randomly select an existing sample from the replay memory M and replace it.

[0073] Specifically, the value of M is set to 5% of the training data size (rounded up).

[0074] The above method ensures that at any given time, each sample in the replay memory M has an equal probability of being retained, and the total capacity remains constant at K.

[0075] When using a playback memory, the overall loss function of the model is expressed by the following formula:

[0076] ;

[0077] ;

[0078] In formula (12), This represents the expanded training dataset of version t. This indicates the playback memory. This represents the weight value of the second knowledge distillation loss term. This represents the loss function for each training data point. Let be the overall loss function of the technical and tactical classification model, and let represent the loss function for all training samples. The average value; the overall loss function of the technical and tactical classification model is a piecewise function.

[0079] The overall loss function of the model is a piecewise function, when When the overall loss function consists only of the classification loss term, and the number of samples stored in the replay memory is 0; when At this time, the overall loss function of the model consists of the incremental learning loss function, and the samples in the replay memory will also participate in the training.

[0080] To verify the technical effectiveness of the proposed solution and to comprehensively evaluate its performance, the inventors set up several comparative baseline and incremental learning scenarios: 1. Baseline models include the standard BERT classifier, BERT-LWAN, and large language models called only through prompt engineering (such as DeepSeek-R1-70B and Qwen2.5-Max); 2. Incremental learning scenarios cover fine-tuning, knowledge distillation, data replay, full joint learning, and the fusion strategy proposed in this invention (knowledge distillation + replay).

[0081] To more effectively evaluate incremental learning performance, the dataset used in this invention's experiments is based on the MITRE ATT&CK knowledge base. MITRE divides this knowledge base into three domains: Enterprise, Mobile, and Industrial Control Systems (ICS). This experiment collected and organized all threat intelligence texts and tactical descriptions originating from the "Enterprise" domain from versions 11 to 16, classifying and organizing them according to version order to form the original dataset. This dataset contains TTPs from six versions, covering the evolving attack techniques and tactics over time, effectively reflecting the dynamic and continuously evolving characteristics of the actual threat environment. Data for each version is independently labeled, ensuring that the technical content at different time stages can be clearly distinguished, thereby supporting task partitioning and model training validation in incremental learning scenarios.

[0082] The experimental evaluation metrics used in this invention are Precision, Recall, F1-score, and Hamming Loss, which are commonly used in multi-label classification tasks. Considering that accuracy is too stringent and not entirely applicable in TTP classification, the accuracy metric was not used in the experimental evaluation of this invention.

[0083] Hamming Loss: This is a metric used to evaluate the performance of multi-label classification models. It measures the frequency of prediction errors across all samples and labels, as shown in Equation (13). The lower the Hamming Loss, the better the classification performance of the algorithm.

[0084] ;

[0085] In the formula, The indicator function is represented by Nm, which is the size of the sample set; Lt is the number of technical and tactical labels that the model has mastered after the t-th round of training. This represents the true value of the i-th sample at the j-th label (its value is 0 or 1); This represents the model's predicted value for the j-th label of the i-th sample (its value is 0 or 1).

[0086] In practical calculations, this invention uses the scikit-learn tool library to calculate the aforementioned metrics. Specifically, when calculating Precision, Recall, and F1-score, the `average` parameter is set to "samples," meaning the metric is calculated individually for each sample and then averaged. For example, when calculating Precision, assuming the predicted value for a batch of samples is: The corresponding real tags are: The precision of this batch of samples is given by equations (14)-(16). This more fairly reflects the overall performance of the model in the multi-label classification task.

[0087] ;

[0088] ;

[0089] ;

[0090] The performance evaluation process is designed as follows: after the model has learned the first... After obtaining the latest version of the MITRE knowledge base, the tags for threat intelligence texts in the previous test set are updated based on this latest version of the MITRE knowledge base. Subsequently, the aforementioned evaluation metrics are used to assess the model presented in this paper across all test sets in different learning scenarios. The evaluation is conducted. Recall, precision, and F1 score are all expressed as percentages.

[0091] Furthermore, to quantitatively evaluate the advantages of the model described in this invention in terms of resource consumption, its classification performance was compared with that of the BERT model in a full-scale learning scenario. Simultaneously, to verify its incremental learning capability under resource-constrained conditions, it was further evaluated using a test set rewritten with a Large Language Model (LLM). The expansion of the training data was achieved through BERT's Masked Language Modeling (MLM) strategy.

[0092] Table 2 presents the experimental evaluation results. In the fine-tuning learning scenario, all comparative models exhibited significant catastrophic forgetting, meaning that the model's ability to retain knowledge of old tasks decreased sharply when learning new tasks. However, different model architectures showed varying sensitivities to forgetting. Specifically, models employing label-aware attention mechanisms (the model proposed in this invention and BERT-LWAN) significantly outperformed models based on standard BERT fine-tuning at all stages (v11–v16), consistently leading in both precision and recall. Especially in the initial learning stage (v11), the label-attention-based model demonstrated stronger task discrimination capabilities.

[0093] Table 2. Experimental Evaluation Results .

[0094] In the knowledge distillation scenario, this invention introduces the KL divergence loss function into the learning process of the BERT and BERT-LWAN models to constrain the consistency of the output probability distributions of the old and new models, thereby comparing them with the model of this invention as fairly as possible. Experimental results show that, compared with the fine-tuning strategy, knowledge distillation significantly improves the incremental learning ability of the model and effectively alleviates the knowledge forgetting problem in TTPs classification tasks. Furthermore, the model of this invention, which combines two types of knowledge distillation (this invention-distillation), outperforms the BERT-LWAN-distill model with single-class knowledge distillation in both average precision and recall, indicating that two types of knowledge distillation help to more efficiently integrate new and old task knowledge.

[0095] In the replay learning scenario, the model is retrained by storing a small number of historical samples, which also significantly reduces the degree of forgetting. Compared with knowledge distillation, the replay strategy performs better in suppressing forgetting. It is worth noting that the model of this invention achieves higher recall in the distillation scenario and higher precision in the replay scenario. By combining the advantages of the two strategies, this invention achieves a good balance between precision and recall, demonstrating excellent generalization ability.

[0096] Furthermore, the experiments in this invention employ a prompt-based approach to invoke Large Language Models (LLMs) to perform TTPs classification tasks. Experimental results show that, under the same dataset and evaluation conditions, the model proposed in this invention outperforms baseline methods based on General Large Language Models (LLMs) in terms of classification accuracy, F1 score, and other metrics across various learning scenarios.

[0097] Table 3 Comparison of Resource Consumption .

[0098] Table 3 shows a comparison of the classification performance of the present invention with that of the BERT model in a full-training scenario. The average F1 score of the present invention model in six iterations (v11–v16) is 85.5, slightly lower than that of the full-training model (88.0), and the Hamming loss is slightly higher by 0.0002. However, in terms of computing resources, the present invention model consumes only 65.1% of the GPU resources required for full training, significantly improving training efficiency and making it suitable for resource-constrained deployment environments.

[0099] Table 4 Comparison of F1 scores under resource-constrained conditions .

[0100] Table 4 shows the incremental learning capability of this invention under resource-constrained conditions (training data augmentation is achieved through BERT's masked language modeling strategy). The model's average F1 score on the original test set is 85.5, and it maintains an F1 score of 82.8 on the rewritten test set, with a decrease of only 2.7 percentage points. This result demonstrates that the model of this invention possesses good incremental learning capability under resource-constrained conditions.

Claims

1. A tactical and technical classification method based on incremental learning, characterized in that, Includes the following steps: Step 1: Data Collection: In the t-th round of incremental learning, acquire the incremental update data of the MITRE ATT&CK knowledge base; the incremental update data is the TTPs information added or updated in the t-th round; Where t is an integer greater than or equal to 1; perform data preprocessing on the incremental update data; Step 2: Construct a basic training dataset based on the incrementally updated data, and then perform semantic enhancement on the threat intelligence text in the basic training dataset using a large language model to obtain the expanded training dataset. ; Step 3: Construct a low-rank adaptive fine-tuned SecureBERT model as the semantic understanding foundation for a tactics and techniques classification model. This model extracts semantic features from threat intelligence text and tactical label description text, and classifies attack tactics and techniques. The tactical classification model includes a tactical encoder, a label attention mechanism, and a label classifier, and uses the expanded training dataset. Compared to a playback memory that employs a reservoir sampling strategy As input, At that time, the number of samples in the replay memory was empty; Step 4: When t=1, based on the expanded training dataset, the low-rank adaptive technique is used to train the tactical classification model to obtain the initial tactical classification model; When t>1, based on the expanded training dataset and the training samples selected by the reservoir sampling in the first t-1 rounds in the replay memory M, the low-rank adaptive technique is used to incrementally train the t-1 round training of the t-t t-t classification model to obtain the t-t round t-t classification model; and the t-t round t-t classification model is used to output the classification results of the t-t skills and tactics in the threat intelligence text. The incremental learning process is constrained by the incremental learning loss function. After each round of incremental learning, the reservoir sampling algorithm is used to incrementally add the training samples of the current round to the playback memory to update the playback memory.

2. The tactical classification method based on incremental learning according to claim 1, characterized in that, In step 1, the data preprocessing of the incremental update data is specifically performed as follows: various vulnerability indicators appearing in the threat intelligence text and tactical tag description text in the incremental update data are replaced with preset general placeholders that represent their types.

3. The tactical and technical classification method based on incremental learning according to claim 2, characterized in that, In step 2, expanding the training dataset specifically involves: By using a pre-trained model based on masked language modeling, words in similar threat intelligence texts are randomly masked, and the pre-trained model predicts the masked content to generate semantically preserved fill-in-the-blank samples; or by using a pre-trained language model based on an autoregressive mechanism, the original threat intelligence text is used as input prompts, and text generation technology is used to generate rewritten samples with the original meaning unchanged but different expression forms, thereby expanding the training data. The text generation technology employs the following methods: 1) Pre-training a SecureBERT model on a cybersecurity corpus to perform a masked language modeling task, randomly masking several words in the input text, using the model to predict the masked content, and generating new legal sentences; or 2) utilizing the text generation capabilities of a generative language model to perform synonym rewriting, generating diverse training samples.

4. The tactical and technical classification method based on incremental learning according to claim 1, characterized in that, In step 3, the tactical encoder is a BERT model based on low-rank adaptive fine-tuning, whose input only includes threat intelligence text and label description text; wherein, during the training phase... At that time, the BERT model used is the SecureBERT model pre-trained on a cybersecurity corpus; during the training phase At that time, the initial parameters of the technical and tactical encoder are taken from the technical and tactical encoder parameters of stage t-1; The label attention mechanism includes a matrix representing technical and tactical labels, which has learning capabilities; The label classifier includes a set of learnable weight vectors, takes the output of the label attention mechanism as input, and transforms it into... This indicates the classification of threatening behaviors; Both the tag attention mechanism and the tag classifier support the addition of new tactical tags and achieve near-linear parameter growth through a lightweight design.

5. The tactical and technical classification method based on incremental learning according to claim 1, characterized in that, In step 4, when t=1, during the training process, only the low-rank adaptation parameters and classification-related parameters introduced by the low-rank adaptation technique are updated, while the original parameters of the SecureBERT model are frozen. When t>1, during incremental training, only the low-rank adaptation parameters and classification-related parameters introduced by the low-rank adaptation technique are updated, while the SecureBERT model parameters in the classification model in the (t-1)th round remain unchanged. Training or incremental training using low-rank adaptive techniques includes: introducing a low-rank adaptation module next to the attention layer of the SecureBERT model, wherein the low-rank adaptation module consists of a pair of low-rank decomposition matrices, and residual superimposing the output of the low-rank adaptation module with the output of the original attention layer.

6. The tactical classification method based on incremental learning according to claim 1, characterized in that, The incremental learning loss function is a weighted combination of the following three factors: 1) Classification loss item: Based on the samples in the replay memory and the expanded training dataset of the current round, a weighted binary cross-entropy loss is calculated; specifically, for each sample, if the label is positive, the positive label is assigned a weight of . The negative label weight remains at 1; the specific calculation is as follows: The binary cross-entropy loss It can be expressed by the following formula: ； In the formula, Represents a threat intelligence text. represent The tactical tag in the multi-hot encoding format corresponding to version t; Total number of technical and tactical tags; Positive label weight; Representing the One-hot encoding for each label Corresponding model for the first Predicted values ​​for each technical and tactical tag; It is calculated using the following formula: ； ； ； ； ； In formula (2), TTPEnc represents the technical encoder; This represents the high-dimensional vector representation of threat intelligence text after it has been encoded by a tactical encoder. The dimension is... ,in This represents the number of tokens obtained after word segmentation preprocessing for each threat intelligence text. This represents the high-dimensional vector dimension of each token after encoding; Formula (3) represents the label attention mechanism of the model, where Indicates about the first A label vector; This is the high-dimensional vector representation of the nth token; Let H be the nth column vector representation; for and Attention scores between; In formula (4), This indicates that the threat intelligence text is related to the first... Vector representation of each label; Formulas (5) and (6) represent the working mechanism of the tag classifier, which is used to calculate the probability distribution of the threat intelligence text with respect to each technical and tactical tag; For the current model, the x-th The logit value of the technical and tactical tag; For the first The sigmoid activation function associated with each technical and tactical tag; This is the weight vector of the corresponding sigmoid function; This is the bias term of the corresponding sigmoid function; 2) First knowledge distillation loss term: Using the first The classification model obtained from the first training round serves as the teacher model, generating soft labels for the samples in the replay memory and the training data of the current round. The student model, through a temperature hyperparameter scaling mechanism, calculates the KL divergence between the student model's output and the soft labels generated by the teacher model, as the knowledge distillation loss. As shown in the following formula: ； In the formula, Indicates the first The number of tactical and technical labels learned by the version model; and They represent the first Version model and current model for X in the 1st month The predicted probability of technical and tactical tags is calculated using the following formula: ； In the formula, Indicates the first Version model for X in the 1st The logit value of the technical and tactical tag; T represents the scaling temperature; (3) Second knowledge distillation loss item: Based on the semantic embedding information of technical and tactical tags, the consistency between the classification head parameters and their corresponding tag semantic representations is constrained; the loss function is constrained. As shown in the following formula: ； In the formula, For the first The semantics of the tactical tag, representing the meaning of the first tactical tag. Embedded representation of a tag; The embedding of the tags is solely derived from BERT. Vectors are represented as follows: ； In the formula, For the t-th round of incremental training, the MITRE ATT&CK knowledge base contains the knowledge of the t-th... Each tactical / technical tag description text; express For a dimension , where It is a positive integer.

7. The tactical classification method based on incremental learning according to claim 6, characterized in that, The update of the playback memory specifically involves: Step a: Initialize a playback memory M with a capacity of K; Step b: For the i-th new training sample received, i.e. threat intelligence text, if i<=K, then directly add it to M; Step c: If i > K, decide whether to put it into the playback memory bank with probability K / i; Step d: If it is decided to add, then randomly select an existing sample from the replay memory M and replace it.

8. The tactical classification method based on incremental learning according to claim 7, characterized in that, When using a playback memory, the overall loss function of the tactics classification model is expressed by the following formula: ； ； In the formula, This refers to the expanded training dataset. Indicates the playback memory; This represents the weight value of the second knowledge distillation loss term; This represents the loss function for each training data point. Let be the overall loss function of the technical and tactical classification model, and let represent the loss function for all training samples. The average value; the overall loss function of the technical and tactical classification model is a piecewise function; when At this time, the overall loss function consists only of the classification loss term, and the number of samples stored in the replay memory is 0. when At this time, the overall loss function consists of the incremental learning loss function, and the samples in the replay memory will also participate in the training.

Images