Training a matching model for recognizing tactics, techniques, and procedures within text segments

Abstract

The disclosure relates to training a matching model for recognizing Tactics, Techniques, and Procedures (TTPs) within text segments. The disclosure proposes a device, and a corresponding method. The method comprises obtaining one or more batches, wherein each batch comprises one positive pair for a text segment of the batch, one or more negative pairs for the text segment of the batch, and two or more labels for said two or more pairs, wherein each positive pair comprises a text segment and a TTP candidate that match with each other and each negative pair comprises a text segment and a TTP candidate that do not match with each other, determining, for each pair of each batch, a matching score based on the matching model and the pair, and adjusting the matching model based on the matching score of each pair of each batch and the two or more labels.

Description

[0001] TRAINING A MATCHING MODEL FOR RECOGNIZING TACTICS, TECHNIQUES, AND PROCEDURES WITHIN TEXT SEGMENTS

[0002] TECHNICAL FIELD

[0003] The disclosure relates to a model for recognizing Tactics, Techniques, and Procedures (TTPs). The disclosure proposes a device, and a corresponding method for training the model.

[0004] BACKGROUND

[0005] Cyber Threat Intelligence (CTI) is information about security threats in the cyber domain. CTI describes real-world entities (e.g., threat actors, victims, organizations, individuals, etc.), cyber entities (e.g., domain names, IP addresses, email addresses, files, etc.), geographic locations (e.g., victim address), patterns for threat detection (e.g., file hashes, domain names, etc.) and many other concepts related to cybersecurity.

[0006] Indicator of Compromise (IOC) is a concrete evidence of an attack. For example, the hash of a malware executable, the IP address of a command-and-control server, the name of a domain used for sending phishing email, etc.

[0007] TTPs are a high-level characterization of an attack method. The purpose of TTPs is to systematize known attack methods by providing a clear description of their properties. For example, “compromise via removable media” achieves the tactic of gaining access to a device, using the technique of getting the target device to process removable media, which may be implemented using a USB stick or CD (procedure). The abstractions provided by TTPs are valuable to defenders as they enable them to reason about attacker goals and behavior at the tactical level, which is not possible with loCs.

[0008] One of the most valuable and challenging subtasks of extracting CTI from textual sources (e.g., threat reports) is the mining of TTPs. The task can be defined as understanding an unstructured text, which describes one or more low-level threat actions, and linking it to techniques described and categorized by a common knowledge base (e.g., MITRE ATT&CK). Knowing what techniques are used by the attacker, and their temporal order, enables security analysts to know their enemy, their goals and modus operandi, which gives them an advantage in securing their assets. There are over 500 techniques and subtechniques described in MITRE ATT&CK, which is an industry standard knowledge base of TTPs. As defined by MITRE, a technique is associated to at least one tactic (for example, the technique “Hijack Execution Flow” is under 3 distinct tactics: Persistence, Privilege Escalation and Defense Evasion) and often has several subtechniques. The large number of techniques, their variety, complex inter-dependencies and hierarchical structure make the task of mining techniques from CTI reports challenging. Additionally, CTI reports are written with a human audience in mind, and do not mention techniques explicitly. Instead, they only describe low-level threat actions, from which the reader then needs to infer the techniques and tactics.

[0009] Conventionally, TTP mining has been addressed by, for example, keyword- and patternmatching, which suffers from low accuracy, and leveraging a probabilistic relevance framework to quantify the similarity between two bag-of-words representations: a TTP description (e.g., from MITRE ATT&CK) and the text of interest. However, such a conventional method can only work in restricted settings where a technique is represented as a single action in a predefined ontology and the vocabulary used in CTI reports is greatly simplified and overlapped with the ontology.

[0010] Other conventional approaches are based on a multi-label / multi-class classification learning paradigm and / or a term frequency-inverse document frequency (TF-IDF) score, for example, of the independent noun phrase in security analysis articles, to find the keywords to represent the TTPs, wherein Keywords are used to query analysis articles in their corpus. However, such approaches require large existing labeled datasets, wherein the cost of labeling due to the need for expert security analysists is high. Further, generally, a long-tailed skewed data distribution across labels is expected, i.e., some techniques are very rare compared to others, and only few examples exist in any dataset.

[0011] The complexity cost of traditional (e.g., tree-based) and modern (i.e., / tmax-based) machine learning (ML) approaches rises proportionally with the number of classes. The hierarchical softmax and negative sampling approaches showed their effectiveness in self-supervised learning settings, however, these approaches show limitations when applied to data-scarce scenarios, as the signal-to-noise ratio increases. SUMMARY

[0012] In view of the above, this disclosure aims to provide an improved training method that enables more accurate and / or efficient TTP mining. One objective is to minimize the cost of data labeling, which is very expensive in this domain, by utilizing data-efficient machine learning methods. Another objective is to identify all implicitly mentioned TTPs in texts of arbitrary length. Further, another objective is to enable to map to long-tailed TTPs, e.g. TTPs with much fewer number of samples in training, and / or to identify completely novel TTPs, e.g. TTPs with no examples in the training set.

[0013] These and other objectives are achieved by this disclosure as described in the enclosed independent claims. Advantageous implementations are further defined in the dependent claims.

[0014] A first aspect of this disclosure provides a method for training a matching model for recognizing TTPs within text segments, for example natural language text segments, wherein the method comprises: obtaining one or more batches, wherein each batch comprises one positive pair for a text segment of the batch, one or more negative pairs for the text segment of the batch, and two or more labels for said two or more pairs, wherein each label indicates whether a respective pair is a positive pair or a negative pair, wherein each positive pair comprises a text segment and a TTP candidate that match with each other and each negative pair comprises a text segment and a TTP candidate that do not match with each other, determining, for each pair of each batch, a matching score based on the matching model and the pair, and adjusting the matching model based on the matching score of each pair of each batch and the two or more labels.

[0015] The method may be based on a new learning paradigm according to this disclosure. The learning paradigm may be applied for other applications for training a model. The training of the model may comprise learning by the model to recognize TTPs within text segments.

[0016] The method may be considered a method for learning with a matching model.

[0017] The TTP candidates of each pair of each batch may form a set of TTP candidates. Each pair may only comprise TTP candidates that are comprised in the set of TTP candidates. The set of TTP candidates may be predetermined and / or may be based on a knowledge base (KB), for example, the MITRE KB or the CAPEC KB. For example, the set of TTP candidates may be determined based on a probabilistic retrieval model.

[0018] Each batch may comprise only one positive pair for the text segment of the batch.

[0019] Each batch may comprise two or more negative pairs for the text segment of the batch.

[0020] Each batch may comprise more negative pairs for the text segment of the batch than positive pairs for the text segment of the batch. For example, each batch may comprise two times more negative pairs for the text segment of the batch than positive pairs for the text segment of the batch.

[0021] Each batch may be associated with at least one text segment. For example, each batch may be associated with only one text segment.

[0022] Each batch may comprise respectively only one text segment, and / or wherein, for each batch, each pair of the batch may comprise the same text segment. Alternatively, each batch may comprise respectively two or more text segments.

[0023] The text segment of the batch may be comprised in a set of text segments of the batch, and each batch may respectively comprise for some or each text segment of the set of text segments of the batch: one positive pair, for example only one positive pair, one or more negative pairs, for example two or more pairs, and two or more labels for said two or more pairs.

[0024] Each batch may respectively comprise, for some or each text segment in the set of text segments of the batch, more negative pairs than positive pairs. For example, each batch may comprise, for each text segment in the set of text segments of the batch, two times more negative pairs for the text segment of the batch than positive pairs for the text segment of the batch.

[0025] The matching model may be a security technique mining model or may be comprised in a security technique mining model. The method and / or performing inference with the matching model may comprise, for example for each forward pass, receiving a text segment, mining the semantics of the text segment, and associating the text segment to at least one TTP.

[0026] The text segment and / or the text segments in the set of text segments may be natural language text segments.

[0027] In an implementation form of the first aspect, the method further comprises encoding and comparing, for each pair of each batch, the pair based on the matching model.

[0028] In a further implementation form of the first aspect, the matching model comprises an encoding layer, wherein the method comprises: encoding, for each pair of each batch, the text segment and the TTP candidate of the pair into a joint vector space based on the encoding layer to form two vectors, determining, for each pair of each batch, the matching score based on the matching model and the two vectors, adjusting the matching model, for example the alignment layer of the matching model, and / or the encoding layer based on the matching score of each pair of each batch and the two or more labels.

[0029] Adjusting the matching model may comprise adjusting the alignment layer of the matching model. Alternatively or additionally, adjusting the matching model may comprise adjusting other layers of the matching model.

[0030] In a further implementation form of the first aspect, for each pair of each batch, the text segment and the TTP candidate are encoded separately into a respective vector of the two vectors, and wherein the method comprises: determining, for each pair of each batch, the matching score based on the matching model and the two vectors by comparing the two vectors with each other, adjusting the matching model and / or the encoding layer based on comparing the matching score of each pair of each batch with the two or more labels.

[0031] In a further implementation form of the first aspect, for each pair of each batch, the text segment is at least a part of a CTI report, and / or the TTP candidate is a predefined textual definition of an attack method that is described in the at least part of the CTI report, if the pair is a positive pair, or the TTP candidate is a predefined textual definition of an attack method that is not described in the at least part of the CTI report, if the pair is a negative pair. In a further implementation form of the first aspect, the method further comprises training the matching model based on a Noise Contrastive Estimation (NCE) loss function, an InfoNCE loss function, or a relaxed version of an InfoNCE loss function for enabling to rank, for each batch, the TTP candidates of the two or more pairs.

[0032] For example, a relaxed version of an InfoNCE loss function may relaxably admit the ranking of TTP candidates, for example the set of TTP candidates. For example, a higher label score (e.g., 5 on a scale from 0-5) may be given to a first TTP candidate 106a compared to other TTP candidates 106b, indicating that it is more important to match to the first TTP candidate 106a. It may be more important to match to the first TTP candidate if, for example, a security analyst considers the first TTP candidate as a most appropriate match to the respective text segment. The importance of TTP candidates for each text segment may be predetermined.

[0033] In a further implementation form of the first aspect, the one or more batches are based on corpus-level negative sampling comprising at least one of random sampling, retrieval-based sampling for deriving hard-negative samples from a probabilistic retrieval model, and semantic structure-based sampling for deriving hard-negative samples from sibling TTPs.

[0034] In a further implementation form of the first aspect, the matching model comprises at least one of an embedding layer, an encoder layer, for example the encoding layer of above, an alignment layer, a fusion layer, a residual block layer, a pooling layer, and a merging layer.

[0035] In a further implementation form of the first aspect, the method further comprises token-level embedding, with the embedding layer, of each pair of each batch for forming a respective local representation of the text segment and the TTP candidate of the pair.

[0036] In a further implementation form of the first aspect, the method further comprises determining, with the encoder layer, for each pair of each batch, the two vectors based on the pair, for example based on the respective local representation of the text segment and the TTP candidate.

[0037] The method may further comprise receiving, with the encoder layer, for each pair of each batch, the respective local representations of the text segment and the TTP candidate. The encoder layer may be a multi-layer convolutional network.

[0038] In a further implementation form of the first aspect, the method further comprises receiving, with the alignment layer, for each pair of each batch, the two vectors from the encoding layer, determining, with the alignment layer, for each pair of each batch, two interaction-driven representations of the two vectors, for example based on a trainable projection matrix.

[0039] Adjusting the matching model may comprise adjusting the trainable projection matrix.

[0040] In a further implementation form of the first aspect, the method further comprises receiving, with the fusion layer, for each pair of each batch, two intermediate vectors based on the two vectors from previous layers of the matching model, for example the two interaction-driven representations of the two vectors, comparing and fusing together, with the fusion layer, for each pair of each batch, the local representations of the pair and / or the two intermediate vectors, for example the two interaction-driven representations of the two vectors, to form two fused vectors.

[0041] In a further implementation form of the first aspect, the method further comprises: receiving, with the pooling layer, for each pair of each batch, two intermediate vectors based on the two vectors from previous layers of the matching model, for example the two fused vectors, consolidating, with the pooling layer, for each pair of each batch, the two intermediate vectors, for example the two fused vectors, into fixed-length vector representations to form two output vectors.

[0042] In a further implementation form of the first aspect, the method further comprises creating, with the residual block layer, a plurality of blocks, wherein each block comprises the encoder layer, the alignment layer, and the fusion layer, and wherein the method comprises determining the matching score based on each block of the plurality of blocks.

[0043] For each pair of each batch, the encoder layer, the alignment layer, and the fusion layer may be used multiple times. For example, the method may comprise, for each pair of each batch, multiple forward passes of the pair through the encoder layer, the alignment layer, and the fusion layer. The matching score of the pair may be based on said multiple forward passes. Each forward pass of the pair may be considered as one block of the residual block layer. For each pair of each batch, the output of the residual block layer may be two output vectors.

[0044] In a further implementation form of the first aspect, the method further comprises: receiving, with the merging layer, for each pair of each batch, two intermediate vectors based on the two vectors from previous layers of the matching model, for example the two output vectors, determining, with the merging layer, for each pair of each batch, similarities between the two intermediate vectors, for example the two output vectors, to form the matching score.

[0045] For example, the method may comprise applying with the merging layer a simple inner product and / or another Multi-Layer Perception (MLP).

[0046] In a further implementation form of the first aspect, the matching model is based on a dualencoder architecture or a cross-encoder architecture.

[0047] In a further implementation form of the first aspect, the cross-encoder architecture comprises an alignment layer before or together with the encoding layer.

[0048] In a further implementation form of the first aspect, determining, for each pair of each batch, the matching score is based on exploiting semantic similarities of the text segment and the TTP candidate.

[0049] In a further implementation form of the first aspect, the method may further comprise iterating the above method steps of the first aspect and / or the implementation forms of the first aspect multiple times, for example, until convergence of the matching model.

[0050] A second aspect of this disclosure provides a device for training a matching model for recognizing TTPs within text segments, wherein the device comprises a processor configured to: obtain one or more batches, wherein each batch comprises one positive pair for a text segment of the batch, one or more negative pairs for the text segment of the batch, and two or more labels for said two or more pairs, wherein each label indicates whether a respective pair is a positive pair or a negative pair, wherein each positive pair comprises a text segment and a TTP candidate that match with each other and each negative pair comprises a text segment and a TTP candidate that do not match with each other, determine, for each pair of each batch, a matching score based on the matching model and the pair, and adjust the matching model based on the matching score of each pair of each batch and the two or more labels.

[0051] The device of the second aspect may have implementation forms that correspond to the implementation forms of the method of the first aspect. The device of the second aspect and its implementation forms achieve the advantages and effects described above for the method of the first aspect and its respective implementation forms.

[0052] A third aspect of this disclosure provides a computer program product comprising a program code for performing, when the program code is executed on a computer, the method according to one of the implementation forms of the first aspect.

[0053] In this disclosure the phrase “matching model” and “matching network” may be used interchangeably.

[0054] Further, in this disclosure the phrase “network” and “layer” may be used interchangeably.

[0055] Further, in this disclosure the phrase “encoding layer” and “encoder layer” may be used interchangeably.

[0056] Further, in this disclosure the phrase “merge layer” and “merging layer” may be used interchangeably.

[0057] Further, in this disclosure the phrase “classifying TTPs” and “recognizing TTPs” may be used interchangeably.

[0058] Further, in this disclosure the phrase “technique” may refer to a “technique” and / or a “subtechnique”.

[0059] It has to be noted that all devices, elements, units and means described in the disclosure could be implemented in the software or hardware elements or any kind of combination thereof. All steps which are performed by the various entities described in the disclosure as well as the functionalities described to be performed by the various entities are intended to mean that the respective entity is adapted to or configured to perform the respective steps and functionalities. Even if, in the following description of specific embodiments, a specific functionality or step to be performed by external entities is not reflected in the description of a specific detailed element of that entity which performs that specific step or functionality, it should be clear for a skilled person that these methods and functionalities can be implemented in respective software or hardware elements, or any kind of combination thereof.

[0060] BRIEF DESCRIPTION OF DRAWINGS

[0061] The above described aspects and implementation forms will be explained in the following description of specific embodiments in relation to the enclosed drawings, in which

[0062] FIG. 1 shows a method according to an embodiment of this disclosure.

[0063] FIG. 2 shows a device according to an embodiment of this disclosure.

[0064] FIG. 3 shows a flowchart for TTP Mining according to an embodiment of this disclosure.

[0065] FIG. 4 shows a diagram for matching an input space to a label space according to an embodiment of this disclosure.

[0066] FIG. 5 shows a matching model according to a new learning paradigm for label informed text matching according to an embodiment of this disclosure.

[0067] FIG. 6 shows an exemplary system for TTP mining according to an embodiment of this disclosure.

[0068] FIG. 7 shows a flowchart of exemplary steps for deploying a system according to an embodiment of this disclosure.

[0069] FIG. 8 shows an exemplary system pipeline for training and inference according to an embodiment of this disclosure.

[0070] FIG. 9 shows an exemplary component-based architecture of a matching model according to an embodiment of this disclosure.

[0071] FIG. 10 shows a cross-encoder architecture of a matching model according to an embodiment of this disclosure.

[0072] DETAILED DESCRIPTION OF EMBODIMENTS

[0073] FIG. 1 shows a method 200 according to an embodiment of this disclosure. The method 200 may be performed by a device 100, for example, as shown in FIG. 2. The method 200 comprises a step 201 of obtaining 201 one or more batches 102, wherein each batch comprises one positive pair 103a for a text segment of the batch 102a, one or more negative pairs 103b for the text segment 105a of the batch 102a, and two or more labels 104 for said two or more pairs 103. Further, the method 200 comprises a step 202 of determining 202, for each pair of each batch, a matching score 107 based on the matching model 101 and the pair 103a, 103b. Further, the method 200 comprises a step 203 of adjusting 203 the matching model 101 based on the matching score 107 of each pair of each batch and the two or more labels 104.

[0074] Generally, each label 104a, 104b indicates whether a respective pair is a positive pair 103a or a negative pair 103b. Each pair 103a, 103b comprises a text segment 105a and a TTP candidate 106a, 106b. Each positive pair 103a comprises a text segment 105a and a TTP candidate 106a that match with each other and each negative pair 103b comprises a text segment 105a and a TTP candidate 106b that do not match with each other.

[0075] FIG. 2 shows a device 100 according to an embodiment of this disclosure. The device 100 comprises a processer 108 configured to obtain one or more batches 102. Further, FIG. 2 shows two or more pairs 103 and two or more labels 104 that are comprised in one batch 102a. Each batch 102a comprises one positive pair 103a for a text segment 105a of the batch 102a, one or more negative pairs 103b for the text segment 105a of the batch 102a, and two or more labels 104 for said two or more pairs 103.

[0076] Each pair 103a, 103b comprises a text segment 105a and a TTP candidate 106a, 106b. Each positive pair 103a comprises a text segment 105a and a TTP candidate 106a that match with each other and each negative pair 103b comprises a text segment 105a and a TTP candidate 106b that do not match with each other. Each pair of the two or more pairs 103 may correspond to only one label of the two or more labels 104. Generally, each label 104a, 104b indicates whether a respective pair is a positive pair 103a or a negative pair 103b.

[0077] Each batch of the one or more batches 102 may further comprise additional pairs and corresponding labels for other text segments of the batch 102a, wherein each batch comprises for each text segment of the batch 102a one positive pair 103 a, one or more negative pairs 103b, and two or more labels 104.

[0078] The processer 108 is configured to determine 202, for each pair of each batch, a matching score 107 based on the matching model 101 and the pair 103a, 103b, and adjust 203 the matching model 101 based on the matching score 107 of each pair of each batch and the two or more labels 104. FIG. 3 shows a flowchart for TTP Mining according to an embodiment of this disclosure. TTP mining may be performed by the device 100 or another device. The first step of the flow chart shows an exemplary text segment 105a to be mined. The second step of the flowchart indicates performing TTP mining, for example the text segment 105a may be analyzed and mapped to one relevant TTP candidate 106a in a KB, for example, the MITRE KB. The third step of the flow chart shows an exemplary TTP 106a that was mapped or matched to the text segment 105a.

[0079] The TTP mining according to embodiments of this disclosure may be considered as learningbased label-informed text matching, which utilize matching and classification.

[0080] The mapping / classification task of conventional methods may be replaced with a matching task for training and / or inference with a matching model according to this disclosure. Orthogonal to comparing between TTP candidates 106a, 106b in the KB, the driving signal for matching may come from the direct alignment between a text segment 105a and a TTP candidate 106a.

[0081] The optimization for the differentiation of data points in a large label space, which often is prohibitively expensive, can be avoided. Instead, the direct semantic similarity of the input- versus-label pair to derive a calibrated assignment score may be orthogonally exploited. Thus, the classification function that separates labels may be turned into a unified form of matching (comparing) pairs of text. This unified framework may alleviate the problem of the long-tailed class label distribution, and at the same time may offer a natural inductive bias that may make it work also for unseen TTPs, e.g. TTPs with no examples in the training set.

[0082] The KB, e.g., MITRE ATT&CK, may be evolving through time and may receive regular updates. Thus, a TTP mining approach that can identify completely novel TTPs, e.g. TTPs with no examples in the training set, can be automatically updated based on the ATT&CK update.

[0083] Textual profiles of TTPs may be created, which may be derived from their high-quality textual descriptions in, for example, the MITRE ATT&CK or CAPEC KB. This textual resource, for example said high-quality textual descriptions, may be neglected in the classification settings and may reduce the requirements for labeled data. Further, input text may be projected into the same latent space as the TTPs, wherein both mappings may be learned, not manually engineered. Then, the joint latent space can enable straightforward similarity measurements between pairs (input text, TTP description). Further, an NCE loss to encourage a discriminative distance between positive and negative samples and at the same time increase training efficiency may be applied.

[0084] Instead of comparing a text segment 105a with all TTP candidates at once, a sample of negative TTP candidates may be selected and compared against at training time on a batch-level. This allows a label discriminating operation to perform, similar to a classification setting, while enhancing efficiency in the relatively large label space. The application of a NCE based loss function is mathematically sound and shows effectiveness in empirical (experimental) studies.

[0085] FIG. 4 shows a diagram for matching an input space to a label space according to an embodiment of this disclosure. Points in the input space, for example text segments from a CTI report, and points in the label space, for example TTP definitions, are projected into a latent space according the functions f() and h(), respectively, which enables direct similarity measurements. The method 200 fortraining may comprise performing said projections. Further, a method for inference with the matching model 101 may comprise performing said proj ections.

[0086] Formally, a task according to embodiments of this disclosure may be defined as follows:

[0087] Given the training data X G RnxmtY G Rnxlwhere I < co and the supervision {x -> y}n= {0,l]n, such as x E X and y E Y. The target is to learn a matching family of functions fewith the following inductive bias properties:

[0088] • Asymmetric, to satisfy the one-to-many mapping, asymmetric properties may be determined and / or detected.

[0089] • Boundedness and non-negativity-. the similarity score is in [0, 1],

[0090] • Indiscernibility of identicals or strong reflexivity-, the similarity of a text to itself is 1.

[0091] • Triangle inequality or transitivity-, this geometric property holds true for any metric space.

[0092] Embodiments of this disclosure may be based on a new learning paradigm for TTP mining that exploits altogether (1) the information sharing signal between a pair, for example a textual input pair, and (2) the discriminative supervisory signal in the label space. With an adoption of the NCE loss, the method 200 may alleviate problems in classical multi-label classification settings. To exploit the information exchange between the target and the source text, an interaction-based neural matching architecture that is tailored to the task may be used. Corpus-level negative sampling strategies for effective label discrimination and training efficiency may be used.

[0093] FIG. 5 shows a matching model 101 according to a new learning paradigm for label informed text matching according to an embodiment of this disclosure. A matching model 101 may receive one or more batches 102, wherein each batch at least comprises one text segment 105a, one positive TTP candidate 106a, and one or more negative TTP candidates 106b, wherein the positive TTP candidate 106a matches with the text segment 105a, and wherein the one or more negative TTP candidates 106b do not match with the text segment 105a. The matching model 101 may comprise an encoder, a ranker, for example an alignment layer, and an aggregator. The aggregator may aggregate binomial probability distributions, which indicate the relevancy of a text segment 105a to a certain TTP candidate 106a, 106b. The new (aggregated) score distribution over all TTP candidates may be leveraged for determining a cut-off between relevant and irrelevant TTP candidates regarding the text segment 105a.

[0094] FIG. 6 shows an exemplary system 500 for TTP mining according to an embodiment of this disclosure. The system may comprise the device 100. Alternatively, the system may not comprise the device 100 and training the matching model may not be performed with the system 500. The input of the system 500 may be a threat report, e.g., a recently to the Web released threat report. The system comprises a backend knowledge base (e.g., MITRE ATT&CK). The applications where the system’s output can be used are, for example, threat hunting and automated detection and response (ADR). The system may be configured to perform inference with the matching model 101, wherein the matching model 101 may be trained in the device 100.

[0095] The system 500 may comprise at least one of

[0096] • A web crawler: This component browses the Web, detects fresh CTI reports from reliable sources and downloads them. Reliable sources can be pre-identified or discovered using an authority estimation algorithm (e.g., PageRank).

[0097] • A HTML / PDF parser: This component reads the content of the downloaded resources, often in HTML or PDF format, parses them, and extracts clean and relevant text. It applies off-the-shelf or customized boilerplate removal to discard irrelevant text, such as HTML navigation elements or copyright notices. Further advanced steps are also used here to discard e.g., source code, lists, images, etc.

[0098] • A Text pre-processing component: This component applies text standardization and transformation techniques (e.g., tokenization, lower-casing, stop-word removal). For CTI reports, a regex-based tokenizer may be implemented sothat lOCs (e.g., URLs, file names, email addresses, etc.) are correctly tokenized. For indexing (e.g., of the knowledge base, see TTP indexer), further preprocessing techniques (e.g., stemming, lemmatization) are conventionally used.

[0099] • A text splitter: This component splits the textual document into text segments of a granularity of choice (e.g., sentence or paragraph). For accurate splitting, it may be necessary to use semantic parsing techniques (i.e., leveraging HTML tags, article layout or heuristics).

[0100] • (TTP) indexer: Depending on the requirements of the TTP mining component, the index may be constructed in a list-based (i.e., of TTP textual names for text classification) or a tree-based (e.g., B-tree) or inverted-index structure that supports full-text search of TTPs (based on TTP textual descriptions). The inverted-index, which is more commonly used for full-text search, can be understood as the postings lists. Whereas, a postings is essentially a list of documents (document identifiers, or IDs) in which an associated term occurs at least once.

[0101] • A TTP mining component: This component may comprise a device 100 for training the matching model 101. The component may comprise a device for recognizing TTPs based on the matching model 101 according to an embodiment of this disclosure. The component takes as input a text segment 105a, mines its semantics, and tries to associate it to at least one of the TTPs in the KB. For mining the text semantics, additional data extraction techniques may be applied: o Named Entity Recognition: identify entities / IOCS in the text o Relation Extraction: identify relations among entities and IOCs o Co-reference resolution: find mentions of an entity in the text

[0102] FIG. 7 shows a flowchart for exemplary steps of deploying a system according to an embodiment of this disclosure. For example, the system 500 shown in FIG. 6 may be deployed according to said steps. The candidate generation path is optional (denoted in dashed boxes), where all TTPs may be returned instead. For training a matching model 101, embodiments of this disclosure may exploit labeled datasets, which are generally limited and expensive (supervision), for learning an end-to-end matching function. The matching model 101 may include embed, encode, project and compare components. Learning a high-fidelity representation of text and TTPs may be achieved. This high fidelity is encouraged by the gradient guidance of orthogonal supervisory signals to the encoding layer(s).

[0103] FIG. 8 shows an exemplary system pipeline for training and inference of the matching model 101 according to an embodiment of this disclosure. The exemplary system pipeline comprises a training part and an inference part.

[0104] Performing inference of the matching model 101 and / or TTP mining may be performed with the device 100 or with another device. Training the matching model may be performed with the device 100.

[0105] An exemplary training procedure of the system may be summarized as follows:

[0106] 1. A batch of one positive pair 103a (labeled as 1) and k-1 negative ones (labeled as 0 and generated according to a sampling policy) is generated, wherein each pair comprises a text segment 105a (Text j and a TTP candidate 106a, 106b (TTPj).

[0107] 2. The batch is passed through a dual-encoder matching network 101 to learn the representations of the input pair. The matching network 101 may be based on the exemplary functions f(), h(), g() and mteract().

[0108] 3. The matching network 101 may be optimized using an NCE loss, e.g., InfoNCE.

[0109] 4. The above steps are iterated multiple times until the convergence of the matching model 101.

[0110] For inference, several efficient techniques may be employed:

[0111] • A candidate selection technique, for example based on a function find candidates() , to avoid pair-wise comparisons of the entire set of TTP candidates or all TTP candidates.

[0112] • An embedding index for learned representations of the TTP candidates, for example the set of TTP candidates, for fast and efficient lookup, which are for example learned and stored during the training procedure, for example based on a function store(). An exemplary procedure at inference phase may comprise:

[0113] 1. A text segment, e.g. Texp, from a CTI report is fed into the system as input.

[0114] 2. The text is transformed into a query and TTP candidates are selected from the top-k matched results of a retrieval model.

[0115] 3. TTP candidates are encoded into a vector representation. The encoded vectors can also be retrieved from a cached embedding lookup.

[0116] 4. The input text is encoded into a vector representation.

[0117] 5. The pair (input text vector, TTP candidate vector) is fed into an alignment model to get a normalized alignment score, wherein the alignment model may be based on an inter act () function.

[0118] 6. A subset of TTP candidates are selected as matched TTPs based on the (ideally) longtailed alignment distribution (over all TTP candidates).

[0119] Except for the core learning components, a query expansion-based retrieval model may be leveraged for two tasks:

[0120] 1. Negative Sampling - at the learning phase. An inverted index to represent all TTPs in the MITRE KB may be employed. The inverted index enumerates for every unique term its postings list. A posting list is defined as a list of all TTPs for which occurs in their textual content (e.g., the TTP description).

[0121] 2. Candidate Selection - at the inference phase. A strategy may be employed to efficiently obtain the top-k results from the inverted index without exhaustively scoring all possible candidates at inference time.

[0122] Similarly, an inverted index to represent all TTPs in the MITRE KB may be used. To answer a query Q, the retrieval model consults the index to identify the best k matches as scored by a ranking model that assigns each TTP d a score S(d, Q) estimating its relevance to Q. The score

[0123] S(d, Q) may be calculated as where wtis the query-dependent weight (e.g., TF-IDF) of t. The top-k matches may be subsequently re-ranked by the matching model 101.

[0124] The matching model 101 may not only output a score, for example a probabilistic score, indicating a match between the textual input and a TTP candidate 106a, but may also generate a long-tailed distribution of matching scores 107 over the TTP collection. The long-tailed structure may be necessary so that at prediction time, there exists a cut-off threshold (e.g., based on 99th percentile) which clearly distinguishes between the significant (matched) and insignificant parts (not matched). Thus, a general objective, for example a general learning objective, may be to find a surrogate objective function for a joint (efficient) optimization on (1) matching textual pairs (similar to the conventional text matching) and (2) searching for the right text labels from the entire label space (similar to classification). By leveraging these orthogonal supervisory signals, the former may provide a calibrated matching score 107 while the latter may enforce a natural competition between the positive and negative (majority) labels. Negative matches would be tapered off towards the end of the (heavy) long-tailed distribution of the prediction scores.

[0125] Based on these requirements, InfoNCE may be adopted as the loss function of the matching framework. Generally speaking, InfoNCE is derived from the categorical cross-entropy, and is approximated by the NCE-based sub-sampling of p(x,y).

[0126] Cross-Entropy Loss is a conventional effective loss when applied to (multi-label, multi-class) classification. The loss tries to minimize the distance between the label and prediction distributions. To measure the probability distance, a mutual-information-based metric i.e., KL- divergence is used. Thus, cross-entropy looks for a global (inter-class) discrimination, and at the same time minimizes the intra-class similarity that is needed in a typical classification setting.

[0127] Noise Contrastive Estimation (NCE) is a theoretically-sound efficient method for parameter estimation of p0(y|x), with x,y are the input and label distribution, respectively. NCE is often leveraged when |y| is large, making the calculation of pg(ylx) become computationally expensive. Notionally, in NCE, the approximation of p0(y|x) is reduced to p0(l|x,yi1 :k;) where k « |y| and y{i:fc) is sampled from a noise distribution q(y). y{i:fc} is often a mixture of positive samples (labeled as 1) and negative samples (noise, labeled as 0) and the general task is to discriminate the two sets of samples. Formally, the loss of NCE is defined as: [logp(l|x, y) + kEy'qlogp(0|x,y')]

[0128] The objective may be a binary (or local) classification. InfoNCE may be considered a global version of NCE that leverages mutual information to measure the distance between the prediction and (k-sized) label distributions. InfoNCE constructs a variational lower bound to the intractable mutual information. The loss of InfoNCE may be calculated as follows:

[0129] While InfoNCE requires a binary separation between positive and negative samples, in the context of many-to-one mapping this restriction might be sub-optimal, as acquiring a full set of labels for every input might not be trivial. For example, when a text-TTP sample is given, it is not clear whether this TTP is the only label of the textual input or there are more relevant (missing) labels. One could leverage a variant of InfoNCE that relax-ably admits the ranking of these samples. In this sense, a higher label score (e.g., 5 on a scale from 0-5) may be given to one TTP candidate 106a than to other TTP candidates 106b, indicating that it is more important to match to that TTP candidate 106a.

[0130] InfoNCE may be replaced with marginal losses such as: the triplet loss or contrastive loss (where a pair of negative and positive samples are taken simultaneously). The focus of these losses may be intuitively explained as pushing negative samples away from positive ones in the metric space. However, it is often cumbersome when working on the pair-wise level, as it neglects the globality as opposed to how it is accounted for with mutual information in the cross entropy-based losses.

[0131] Conventional negative sampling methods are applied in-batch. Thus, the effects of such strategies essentially rely on the size of the mini-batches. As the matching model 101 may be trained in a low-resource setting, this approach is hindered by the skewed label distribution (the frequency distribution of labels in the collection, indicating the labels are unevenly available to only a small number of ‘popular’ TTPs; not to mistaken with the desired long-tail distribution of prediction score). A corpus-level negative sampling strategy may be used. A larger part of the TTP corpus may be irrelevant to the positive paired sample 103a. In addition, noisy samples may inherently be cancelled out while the information related to learning signals may remain in the matching model 101. The strategy may comprise augmenting a dataset with a substantial number of free and / or additional and diverse negative samples. The sampling policies may include at least on of:

[0132] • Random sampling: A simple uniform distribution q(y) = 1 / | Y| and the population marginal distribution q(y) = Px,y(y)> or in practice, an empirical approximation of it.

[0133] • Retrieval-based sampling: The hard-negative samples from a probabilistic retrieval model, similar to a model used for the TTP candidate selection, may be derived. Here the query Q is reformulated from three sources: o TTP name in the KB of the label o TTP description of the label o or the target textual input

[0134] • Semantic structure-based sampling: The hard-negative samples may be sampled from sibling TTPs in the MITRE ATT&CK hierarchy. Sibling TTPs may be sub-techniques from the same technique or techniques in the same tactic.

[0135] • Mixed sampling: This is the combination of the above three sampling strategies.

[0136] A matching network 101 may be based on a dual-encoder architecture, where two independent networks may be used to learn the representations of a text segment and a TTP description. Note that, to satisfy the anti-symmetric property, weight-sharing layers as in Siamese Networks may not be used. The framework may be considered as late- interaction-based, as different from cross-encoder networks. The replacement of a bi-encoder to a cross-encoder may be considered. However, a cross-encoder may lead to high computational costs of learning (at both training and inference).

[0137] FIG. 9 shows an exemplary component-based architecture of the matching model 101 according to an embodiment of this disclosure. A matching model 101 may comprise at least one of:

[0138] An embedding layer: The embedding layer may provide word- and entity-level text representations, for example local representations, for TTP candidates and input text segments. Token-level embedding may be trained from scratch separately or together with a pretrained (either context-free or context-specific) large language model (e.g., BERT). A part of the embedding layer that may be preferably used is a custom tokenizer for security domains, for example, with word- and entity-level granularity or wherein the word- and entity-level granularity may be emphasized (i.e., instead of n-grams or character-level). The embedding granularity may allow the adoption of pretrained large language models. For instance, the embedding of an entity can be learned from a near domain with richer resource (e.g., StackExchange Information Security).

[0139] To facilitate learning the general embeddings of entities with different semantic roles, (e.g., IP address, threat actor, malware, domain name, and other relevant entity types), the emitted embedding vectors might be concatenated with entity type labels, prior to being used in upper layers. For example, the entity types might be 1-hot-encoded and appended to the embedding vector. The identification of entity types may come from a manual or automated labeler, e.g., another CTI mining system.

[0140] For additional data efficiency (in terms of reducing the vocab size), the tokenizer may be configured to preserve more information by handling some entity types as follows:

[0141] 1. Split a vulnerability identifier, often in the form of CVE-YYYY-NNNNN, into [‘CVE, ‘YYYY, ‘NNNNN’]

[0142] 2. Split a file path or file name with extension separately, e.g., bat.exe into [‘bat’, ‘.exe’]

[0143] 3. Split a file path by the path separator (e.g., backward slash)

[0144] 4. Split an URL into smaller semantics (e.g., domain name, path, port number)

[0145] 5. Avoid splitting a multi-word named entity (e.g., threat actor ‘Dangerous Potato’ remains [‘Dangerous Potato’]. However, this splitting error may be to some extent tolerable.

[0146] Similar strategies can be adopted in general for other relevant entity types.

[0147] An encoder layer: The encoder layer may be a conventional multi-layer convolutional network with, e.g., a same padding. Alternatively or additionally, other types of encoders (e.g., RNNs, tree-based, graph-based) may be used. The text segments and TTP descriptions of the TTP candidates 106a, 106b may be in long-form text, e.g., they may comprise one to several sentences (text), or one to several paragraphs (TTP description). Thus, the encoder may model the finer granularity (e.g., sentence-level), or simply ignore the hierarchical (sentence- vs paragraph-level) semantic structure. The encoder layer may determine, for each pair, two encoded representations, for example based on two local representations of the pair.

[0148] An alignment layer: The alignment layer, for example an interaction layer, may take, for each pair, the two encoded representations, for example the two vectors based on the pair, as input and may output the respective interaction-driven representations, for example aligned representations. Any cross-attention model may be adopted. Formally, given the input representation of the text-TTP pair as t = (a), . . . , ) and tip = (b , bt), the unnormalized attention weights may be de-composedly calculated as: where Wall9nis a trainable projection matrix. The normalized weights for each token atand bj , denoted as a^ll9nand b^ll9nmay be derived. This model may be considered a much lighter-weight version of a self-attention in Transformer models, where notionally, atcan be mapped to query and bj to key.

[0149] A fusion layer: The fusion layer may perform various comparisons of local and aligned representations and may fuse the interaction vectors together. Here, O denotes the Hadamard product, and ; denotes the concatenation operator. at= Wfusion(,[a ; a ; al]')

[0150] A residual block layer: A block may comprise the encoder, alignment and fusion layers. The input to the nthblock d^ (where n > 2) is defined as:

[0151] The residual block layer may allow an effective stacking of multiple blocks (e.g., avoid vanishing gradient) while at the same time, adding complementary features to the alignment process. This is an optional component. The component may be replaced, or instead of stacking the blocks on top of each other, the blocks may be trained in parallel.

[0152] A pooling layer: The general purpose of the pooling layer may be to consolidate and { i}lttP (output from the fusion layer) into fixed-length vector representations. Here, a vanilla pooling operation, such as (non-) weighted average or max-pooling, may applied.

[0153] A merge layer: The merge layer may be for deriving the similarity between two dense vectors in the metric space. For example, a simple inner product and / or another Multi-Layer Perception (MLP) may be applied. The scalar output of the network may then be plugged into the optimization framework, e.g. with the NCE-based cross entropy loss.

[0154] FIG. 10 shows a cross-encoder architecture of the matching model 101 according to an embodiment of this disclosure. A cross-encoder architecture of the matching model 101 may be an alternative to a dual-encoder architecture of the matching model 101. The main difference of the cross-encoder architecture from the dual-encoder architecture is an earlier interaction between the elements of the input pair. ‘Early’ indicates that instead of stacking the alignment network on top of the encoding layers, the input pair interact at the lower layers of the network. Specifically, the alignment (interaction) layer may be placed before or merged to the encoder layer.

[0155] Once extracted after inference with the matching model, TTPs may be used in one of 2 modes:

[0156] MODE 1 : Used by human security analysts to gain a high-level (tactical and strategic) understanding of a particular threat (i.e., malware, attack campaign or threat actor), for purposes of, for example, at least one of

[0157] 1. Incident response

[0158] 2. Security operations

[0159] 3. Vulnerability management

[0160] 4. Risk analysis

[0161] 5. F raud prevention

[0162] 6. Security strategy management

[0163] MODE 2: Used to populate a KB which associates threats with the TTPs they were found to use. This KB is then used for the same purposes as humans use it manually, but in an automated fashion. Classes of security products which use this may be for example:

[0164] 1. Automated threat detection and response products and services (software or hardware products which automatically detect threats or automatically respond to threat activity), for example: a. Managed Detection & Response (MDR) b. Security Information and Event Management (SIEM) c. Security Orchestration, Automation and Response (SOAR) d. Network Detection and Response (NDR) e. Endpoint Detection and Response (EDR) f. Extended Detection and Response (XDR) g. User and Entity Behavior Analytics (UEBA)

[0165] 2. Identity and Access Management (IAM) products and services

[0166] 3. Fraud Protection products and services

[0167] 4. Data Loss Prevention (DLP) products and services

[0168] 5. Risk and Vulnerability Assessment

[0169] In this disclosure, the following terms may respectively refer to:

[0170] Threat action: a synonym for Procedure from Tactics, Techniques and Procedures.

[0171] Threat actor: individual, organization or political entity behind a cybersecurity attack.

[0172] Threat report, alias CTI report: a textual report describing a cybersecurity threat or incident. Common topics are malware, threat actors and threat campaigns. A typical threat report describes the attacking entity, its modus operandi, and provides related IOCS. Threat reports are authored by cybersecurity professionals and often contain non-textual supplementary data such as images and tables.

[0173] TTP mining / mapping / extraction / identification: synonyms for the problem of accurately identifying all implicit appearances of TTPs in the text of a threat report. In this disclosure the term “TTP mining” is used preferably rather than the above listed alterative terms.

[0174] Text matching: can be loosely defined as the task of matching a target text to a source text, based on their (semantic) similarity. It is the fundamental sub-task for many NLP / IR applications, e.g., question answering, document retrieval and natural language inference. In the common neural paradigm, a matching model receives a textual input pair, encodes the two parts and extracts their fine-grained (token-level) interactions. The captured interaction (information sharing) features are then leveraged for the matching estimation.

[0175] Inductive bias: the ability of the model to be applied to unseen input, based on the learned assumptions during training. In this disclosure, the term suggests that an inductive matching model 101 is able to generalize to unseen TTPs (labels). As long as the new unseen label can be described to a large part using the existing vocabulary, the ML model is capable of deriving the right matching based on the patterns it learns in the learning process.

[0176] Noise contrastive estimator: a crucial technique that reduces the complexity of comparing over the large label space. By sampling out (thus, adding noise) the space, the statistical (label) discriminator only needs to work on a much smaller sub-space, and thus, increasing training efficiency while being less data demanding.

[0177] Few-shot learning: a part of the meta-learning family, where a model is trained on one-to- several related tasks and is capable of learning over a new task with very small amount of data. The training method according to embodiments of this disclosure may comprise learning a unified matching function of relatively, any pair of text. This unified framework makes it possible to pre-train the matching model 101 in a general (or nearby) domain with rich resources (i.e., labels). Then, the model, theoretically, can be adapted to a target domain with fewer shots (e.g., less data demanding).

[0178] (Full-text) index: a data structure that facilitates fast lookup of documents in a text collection or database. The full-text index allows the retrieval of ‘relevant’ documents (written in natural language form) with regards to a textual query.

[0179] Query expansion-based retrieval model: query expansion is a common technique for information retrieval (IR) that reformulates the original query to improve the recall of the retrieval model. Conventionally, keywords in the query are expanded with semantically related terms. A retrieval model with a query expansion component is called a query expansion-based retrieval model.

[0180] Label-informed text matching: Label-informed text matching based TTP mining may combine the advantages of both conventional text matching and classification methods.

[0181] The approach label-informed text matching, in the learning phase, may exploit the shared information between a pair of text (similar to text matching) and altogether, may attempt to discriminate the positive labels versus the rest in the label space (similar to classification). Empowered by an NCE variant, this label-informed text matching may be capable of learning effectively and efficiently under data constraints. Generally, the processor 108 may be configured to perform, conduct or initiate the various operations of the device 100 described herein. The processor 108 may comprise hardware and / or may be controlled by software. The hardware may comprise analog circuitry or digital circuitry, or both analog and digital circuitry. The digital circuitry may comprise components such as application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), digital signal processors 108 (DSPs), or multi-purpose processors 108. The device 100 may further comprise memory circuitry, which stores one or more instruction(s) that can be executed by the processor 108, in particular under control of the software. For instance, the memory circuitry may comprise a non-transitory storage medium storing executable software code which, when executed by the processor 108, causes the various operations of the device 100 to be performed. In one embodiment, the device 100 may comprises one or more processors 108 and a non-transitory memory connected to the one or more processors 108. The non-transitory memory may carry executable program code which, when executed by the one or more processors 108, causes the device 100 to perform, conduct or initiate the operations or methods described herein.

[0182] The disclosure has been described in conjunction with various embodiments as examples as well as implementations. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed matter, from the studies of the drawings, this disclosure and the independent claims. In the claims as well as in the description the word “comprising” does not exclude other elements or steps and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.

Claims

CLAIMS1. A method (200) for training a matching model (101) for recognizing Techniques, Tactics, and Procedures, TTPs, within text segments (105a), wherein the method (200) comprises: obtaining (201) one or more batches (102), wherein each batch (102a) comprises one positive pair (103a) for a text segment (105a) of the batch (102a), one or more negative pairs (103b) for the text segment (105a) of the batch (102a), and two or more labels (104) for said two or more pairs (103), wherein each label (104a, 104b) indicates whether a respective pair is a positive pair (103a) or a negative pair (103b), wherein each positive pair (103a) comprises a text segment (105a) and a TTP candidate (106a) that match with each other and each negative pair (103b) comprises a text segment (105a) and a TTP candidate (106b) that do not match with each other, determining (202), for each pair of each batch (102a), a matching score (107) based on the matching model (101) and the pair (103a, 103b), and adjusting (203) the matching model (101) based on the matching score (107) of each pair of each batch (102a) and the two or more labels (104).

2. The method (200) according to claim 1, wherein the method (200) further comprises encoding and comparing, for each pair of each batch (102a), the pair (103a, 103b) based on the matching model (101).

3. The method (200) according to claim 2, wherein the matching model (101) comprises an encoding layer, wherein the method (200) comprises: encoding, for each pair of each batch (102a), the text segment (105a) and the TTP candidate (106a, 106b) of the pair (103a, 103b) into a joint vector space based on the encoding layer to form two vectors, determining, for each pair of each batch (102a), the matching score (107) based on the matching model (101) and the two vectors, adjusting the matching model (101) and / or the encoding layer based on the matching score (107) of each pair of each batch (102a) and the two or more labels (104).

4. The method (200) according to claim 3, wherein, for each pair of each batch (102a), the text segment (105a) and the TTP candidate (106a, 106b) are encoded separately into a respective vector of the two vectors, and wherein the method (200) comprises: determining, for each pair of each batch (102a), the matching score (107) based on the matching model (101) and the two vectors by comparing the two vectors with each other, adjusting the matching model (101) and / or the encoding layer based on comparing the matching score (107) of each pair of each batch (102a) with the two or more labels (104).

5. The method (200) according to any of the preceding claims, wherein, for each pair of each batch (102a), the text segment (105a) is at least a part of a Cyber Threat Intelligence, CTI, report, and / or the TTP candidate (106a, 106b) is a predefined textual definition of an attack method (200) that is described in the at least part of the CTI report, if the pair is a positive pair (103a), or the TTP candidate is a predefined textual definition of an attack method (200) that is not described in the at least part of the CTI report, if the pair is a negative pair (103b).

6. The method (200) according to any of the preceding claims, wherein the method (200) further comprises training the matching model (101) based on a Noise Contrastive Estimation, NCE, loss function, an InfoNCE loss function, or a relaxed version of an InfoNCE loss function for enabling to rank, for each batch (102a), the TTP candidates (106a, 106b) of the two or more pairs (103).

7. The method (200) according to any of the preceding claims, wherein the one or more batches (102) are based on corpus-level negative sampling comprising at least one of: random sampling, retrieval-based sampling for deriving hard-negative samples from a probabilistic retrieval model, and semantic structure-based sampling for deriving hard-negative samples from sibling TTPs.

8. The method (200) according to any of the preceding claims, wherein the matching model (101) comprises at least one of:an embedding layer, an encoder layer, an alignment layer, a fusion layer, a residual block layer, a pooling layer, and a merging layer.

9. The method (200) according to claim 8, wherein the method (200) further comprises token-level embedding, with the embedding layer, of each pair of each batch (102a) for forming a respective local representation of the text segment (105a) and the TTP candidate (106a, 106b) of the pair (103a, 103b).

10. The method (200) according to claim 8 or 9, wherein the method (200) further comprises determining, with the encoder layer, for each pair of each batch (102a), the two vectors based on the pair (103a, 103b).

11. The method (200) according one of the claims 8 to 10, wherein the method (200) further comprises receiving, with the alignment layer, for each pair of each batch (102a), the two vectors from the encoding layer, determining, with the alignment layer, for each pair of each batch (102a), two interaction-driven representations of the two vectors.

12. The method (200) according to one of the claims 8 to 11, wherein the method (200) further comprises receiving, with the fusion layer, for each pair of each batch (102a), two intermediate vectors based on the two vectors from previous layers of the matching model (101), comparing and fusing together, with the fusion layer, for each pair of each batch (102a), the local representations of the pair (103a, 103b) and / or the two intermediate vectors to form two fused vectors.

13. The method (200) according to one of the claims 8 to 12, wherein the method (200) further comprises: receiving, with the pooling layer, for each pair of each batch (102a), two intermediate vectors based on the two vectors from previous layers of the matching model (101), consolidating, with the pooling layer, for each pair of each batch (102a), the two intermediate vectors into fixed-length vector representations to form two output vectors.

14. The method (200) according to one of the claims 8 to 13, wherein the method (200) further comprises creating, with the residual block layer, a plurality of blocks, wherein each block comprises the encoder layer, the alignment layer, and the fusion layer, and wherein the method (200) comprises determining the matching score (107) based on each block of the plurality of blocks.

15. The method (200) according to one of the claims 8 to 14, wherein the method (200) further comprises: receiving, with the merging layer, for each pair of each batch (102a), two intermediate vectors based on the two vectors from previous layers of the matching model (101), determining, with the merging layer, for each pair of each batch (102a), similarities between the two intermediate vectors to form the matching score (107).

16. The method (200) according to any of the preceding claims, wherein the matching model (101) is based on a dual-encoder architecture or a crossencoder architecture.

17. The method (200) according to claim 16, wherein the cross-encoder architecture comprises an alignment layer before or together with the encoding layer.

18. The method (200) according to any of the preceding claims, wherein determining, for each pair of each batch (102a), the matching score (107) is based on exploiting semantic similarities of the text segment (105a) and the TTP candidate (106a, 106b).

19. A device (100) fortraining a matching model (101) for recognizing Techniques, Tactics, and Procedures, TTPs, within text segments (105a), wherein the device comprises a processor (108) configured to: obtain one or more batches (102), wherein each batch (102a) comprises one positive pair (103 a) for a text segment (105a) of the batch (102a), one or more negative pairs (103b) for the text segment (105a) of the batch (102a), and two or more labels (104) for said two or more pairs (103), wherein each label (104a, 104b) indicates whether a respective pair is a positive pair (103a) or a negative pair (103b), wherein each positive pair (103a) comprises a text segment (105a) and a TTP candidate (106a) that match with each other and each negative pair (103b) comprises a text segment (105a) and a TTP candidate (106b) that do not match with each other, determine, for each pair of each batch (102a), a matching score (107) based on the matching model (101) and the pair (103a, 103b), and adjust the matching model (101) based on the matching score (107) of each pair of each batch (102a) and the two or more labels (104).

20. A computer program product comprising a program code for performing, when the program code is executed on a computer, the method (200) according to one of the claims 1 to 18.

Images