Processing cybersecurity telemetry data

Abstract

A computer-implemented method of rendering a graphical user interface (GUI) for querying a collection of structured cybersecurity telemetry data, the method comprising: accessing in an experience database at least one observation, the at least one observation having been generated automatically by at least one detector operating on a collection of structured cybersecurity telemetry data; based on the at least one observation, generating a knowledge graph comprising (i) entity nodes representing entities associated with the at least observation, and (ii) edges between the entity nodes, the edges representing relationships between the entities; generating a visualization of the knowledge graph in the GUI; receiving a first user input denoting selection of at least a first entity node of the knowledge graph, the first entity node representing a first entity; based on the first user input, extracting from the knowledge graph one or more pieces of context data and generating a structured query comprising the one or more pieces of context data; identifying, by running the structured query on the collection of structured cybersecurity telemetry data, a second entity having a relationship to the first entity; displaying in the GUI details of the second entity; receiving a second user input denoting acceptance of the second entity; and based on the second user input, generating within the knowledge graph: a second entity node representing the second entity and an additional edge between the first entity node and the second entity node, the additional edge representing the relationship to the first entity.

Description

Processing Cybersecurity Telemetry DataTechnical Field

[0001] The present disclosure pertains generally to the processing of telemetry data in a cybersecurity system, and in particular to a graphical user interface that enables structured investigations of telemetry data supported by automated threat detection.Background

[0002] Cyber defense refers to technologies that are designed to protect computer systems from the threat of cyberattacks. In an active attack, an attacker attempts to alter or gain control of system resources. In a passive attack, an attacker only attempts to extract information from a system (generally whilst trying to evade detection). Private computer networks, such as those used for communication within businesses, are a common target for cyberattacks. An attacker who is able to breach (i.e., gain illegitimate access to) a private network may for example be able to gain access to sensitive data secured within it, and cause significant disruption if they are able to take control of resources as a consequence of the breach. A cyberattack can take various forms. A "syntactic" attack makes use of malicious software, such as viruses, worms and Trojan horses. A piece of malicious software, when executed by a device within the network, may be able to spread throughout the network, resulting in a potentially severe security breach. Other forms of "semantic" attack include, for example, denial-of-service (DOS) attacks which attempt to disrupt network services by targeting large volumes of data at a network; attacks via the unauthorized use of credentials (e.g., brute force or dictionary attacks); or backdoor attacks in which an attacker attempts to bypass network security systems altogether. With increasing emphasis on “remote” access, though remote desktop, or virtual private network (VPN) connections and the like, further vulnerabilities and attack opportunities are created.Summary

[0003] Large volumes of telemetry data may be collected in a cybersecurity context. Whilst automated threat detection can be used to isolate relevant telemetry data, manual investigation remains an important aspect of cybersecurity. For example, manual investigation might give further context or insights into a threat or potential threat which has been detected automatically. Tools and mechanisms are provided herein that enable manualinvestigations to be conducted efficiently and systematically, in a manner that complements automated threat detection.

[0004] A first aspect herein provides a computer-implemented method of rendering a graphical user interface (GUI) for querying a collection of structured cybersecurity telemetry data, the method comprising: accessing in an experience database at least one observation, the at least one observation having been generated automatically by at least one detector operating on a collection of structured cybersecurity telemetry data; based on the at least one observation, generating a knowledge graph comprising (i) entity nodes representing entities associated with the at least observation, and (ii) edges between the entity nodes, the edges representing relationships between the entities; generating a visualization of the knowledge graph in the GUI; receiving a first user input denoting selection of at least a first entity node of the knowledge graph, the first entity node representing a first entity; based on the first user input, extracting from the knowledge graph one or more pieces of context data and generating a structured query comprising the one or more pieces of context data; identifying, by running the structured query on the collection of structured cybersecurity telemetry data, a second entity having a relationship to the first entity; displaying in the GUI details of the second entity; receiving a second user input denoting acceptance of the second entity; and based on the second user input, generating within the knowledge graph: a second entity node representing the second entity and an additional edge between the first entity node and the second entity node, the additional edge representing the relationship to the first entity.

[0005] In embodiments, the method may comprise determining that the first user input satisfies one or more applicability criteria associated with a predetermined query action, wherein the structured query may be generated by accessing a query template associated with the predetermined query action and populating the query template based on the one or more pieces of context data.

[0006] The method may comprise displaying, responsive to determining that the first user input satisfies the applicability criteria, an indication of the predetermined query action, wherein the structured query may be run on the telemetry database responsive to a third user input denoting selection of the predetermined query action.

[0007] The method may comprise persistently storing in an investigation history associated with the at least one observation: an indication of the third user input denoting selection ofthe predetermined query action, and an indication of the second user input denoting acceptance of the second entity.

[0008] The one or more pieces of context data may be used to determine that the first user input satisfies the applicability criteria.

[0009] The one or more pieces of context data may comprise: an attribute of the first entity node, an attribute of an edge connected to the first entity node, or an attribute of another entity node in the knowledge graph directly or indirectly connected to the first entity node by one or more edges.

[0010] The structured query may return a plurality of second entities each having a relationship to the first entity, and details of the plurality of second entities may be displayed, wherein the second user input may denote a selection of the second entity from the plurality of second entities.

[0011] The first entity may be a device, and the plurality of second entities may be a plurality of network connections associated with the first entity.

[0012] The knowledge graph may comprise a third entity node related to the first entity and an edge between the first entity node and the third entity node, wherein the second entity may be related to the third entity, and a second additional edge may be added to the knowledge graph between the second entity and the third entity.

[0013] The first and second entities may be processes, and the third entity may be a device on which the processes are or have been executed.

[0014] The knowledge graph may be generated based on a case stored in the experience database, the case having multiple observations associated therewith, wherein the knowledge graph may comprise an observation node for each observation, a plurality of other entity nodes, and an edge between the observation node and each other entity node related to that observation.

[0015] The method may comprise receiving a fourth user input denoting a selection of a plurality of nodes within the knowledge graph; displaying within the GUI a comment field for receiving user-input text; generating in the knowledge graph a comment node and a plurality of further edges between the comment node and the plurality of nodes, the commentnode associated with user-entered text in the comment field; and persistently storing, in association with the at least observation, an indication of the comment node, the plurality of further edges, and the user-entered text.

[0016] The method may comprise persistently storing, in association with the at least observation, an indication of the second entity and the relationship to the first entity.

[0017] The one or more pieces of context data may comprise a first entity identifier of the first entity represented by the first entity node.

[0018] The first user input may denote a selection of multiple first entity nodes, and the one or more pieces of context data may relate to the multiple first entity nodes.

[0019] A second aspect herein provides a computer-implemented method of rendering a graphical user interface (GUI) for querying a collection of structured cybersecurity telemetry data, the method comprising: accessing in an experience database at least one observation, the at least one observation having been generated automatically by at least one detector operating on a collection of structured cybersecurity telemetry data; based on the at least one observation, generating a knowledge graph comprising (i) entity nodes representing entities associated with the at least observation, and (ii) edges between the entity nodes, the edges representing relationships between the entities; generating a visualization of the knowledge graph in the GUI; receiving a first user input denoting selection of at least a first entity node of the knowledge graph, the first entity node representing a first entity; based on the first user input, extracting from the knowledge graph one or more pieces of context data; determining that a predetermined action is applicable to the first user input, by using the one or more pieces of context data to evaluate one or more predetermined applicability conditions associated with the predetermined action; displaying within the GUI an indication of the predetermined action applicable to the first user input; and responsive to a second user input denoting selection of the predetermined action, causing the predetermined action to be performed in relation to the first entity.

[0020] In embodiments, the one or more pieces of context data may comprise: an attribute of the first entity node, an attribute of an edge connected to the first entity node, or an attribute of another entity node in the knowledge graph directly or indirectly connected to the first entity node by one or more edges.

[0021] The predetermined action may, for example, be a query action to obtain additional information relating to the first entity, a cybersecurity intervention action performed on the first entity, or an action to reconfigure the at least one detector based on the first entity.

[0022] The first entity may be a device, and the action may be a cybersecurity intervention action, which comprises isolating the first device from one or more other devices.

[0023] The predetermined action may be an action to reconfigure the at least one detector, which comprises adding the first entity to an exclusion list associated with the at least one detector.

[0024] The predetermined action may be a query action, which is: performed on the collection of structured cybersecurity telemetry data to find one or more pieces of telemetry data related to the first entity, or performed on a database of observations or cases to find one or more further observations relating to the first entity to locate which have been generated automatically, a case being a group of related observations.

[0025] The method may comprise persistently storing in an investigation history associated with the at least one observation an indication of the second user input denoting selection of the predetermined action.

[0026] The method of the first or second aspect may comprise comprising using the investigation history to train a machine learning investigation model to perform automated threat investigation.

[0027] The entity nodes may comprise an endpoint node representing an endpoint entity and a network node representing a network entity, and the edges may comprise an edge between the endpoint node and the network node representing a relationship between the endpoint entity and the network entity.

[0028] Further aspects herein provide computer system comprising one or more computers configured to implement the method of any aspect or embodiment herein, and non-transitory or transitory media embodying computer-readable instruction configured so as, upon execution by one or more computer processors, to cause the one or more computer processors to implement the same.Brief Description of Figures

[0029] For a better understanding of the present subject matter, certain embodiments will now be described by way of example only with reference to the following figures, in which:

[0030] FIG. 1 shows, by way of context, a schematic function block diagram of a cyber defense platform;

[0031] FIG. 2 shows a highly schematic representation of a network event;

[0032] FIG. 3 shows a schematic block diagram of a network which may be subject to a cyber-security analysis;

[0033] FIG. 4 shows a highly schematic representation of an endpoint event;

[0034] FIG. 5 shows a block diagram of an endpoint device running an endpoint agent;

[0035] FIG. 6 shows a high-level functional block diagram of an advanced endpoint agent;

[0036] FIGS. 6 to 16 shows various views rendered within an investigation graphical user interface.Detailed Description

[0037] A range of data may be used in a cybersecurity context as a basis for detecting cybersecurity threats (or potential threats) to any form of computerized infrastructure (such as a computer network, device, system, program or set of programs, database(s) etc.). Collected data indicative of such (potential) threats (alone or in combination with other data) may be referred to herein as “telemetry”. An example cybersecurity platform is described herein that collects multiple modalities (forms) of telemetry from multiple sources and uses those various telemetry modalities as a basis for threat detection.

[0038] To facilitate reliable and efficient processing of telemetry collected from different sources, the telemetry is converted into a structured form prior to analysis (referred to as ‘standardization’ herein). This, in turn, enables structured queries to be run on the structured telemetry by detectors executed within the platform and also by human analysts. In the examples described below, significant detections give rise to “observations”, which may in turn be grouped into “cases” with other potentially related observation(s). A human analyst can access an observation or case, and perform further investigations, which may involve further manual querying of telemetry that has been collected in a structured format.Formulating such queries is time-consuming, and requires expertise and knowledge of the data schemas used to structure the telemetry.

[0039] An investigation graphical user interface (GUI) is described below, which enables an analyst to interact with observations and cases more effectively. Interactive GUI features enable an analyst to conduct manual investigations, and augment observations / cases in a more streamlined and systematic manner. By improving the speed and quality of manual investigations, analysts can respond to cyberthreats more quickly and reliably, ultimately leading to improved cybersecurity.

[0040] Elements associated with an observation or case are represented in a knowledge graph. The knowledge graph supports both data structure representation, and corresponding visual representation, of individual elements associated with a case or observation as nodes with associated node attributes. Relationships between nodes are represented by edges with associated edge attributes. Classes of nodes and edges are represented as "types" assigned to the nodes and edges. The Investigation GUI supports incremental addition of new information into the knowledge graph. The knowledge graph is developed systematically, where each investigation step that adds information to the knowledge graph has a direct link to existing information in the knowledge graph.

[0041] The investigation GUI encourages more structured investigation, which in turn allows more structured recording of investigation histories. An evidence chain is stored, recording the relationship between new information added to the knowledge graph and the existing information in the knowledge graph at each investigation step. This enables better tracking of investigations over time, and improved auditing of investigations. Moreover, structured information with context contained within the knowledge graph supports both human and programmatic analysis. This may, for example, open up greater possibilities for further automation by learning from previous manual investigations (e.g., using machine learning techniques).

[0042] To build upon and / or utilise the knowledge graph, a curated set of predetermined ‘actions’ is provided, which codify knowledge and expertise of a specialist user or team. These include predetermined query actions (or ‘questions’), which simplify the GUI and save user time by abstracting away complexities involved in querying for information from telemetry databases, such as writing SQL, copying and pasting information and keeping trackof query results. The predetermined actions include intelligent, context based filtering rules to ensure that only eligible and relevant actions are suggested and attemptable (e.g., a ‘find parent process’ query action is only askable of a node of "process" type for which a parent process node doesn't already exist within the graph). Further relevancy filtering and actionsuggestion ranking is performed based on a history of previously taken actions by the user and / or a team.

[0043] A question builder interface is provided, which allows new questions to be added. The question builder interface enables a user to define a new question.

[0044] The investigation GUI is described in further detail below. First, additional details of the cybersecurity platform are described to provide relevant context.

[0045] A telemetry “data model” (1701, FIG. 1) is embodied as a set of predefined data schemas (telemetry schemas) that define the precise structure of different telemetry “event” or “record” types. The terms “event” and “record” are used interchangeably herein (unless otherwise indicated) to refer to a data structure having one or more data fields conforming to a predefined telemetry schema. Events may include all of the underlying ‘raw’ telemetry from which they are generated, but structured according to the data model 1701 (no information loss), or a more concise metadata summary of relevant aspects of the underlying raw telemetry (Tossy’ conversion whereby information is selectively discarded, e.g., because it is of low analytical value). Improved reliability is achieved because downstream systems know the exact format of data they will receive. Efficiency gains can be achieved by reducing the ‘verbosity’ of individual telemetry messages (e.g., by not including field names in each event, but rather just identifiers which relate to the fields defined by a schema; this applies to both lossless and lossy telemetry restructuring).

[0046] The telemetry data model 1701 may, for example, be structured as a set of “packages”, where a package refers to a category of telemetry. For example, ‘network’ and ‘endpoint’ may be packages within the data model 1701. Each package comprises a set of data schemas (‘topics’) for structuring the corresponding category of telemetry (e.g., network data schemas, endpoint data schemas etc.). In other words, a topic defines a specific telemetry record type (e.g., ‘endpoint.process’, ‘network.dns’ etc.) and collections of related record types (e.g., ‘endpoint’, ‘network’ etc.) are referred to as packages.

[0047] Network telemetry refers to network data obtained through some form of network traffic monitoring within a network. Network telemetry can include ‘raw’ network data and / or information about the raw network data (metadata). To provide network traffic monitoring, specialized monitoring component(s), such as TAPs or mirrors, may be deployed at suitable location(s) within the network. Network traffic passes through such components transparently, and each component provides a copy or ‘mirror’ of the network traffic passing through it. Alternatively or in addition, network traffic monitoring may be implemented locally on endpoint devices within the network. For example, a cybersecurity platform is described below, in which an advanced form of endpoint network sensor (EPNS) is deployed to endpoint devices of the network to provide local monitoring and reporting of network traffic flowing to and / or from the endpoint device. Such endpoint network sensors may reduce reliance on other types of monitoring component (such as mirrors / TAPs) and / or complement functionality of other type(s) of monitoring component (e.g., in a deployment with “roaming” endpoints). In the described examples, an EPNS provides network telemetry in the form of structured metadata extracted locally at the endpoint (that is, the EPNS provides telemetry to the platform that it has already structured according to the data model 1701). Such sensors may, for example, be deployed in combination with centralized cybersecurity infrastructure to provide sophisticated threat detection, analytics, response and / or remediation functions. Raw network data may be collected from components (subsystems, devices, software components etc.) across a monitored network, and re-structured into network events based on the data model 1701.

[0048] Endpoint telemetry refers to endpoint data (specific to some endpoint device within the network) that it would generally not be possible to obtain solely through network traffic monitoring. Endpoint telemetry is provided by some form of sensor executed on an endpoint device, which would typically interface with the endpoint’s operating system (OS) to obtain endpoint-specific details. Endpoint monitoring software may be installed on endpoints of the network being monitored. The software monitors local activity at the endpoint on which it is installed, and feeds the resulting data (endpoint data) into the platform for analysis. The endpoint data may also be structured according to the data model 1701 locally at the endpoint, prior to submission to the platform. In the examples described below, the EPNS performs both local network traffic monitoring and additionally collects endpoint telemetry that is related to its collected network telemetry. Such endpoint data could, for example,include details of one or more processes running on an endpoint device and receiving and / or instigating network traffic; files (e.g., downloaded, uploaded or otherwise transferred to an endpoint device); users; other endpoint-type entities; or any combination thereof. Association of network and endpoint data may, for example, be performed locally at an endpoint by an EPNS (such that network and endpoint data are both structured and linked ‘at source’ by the EPNS before submitting the telemetry for analysis). When network traffic monitoring and restructuring is performed at the endpoint device itself, it may, in some cases, be more straightforward and reliable to perform this matching locally at the device. Alternatively or additionally, such linking of different telemetry types may be performed in a backend system, in the manner described below.

[0049] Other telemetry modalities are also considered. For example, “third-party” detection results may be collected as a form of telemetry (third-party telemetry). In this case, the results of third-party analysis (analysis performed outside of the platform) are consumed as inputs to the platform, and re-structured as needed based on the data model 1701 prior to downstream processing / further analysis within the platform itself. Such downstream processing may, for example, include further analysis of third-party telemetry in combination with other form(s) of telemetry to provide a more comprehensive form of threat detection. Third-party telemetry could, for example, include third-party network, endpoint, “Intemet-of-things” (loT) or cloud telemetry etc.

[0050] In general, telemetry can be generated by, collected from or otherwise related to any type of computing (hardware and / or software) infrastructure, including systems, networks, devices (physical and / or virtual), computer programs, databases etc. Such infrastructure can include infrastructure that is local to or managed by an organization and / or cloud-based infrastructure. With multiple, diverse telemetry sources, the benefits of telemetry standardization become even more apparent.

[0051] It can be useful to consider ‘cloud’ telemetry (collected from cloud-based infrastructure) as a category of telemetry in its own right, structured according to a cloud package within the data model 1701, as cloud-based infrastructure can present specific challenges in the context of cybersecurity.

[0052] A cybersecurity analysis is performed by a set of detection modules (or ‘detectors’). Each detection module is a functional component (typically software) that can performqueries on a set of structured telemetry events and looks for particular threat patterns or characteristics within those events. For example, as described in more detail below, a given detection module might be programmed to search the events for indications of a tactic or technique known to be used by cyber attackers. As another example, a detection model might perform anomaly detection using unsupervised machine learning techniques.

[0053] Each detection module is typically programmed to perform a reasonably discrete and self-contained detection task (e.g., looking for a particular tactic / technique of a cyberattack, or for anomalies in a particular subset of the event with respect to a particular set of features etc.) that is chosen in the context of a wider analysis. In addition or as an alternative to ‘hard- coded’ detectors, other forms of detector may be deployed. For example, signature-based detectors may implement configurable signature-based detection, which is a form of pattern recognition to detect patterns (‘signatures’) indicative of known threats.

[0054] In the described platform, structured telemetry records are stored in a telemetry database and used as inputs to detections. Different levels of significance may be attached to a detection (e.g., based on severity and / or confidence of the detection). Significant detections are raised as “observations”. Observations that are determined to be related to one another may, in turn, be clustered into “cases”. Observations and cases are stored in a further database (the case database or observation database, where those terms are used interchangeably) separate from the telemetry database.

[0055] Certain detections may not be significant enough to warrant the creation of an observation. In that case, the output of a detection may be a further telemetry record, which could in turn be an input to some other detection (e.g., if it is more appropriate for another component of the platform to perform further work before raising an observation). A detection could also generate multiple outputs, e.g., a further telemetry record and an observation.

[0056] Certain forms of activity can be detected with high confidence using signatures / rules. In that case, signature / rule matching represents a high confidence detection on its own, without necessarily requiring further analysis. In the platform described below, this might result in the creation of an observation (which might, in turn, be included in a case if it can be related to other observation(s)).

[0057] FIG. 1 shows an example of an integrated cybersecurity platform. The integrated cyber defense platform protects a network against cyberattacks through a combination of comprehensive telemetry collection and organization, and advanced analytics applied to the resulting output within a reasoning framework.

[0058] The cybersecurity platform may serve multiple organizations, each with their own network / infrastructure deployment.

[0059] The described platform operates according to a “triangulation” model in which multiple forms of analysis may be used as a basis for threat detection. To provide effective triangulation, techniques such as anomaly detection, such as rules-based detections and / or detections based on supervised and / or unsupervised machine learning or other statistical methods more generally may be applied in any combination. By way of example, a particular form of threat detection is formulated around the "Mitre ATT&CK framework" (or any other structured source of attack knowledge) is described below. Whilst Mitre is considered, the description applies more generally to other forms of tactics / techniques, including tactic / techniques defined in alternative (e.g., bespoke) threat models, or learned though statistical analysis (such as supervised or unsupervised machine learning).

[0060] The cyber defense platform is a system that operates to collect and analyze various types of telemetry, and which is implemented as a set of computer programs that perform the data processing stages disclosed herein. The computer programs are executed on one or more computer processors, such as central processing units (CPUs), graphical processing units (GPUs) etc.

[0061] In a data optimization stage, telemetry is captured in the form of structured, timestamped events. Both network events and endpoint events (and / or other telemetry events, such as third-party, cloud etc.) are collected at this stage and enhanced for subsequent analysis. Events generated across different data collectors are standardized, as needed, according to the predefined telemetry data model 1701.

[0062] The system is shown to comprise a plurality of data collectors 102 which are also referred to herein as “coal-face producers”. The role of these components 102 is to collect telemetry and, where necessary, process that data into a form suitable for downstream cyber security analysis. This may include the collection of raw network data from components of the network being monitored and conversion of that raw data into structured events (networkevents), as described above. The raw network data is collected based on network tapping, for example (which may be used in cases where endpoint network monitoring and reporting is not viable or is undesired).

[0063] Event standardization components 104 are also shown, each of which receives raw telemetry data outputted from a respective one of the coal-face producers 102. The standardization components 104 standardize these structured events according to the predefined telemetry data model 1701, to create standardized telemetry events.

[0064] The raw network data that is collected by the coal-face producers 102 is collected from a variety of different network components 100. The raw network data can for example include captured data packets as transmitted and received between components of the network, as well as externally incoming and outgoing packets arriving at and leaving the network respectively.

[0065] Additionally, structured endpoint events are collected using endpoint agents 316 executed on endpoints throughout the network. The endpoint agents provide structured endpoint events to the coal-face producers 102 and those events are subject to standardization, enrichment and correlation as above. Whilst it is generally preferred to implement network traffic monitoring and reporting locally at endpoint where possible, and to link the resulting network data with endpoint data locally at the endpoint, there are circumstances where this may not be possible or desired, in which case endpoint data may be received that is not (yet) linked to network data.

[0066] Once standardized, telemetry events (messages) are stored in a message queue 106 (event queue). For a large-scale system, the message queue can for example be a distributed message queue. That is, a message queue 106 embodied as a distributed data storage system comprising a cluster of data storage nodes (not shown in FIG. 1).

[0067] As part of the data optimization, first stage enrichment and joining is performed. This can, to some extent at least, be performed in real-time or near-real time (processing time of around 1 second or less). That is, network and endpoint events are also enriched with additional relevant data where appropriate (enrichment data) and selectively joined (or otherwise linked together) based on short-term temporal correlations. Augmentation and joining are examples of what is referred to herein as event enhancement.

[0068] An event optimization system 108 is shown having an input for receiving telemetry events from the message queue 106, which it processes in real-time or near real-time to provide enhanced events in the manner described below. In FIG. 1, enhanced events are denoted w.esec.t, as distinct from non-enhance events denoted w.raw.t. Non-enhanced events that are stored in the message queue 106 are shown down the left-hand side of the message queue (these are the standardized, structured events provided by the standardization components 104 and / or by endpoint agents) whereas enhanced events are shown on the righthand side. However, it will be appreciated that this is purely schematic and that the events can be stored and managed within the message queue 106 in any suitable manner.

[0069] The event enhancement system 108 is shown to comprise an enrichment component 110 and a joining component 112. The enrichment component 106 operates to augment events from the message queue 106 with enrichment data, in a first stage enrichment. The enrichment data is data that is relevant to the event and has potential significance in a cybersecurity context. It could for example flag a file name or IP address contained in the event that is known to be malicious from a security dataset. The enrichment data can be obtained from a variety of enrichment data sources including earlier events and external information. The enrichment data used to enrich an event is stored within the event, which in turn is subsequently returned to the message queue 106 as described below. In this first-stage enrichment, the enrichment data that is obtained is limited to data that it is practical to obtain in (near) real-time. Additional batch enrichment is performed later, without this limitation, as described below.

[0070] The joining component 112 operates to identify short-term, i.e., small time window, correlations between events. This makes use of the timestamps in the events and also other data such as information about entities (devices, processes, users etc.) to which the events relate. The joining component 112 joins together events that it identifies as correlated with each other (i.e., interrelated) on the timescale considered and the resulting joined user events are returned to the message queue 106. This can include joining together one or more network events with one or more endpoint events where appropriate (applicable to network and endpoint data that has not been linked prior to reporting).

[0071] In Figure 1, the joining component 112 is shown having an output to receive enriched events from the enrichment component 110 such that it operates to join events, asappropriate, after enrichment. This means that the joining component 112 is able to use any relevant enrichment data in the enriched events for the purposes of identifying short-term correlations. However, it will be appreciated that in some contexts at least it may be possible to perform enrichment and correlation in any order or in parallel.

[0072] A telemetry database manager 114 is shown having an input connected to receive events from the message queue 106. The telemetry database manager 114 retrieves telemetry events, and in particular enhanced (i.e., enriched and, where appropriate, joined) events from the message queue 106 and stores them in a telemetry database 116. The telemetry database 116 may be a distributed database. The telemetry database 116 stores events on a longer time scale than events are stored in the message queue 106.

[0073] A batch enrichment engine 132 performs additional (second stage) enrichment of the events in the telemetry database 116 over relatively long time windows and using large enrichment data sets. A batch enrichment framework 134 performs a batch enrichment process, in which events in the telemetry database 116 are further enriched. The timing of the batch enrichment process is driven by an enrichment scheduler 136 which determines a schedule for the batch enrichment process. Note that this batch enrichment is a second stage enrichment, separate from the first stage enrichment that is performed before events are stored in the telemetry database 116.Network and Endpoint Events:

[0074] FIG. 3 shows a schematic block diagram of an example network 300 which is subject to monitoring, such as a private or enterprise network. The network 300 is shown to comprise network infrastructure, which can be formed of various network infrastructure components such as routers, switches, hubs etc. In this example, a router 304 is shown via which a connection to a public network 306 is provided such as the Internet, e.g., via a modem (not shown). This provides an entry and exit point into and out of the private network 300, via which network traffic can flow into the private network 300 from the public network 306 and vice versa. Two additional network infrastructure components 308, 310 are shown in this example, which are internal in that they only have connections to the public network 306 via the router 304. However, as will be appreciated, this is purely an example, and, in general, network infrastructure can be formed of any number of components having any suitable topology.

[0075] In addition, a plurality of endpoint devices 312a-312f are shown, which are endpoints of the private network 300. Five of these endpoints 312a-312e are local endpoints shown directly connected to the network infrastructure 302, whereas endpoint 312f is a remote endpoint that connects remotely to the network infrastructure 302 via the public network 306, using a VPN (virtual private network) connection or the like. It is noted in this respect that the term endpoint in relation to a private network includes both local endpoints and remote endpoints that are permitted access to the private network substantially as if they were a local endpoint. The endpoints 312a-312f are user devices operated by users (client endpoints), but in addition one or more server endpoints can also be provided. By way of example, a server 312g is shown connected to the network infrastructure 302, which can provide any desired service or services within private network 300. Although only one server is shown, any number of server endpoints can be provided in any desired configuration.

[0076] For the purposes of collecting raw network data, a plurality of network data capture components 314a-314c are provided. These can for example be network taps. A TAP is a component which provides access to traffic flowing through the network 300 transparently, i.e., without disrupting the flow of network traffic. TAPs are non-obtrusive and generally non-detectable. A TAP can be provided in the form of a dedicated hardware TAP, for example, which is coupled to one or more network infrastructure components to provide access to the raw network data flowing through it. In this example, the taps 314a, 314b and 314c are shown coupled to the network infrastructure component 304, 308 and 310 respectively, such that they are able to provide, in combination, copies 317 of any of the raw network data flowing through the network infrastructure 302 for the purposes of monitoring. It is this raw network data that is processed into structured network events for the purpose of analysis.

[0077] FIG. 2 shows a schematic illustration of an example network event 200.

[0078] The network event 200 is shown to comprise a timestamp 204, an entity ID 206 and network event description data (network event details) 208. The timestamp 204 and entity ID 206 constitute metadata 207 for the network event details 208.

[0079] The network event description data 208 provides a network event description. That is, details of the activity recorded by the network event that has occurred within the network being monitored. This activity could for example be the movement of a network packet orsequence of network packets through infrastructure of the network, at a particular location or at multiple locations within the network.

[0080] The network event data 208 can for example comprise one or more network event type indicators identifying the type of activity that has occurred. The entity ID 206 is an identifier of an entity involved in the activity, such as a device, user, process etc. Where multiple entities are involved, the network event can comprise multiple network event IDs. Two important forms of entity ID are device ID (e.g., MAC address) and network address (e.g., IP address, transport address (IP address plus port) etc.), both of which may be included in a network event.

[0081] As well as being used as part of the analysis (in conjunction with the timestamps 204), entity IDs 206 and network event description data 208 can be used as a basis for querying enrichment data sources for enrichment data.

[0082] The timestamp 204 denotes a timing of the activity by the network event 200. Such timestamps are used as a basis for associating different but related network events, together with other information in the network event 200 such as the entity ID 206 or IDs it contains.

[0083] The network event 200 can have structured fields in which this information is contained, such as a timestamp field, one or more entity ID fields and one or more network event description fields. The field-based format of the network event 200 is defined by the telemetry data model 1701.

[0084] The network event 200 is shown to comprise a network event identifier (ID) 202 which uniquely identifies the network event 200.

[0085] Returning to FIG. 3, for the purpose of collecting endpoint data, endpoint monitoring software (code) is provided which is executed on the endpoints of the network 300 to monitor local activity at those endpoints. This is shown in the form of endpoint agents 316a-316g (corresponding to endpoint agents 316 in FIG.1) that are executed on the endpoints 312a- 312g respectively. This is representative of the fact that endpoint monitoring software can be executed on any type of endpoint, including local, remote and / or server endpoints as appropriate. This monitoring by the endpoint agents is the underlying mechanism by which endpoint events are collected within the network 300. Enhanced endpoint agents thatadditionally implement local network traffic monitoring and reporting (generating pre-joined endpoint and network events ‘at source’) are described later.

[0086] FIG. 4 shows a schematic illustration of a certain high-level structure of an endpoint event 400.

[0087] The endpoint event 400 is shown to comprise at least one endpoint identifier, such as a device identifier (e.g., MAC address) 402 and network (e.g., IP) address 404 of the endpoint to which it relates, and endpoint event description data 406 that provides details of the local activity at the endpoint in question that triggered the creation of the endpoint event 400. Those data are stored in fields, as defined in the telemetry data model 1701.

[0088] One example of endpoint activity that may be valuable from a cyber defense perspective is the opening of a connection at an endpoint. For example, a TCP / IP connection is uniquely defined by a five-tuple of parameters: source IP address (IP address of the endpoint being monitored), source port, destination IP address (IP address of an e.g., external endpoint to which the connection is being opened), destination port, and protocol. A useful endpoint event may be generated and provided to the platform for analysis when an endpoint opens a connection, in which the five-tuple defining the connection is recorded, and well as, for example, an indication of a process (application, task, etc.) executed on the endpoint that opened the connection.

[0089] As noted, one of the key features of the present cyber defense platform is its ability to link together interrelated network and endpoint events. Following the above example, by linking an endpoint event recording the opening of a connection and details of the process that opened it to network events recording the flow of traffic along that connection, it becomes possible to link specific flows of network traffic to that specific process on that endpoint.

[0090] Additional examples of endpoint information that can be captured in endpoint events include information about processes running on the endpoint (a process is, broadly, a running program), the content of files on the endpoint, user accounts on the endpoint and applications installed on the endpoint. Again, such information can be linked with any corresponding activity in the network itself, to provide a rich source of information for analysis.

[0091] Such linking can occur within the platform both as part of the real-time joining performed by the joining component 112.

[0092] However, network and endpoint events can also be linked together as part of the analysis performed by the analysis engine that is inherently able to consider links between events over longer timescales, as will now be described.Detections:

[0093] Returning to FIG.l, in an analytics / detection stage, the collected telemetry events are subject to sophisticated real-time analytics / detections, by an analysis engine 118 (detection engine). This may include the use of statistical analysis techniques commonly known as “machine learning” (ML) and / or as rules-based detection.

[0094] The analysis engine 118 is shown having inputs connected to the event queue 106 and the telemetry database 116 for receiving events for analysis. The events received at the analysis engine 118 from the event queue 106 directly are used, in conjunction with the events stored in the telemetry database 116, as a basis for detections within the analysis engine 118. Queued events as received from the message queue 106 permit real-time analysis, whilst the telemetry database 116 provides a record of historical events to allow threats to be assessed over longer time scales as they develop.

[0095] As noted, significant detections give rise to “observations”, which may, in turn, be compiled (clustered / combined) into “cases”. Detections may, for example, be based on recognized tactics, techniques and / or other threat / attack patterns or anomalies (such as unsupervised anomaly detection). A pipeline is provided to selectively and intelligently alert an analyst to observations or cases that are deemed to be of sufficient significance.

[0096] Observations and cases are stored in a separate ‘experience’ database 124 (which may also be a distributed database). Observations and cases are stored in the same database 124 as each other in the following examples (but separate from the telemetry database 116), and in that context, the terms ‘case database’ and ‘observation database’ are used interchangeably with ‘experience database’. In other implementations cases and observations may be stored in separate case and observation databases.

[0097] Observations may be generated based on events that are received at the analysis engine from the message queue 106, in real-time or near-real time.

[0098] Observations may also be generated based on events that are stored in the telemetry database 116. For example, it may be that an event is only identified as potentially threat- related (triggering a detection) when that event has been enriched in the second stage enrichment.

[0099] An observation is generally generated based on a single event (whether received from the telemetry database 116 or the message queue 106 directly). The single event could be a joint event (meaning that certain short-term correlations may already have been taken into account at the point an observation is generated). Each observation has at least one assigned threat score indicating the significance of the observation. For example, the threat score may denote one or both of confidence and severity (e.g., a high score may indicate a high confidence that an attack of material severity is occurring or has occurred). A threat score may, in that case, increase if the confidence increases, or the severity increases or both. In other implementations, separate confidence and severity scores may be computed and used within the system. Threat scores may be numerical or categorical (e.g., ‘high’, ‘medium’, Tow’). Observations and / or cases may be selectively escalated to an analyst based on their threat scores, to provide targeted alerts and / or reporting (reducing false positives, overreporting etc.).

[0100] Longer-term correlations are accounted for by grouping observations into cases when those observations appear to be related. The grouping of observations into cases considers longer-term temporal correlations between the underlying events. Once created, cases may be developed by matching subsequent observations to existing cases in the case database 124. A case may be assigned a threat score based on its constituent observations. Observations / cases may, for example, be populated with network data, endpoint data or a combination of endpoint and network data (or more generally different forms of telemetry data) obtained from multiple telemetry sources. The following description refers to network and endpoint events, but applies more generally to other forms of structured telemetry received and processed within the system, such as third-party telemetry, cloud telemetry etc.

[0101] A case may, for example, be created for at least one defined threat hypothesis, by clustering together observations of different tactics / techniques associated with the threat hypothesis. More generally, an observation may be generated in response to an event that is classed as potentially malicious, and observations may be grouped into cases when it isdetermined that they might relate to a common threat. Once a case has been created, it may be populated with further observation(s) that are identified as related to the case in question in order to provide a timeline of observations / events that underpin the case.

[0102] New observations can be matched to existing observations or cases using defined event association criteria, as applied to the content of the events - in particular the timestamps, but also other information such as entity identifiers (device identifier, IP address etc.). Three key pieces of metadata that are used as a basis for linking observations in this way are: 1. timestamps; 2. endpoint devices, and / or specific endpoint information (such as endpoint host name and / or endpoint open sockets); and 3. IP address.

[0103] There can be multiple pieces of metadata of each type, for example source and destination IP addresses. Such metadata of cases is derived from the event or events on which the case is based. Note the above list is not exhaustive, and the types of data can be used as a basis for observation linking.

[0104] For example, events may be associated with each other based on IP address where a source IP address in one event matches a destination IP address in another, and those events are within a given time window. IP addresses provide one mechanism by which endpoint events can be matched with related network events.

[0105] As another example, open sockets on an endpoint are a valuable piece of information in this context, as they are visible to the endpoint agent on the endpoint and associate specific processes running on that endpoint with specific network connections ("conversations"). That is, a socket associated with a process running on an endpoint (generally the process that opened the socket) can be associated with a specific five-tuple at a particular moment in time. This in turn can be matched to network activity within that conversation, for example by matching the five-tuple to the header data of packets tapped from the network. This in turn allows that network activity to be matched to a specific socket and the process associated with it. The endpoint itself can be identified by host name, and the combination of host name, five tuple and time is unique (and in many cases the five tuple and time will be unique depending on the network configuration and where the communication is going). This may also make use of the timestamps in the network and endpoint events, as the association between sockets and network connections is time limited, and terminates when a socket is closed.

[0106] As noted already, in networking, a five-tuple is a tuple of (source IP, destination IP, source port, destination port, transport protocol). This uniquely identifies a network connection within relatively small time windows. In order to match events based on network connection, a hash of the five-tuple can be computed from all network data and from endpoint process connection data (data relating to the network conversations individual processes on the endpoint are engaged in). By ensuring that all endpoint data also contains the host name (derived from the endpoint software), this allows any network event to be correlated with any endpoint event (network 5 tuple hash -> endpoint 5 tuple hash -> host name) and vice versa. This provides an efficient mechanism for linking specific network connections to specific programs (processes). Such techniques can also be used to link network activity to other event description data, e.g., a specific user account on an endpoint.

[0107] Note that a detection may be significant enough to result in an observation, but the observation may or may not be significant enough to escalate it to an analyst at that point. Similarly, the analyst will not generally be alerted to every new case. Rather, cases and / or observations may only be reported to an analyst to alert them to a potential threat when their threat scores reach a significance threshold, or meet some other significance condition. Thus, a large number of observations and / or cases may be created in the background to which an analyst is not alerted, because they are not deemed significant enough. As an example, a first observation may be generated which is not deemed significant enough to report. However, when a second observation is subsequently generated, the analysis may indicate a relationship to the first observation, causing those observations to be grouped in a case. In combination, those observations may or may not be significant enough for the case (group of observations) to be reported at that point (e.g., the case may never be reported, or it may only be reported when a further observation(s) has been subsequently added to it).

[0108] As noted, each case / observation is assigned at least one threat score, which denotes its significance. When the threat score for a case reaches a significance threshold or the case / observation meets some other significance condition, this causes the case to be rendered accessible via a case user interface (UI) 126.

[0109] Access to the cases via the case UI 126 is controlled based on the threat scores in the case records in the experience database 124. A user interface controller (not shown) has access to the cases in the experience database 124 and their threat scores, and is configured torender a case accessible via the case UI 126 in response to its threat score reaching an applicable significance threshold.

[0110] Such cases can be accessed via the case UI 126 by a human cyber defense analyst. In this example, cases are retrieved from the experience database 124 by submitting query requests via a case API (application programming interface) 128. The case (UI) 126 can for example be a web interface that is accessed remotely via an analyst device 130.[oni] Thus within the analysis engine there are effectively two levels of escalation.

[0112] Case and observation creation, driven by individual events or groups of events that are identified as potentially threat related.

[0113] Escalation of cases to the case UI 126, for use by a human analyst, only when their threat scores become significant, which may only happen when a time sequence of interrelated events has been built up over time.

[0114] As an additional safeguarding measure, the user interface controller may also escalate a series of low-scoring cases related to a particular entity to the case UI 126. This is because a series of low-scoring cases may represent suspicious activity in themselves (e.g., a threat that is evading detection). Accordingly, the platform allows patterns of low-scoring cases that are related by some common entity (e.g., user) to be detected, and escalated to the case UI 126. That is, information about a set of multiple cases is rendered available via the case UI 126, in response to those cases meeting a collective significance condition (indicating that set of cases as a whole is significant).

[0115] The event-driven nature of the analysis inherently accommodates different types of threats that develop on different time scales, which can be anything from seconds to months. The ability to handle threats developing on different timescales is further enhanced by the combination of real-time and non-real time processing within the system. The real-time enrichment joining and providing of queued events from the message queue 106 allows fastdeveloping threats to be detected sufficiently quickly, whilst the long-term storage of events in the telemetry database 116, together with batch enrichment, provide a basis for non-real time analysis to support this.

[0116] The above mechanisms can be used both to match incoming events from the message queue 106 and events stored in the telemetry database 116 (e.g., earlier events, whoserelevance only becomes apparent after later event(s) have been received) to cases. Appropriate timers may be used to determine when to look for related events in the telemetry database 116 based on the type of event. Depending on the attacker techniques to which a particular event potentially relates, there will be a limited set of possible related events in the telemetry database 116. These related events may only occur within a particular time window after the original event (threat time window). The platform can use timers based on the original event type to determine when to look for related events. The length of the timer can be determined based on the threat hypothesis associated with the case.Analysis Framework:

[0117] The analysis engine is shown to comprise a machine reasoning framework 120 and a human reasoning framework 122. The machine reasoning framework 120 applies computer- implemented data analysis algorithms to the events in the telemetry database 116, such as ML techniques.

[0118] The detectors referred to above form (part of) the machine reasoning framework 120. The detectors run structured queries on the telemetry database 116, create observations in the experience database 124, and query those observations e.g. to group them into cases.

[0119] Individual events may be related to other events in various ways but only a subset of these relationships will be meaningful for the purpose of detecting threats. The analysis engine 118 uses structured knowledge about attacker techniques to infer the relationships it should attempt to find for particular event types.

[0120] This can involve matching a received event or sets of events to known tactics that are associated with known types of attack (attack techniques). As noted, within the analysis engine 118, a plurality of detection modules are provided, each of which queries the events (and possibly other data) to detect suspicious activity. A "triggering event” in this context refers to a specific analytic result or set of analytic results that triggers the creation of an observation. For example, a detection module might be associated with a tactic and technique that describes respective activity it can find. On finding such activity, an observation is created. A hypothesis may, for example, define a set of possible tactics or techniques that may occur proximate in time to each other (and related to the same, or some of the same, infrastructure), enabling corresponding observations to be grouped into a case. Because each hypothesis is expressed as tactics or techniques, there may be many different detectionmodules that can contribute observations to a case. Tactics are high level attacker objectives like "Credential Access", whereas techniques are specific technical methods to achieve a tactic. In practice it is likely that many techniques will be associated with each tactic.

[0121] For example, it might be that after observing a browser crashing and identifying it as a possible symptom of a "Drive-by Compromise" technique (and creating an observation in response), another observation proximate in time indicating the download of an executable file may be recognized as additional evidence symptomatic of "Drive-by Compromise" (enabling those observations to be grouped to build up a case). Drive-by Compromise is one of a number of techniques associated with an initial access tactic.

[0122] As another example, an endpoint event may indicate that an external storage device (e.g., USB drive) has been connected to an endpoint and this may be matched to a potential “Hardware Additions” technique associated with the initial access tactic. The analysis engine 118 then monitors for related activity such as network activity that might confirm whether or not this is actually an attack targeting the relevant infrastructure.

[0123] This is performed as part of the analysis of events that is performed to create new cases and match events to existing cases. As indicated, this can be formulated around the "MITRE ATT&CK framework". The MITRE ATT&CK framework is a set of public documentation and models for cyber adversary behavior. It is designed as a tool for cyber security experts. In the present context, the MITRE framework can be used as a basis for creating and managing cases. In the context of managing existing cases, the MITRE framework can be used to identify patterns of suspect (potentially threat-related behavior), which in turn can be used as a basis for matching events received at the analysis engine 118 to existing cases. In the context of case creation, it can be used as a basis for identifying suspect events, which in turn drives case creation. This analysis is also used as a basis for assigning threat scores to cases and updating the assigned threat scores as the cases are populated with additional data. However it will be appreciated that these principles can be extended to the use of any structured source of knowledge about attacker techniques. The above examples are based on tactics and associated techniques defined by the Mitre framework. The described techniques are not limited to Mitre, and can be applied with other forms of tactics / techniques, e.g., in alternative (including bespoke) threat models, or tactics / techniques that are learned via supervised or unsupervised machine learningprocessing (or other pattern recognition or statistical analysis methods). ‘Learned’ tactics or techniques characterize potential attacks in machine-understandable terms, which may or may not be interpretable to a human. Tactics / techniques may for example be learned by training one or more models on existing or synthetic attack data, and / or from data learned in recording human analyst behavior.Case Content:

[0124] Each case record is populated with data of the observations identified as relevant to the case. A case provides a timeline of observations that have occurred and a description of why it is meaningful, i.e., a description of a potential threat indicated by those events.

[0125] In addition to the event timeline, a case record contains attributes that are determined based on its constituent events. Four key attributes are: 1. people (users); 2. processes; 3. devices; and 4. network connections.

[0126] A case record covering a timeline of multiple observations may relate to multiple people, multiple devices and multiple users. Attribute fields of the case record are populated with these attributes based on its constituent events.

[0127] A database case schema dictates how cases are created and updated, how they are related to each other, and how they are presented at the case UI 126.Enhanced endpoint agent:

[0128] FIG. 5 shows a schematic block diagram of an endpoint device 312 on which an enhanced form of endpoint agent is executed. The enhanced endpoint agent is denoted by reference numeral 616 and may also be referred to herein as an endpoint network sensor (EPNS).

[0129] Whilst the endpoint agent 316 of FIG.1 is deployed for the purpose of endpoint activity monitoring, the EPNS 616 is additionally responsible for monitoring local network traffic. That is, in addition to collecting endpoint data, the EPNS 616 additionally monitors local network traffic to and from the endpoint device 312 in order to collect network data locally at the endpoint device 312. Local network traffic monitoring by the EPNS 616 reduces the reliance on network TAPs and other centralized network monitoring components.The description below may refer to the EPNS 616 as the endpoint agent 616 or the network sensor 616 for conciseness.

[0130] A set of processes 602 is shown to be executed on the endpoint device 312 and the EPNS collects at least some of the endpoint data by monitoring local activity by the processes 602.

[0131] One option is for the EPNS 616 to collect and send copies of all incoming / outgoing network packets for server-side processing, in the manner of a TAP or mirror (but sending a full copy of only its ‘raw’ local network traffic). In this case, the local network traffic copy would be sent to the coal face producers 102 and / or standardizers 104 of FIG.1 for preprocessing into structured events in the same way as raw network traffic received from dedicated monitoring components). However, to reduce transmission overhead, some or all of the functions of the coals face producers 102 / standardizers 104 may be performed locally by the EPNS 616 instead. In such cases, the EPNS 616 instead transmits a more concise summary of its local traffic, in the form of structured network traffic metadata. The term ‘network data’ is used broadly, unless otherwise indicated, and does not necessarily imply ‘raw’ network data (in the context of EPNS reporting, network data can take the form of more-concise network metadata summarizing local network traffic).

[0132] In the following examples, the network data collected by the EPNS 616 takes the form of network traffic metadata summarizing its local network traffic. The EPNS 616 processes the incoming and outgoing local traffic in order to extract such metadata therefrom. The extracted metadata summarizes incoming and outgoing packets of the local traffic. The incoming and outgoing packets carry process data intended for and generated by the processes 602 respectively. The EPNS 616 transmits the extracted metadata to an endpoint server 620 for further processing. The endpoint server 620 forms part of the cybersecurity platform that provides a cybersecurity service implemented remotely from the endpoint device 312.

[0133] A key piece of network information is a connection or other “flow” identifier. As noted, a connection is defined by a tuple of (source IP address, source port, destination IP address, destination port, transport protocol). A “flow” generalizes the concept of a connection to connectionless protocols (see below). The opening of a connection or establishment of a flow is a key piece of information that can be used for threat detection andanalytics. In the described examples, at a minimum, the EPNS 616 reports every flow that is established at the endpoint device 312 (see below for further details), preferably in combination with additional network associated with the flow. Examples of additional types of network metadata are described below.

[0134] The endpoint 312 is shown to execute an operating system (OS) 604, on which the processes 602 and the EPNS 616 run. The processes 602 typically include instances of one or more applications 606 stored in computer storage 608 of the endpoint device 312. One function of the OS 604 is to manage the processes 604 and allocate resources to them. The OS 604 also regulates the flow of network traffic between a network interface 610 of the endpoint device 312 and the processes 602 and the EPNS 616.

[0135] In addition, the OS 604 provides a local traffic access function 612 and a local activity monitoring function 614. These may, for example, be provided as part of one or more application programming interfaces (APIs) of the OS 604. The EPNS 616 uses the local traffic access function 612 in order to obtain duplicate copies of all incoming and outgoing network packets received at the network interface 610. This includes network packets sent to and from the processes 602, carrying inbound and outbound process data respectively. The EPNS 616 may also receive duplicate copies of its own incoming and outgoing network packets. The EPNS 616 processes the duplicate packet in order to extract the network traffic metadata.

[0136] In addition, the EPNS 616 uses the local activity monitoring function 614 to monitor endpoint activity by the processes 602. Examples of the type of endpoint activity that may be monitored may include, but are not limited to, the opening of ports, the accessing of files etc. Such monitoring is used to determine endpoint data. The monitoring may be ongoing, even if the endpoint data is static. For example, processes may be monitored in order to link some activity by a process to one or more network packets. In that case, the endpoint data may take the form of a process identifier (ID) that is associated with a telemetry record(s) summarizing those packet(s) (and conceivably other identifier(s) such as a file identifier if such information is available). Endpoint data can also include user information, such as details of a user account associated with an incident of network activity, or host information about the endpoint device 312. In that case, a telemetry record of the network activity may be associated with a user identifier. For example, the endpoint data collected by the EPNS 616and associated with the network traffic records can comprise any combination of process, host and / or user data, for example. For example, with current operating system APIs, it is generally possible to obtain some or all of the following endpoint data for particular network packets: username, process details, parent process details, process path, process command line string. In the following examples, actions by the processes are monitored, in order to link such identifiers to network packets.

[0137] The endpoint device 312 is an endpoint of a packet-based network 630 to which it is connected to the network interface 610, and through which the incoming and outgoing network traffic flows. The packet-based network 630 could be a “closed” network such as an enterprise or corporate network (e.g., the network 300 of FIG. 3). However, it could alternatively be an “open” network (such as the internet 306). Referring to FIG. 3, the network 630 could be the Internet 306 when the endpoint device 312 is “roaming”, or the private network 300 when the endpoint device is “non-roaming”.

[0138] Even when the endpoint device 312 is not currently connected to the private network 300 (whether directly or via a VPN connection), that does not necessarily mean that it poses no threat to the private network. For example, the endpoint device 312 could still contain sensitive data or, should the endpoint device 312 become infected with some form of malware, that could propagate into the private network 300 when the endpoint device 312 does subsequently connect to it. There are therefore significant benefits to being able to detect cybersecurity threats even when the endpoint device 312 is roaming.

[0139] The endpoint device 312 could, for example, take the form of a user device such as a laptop or desktop computer, tablet or smart phone etc. The primary function of a user device is to provide useful functions to a user of the device. Such functions are implemented by the processes 602. The EPNS 616 is deployed on such a user device to provide secondary network and endpoint monitoring functions and submit cybersecurity data (network metadata and associated endpoint data in the present example) to the cybersecurity platform for analysis.Investigation GUI:

[0140] A graph-based investigation GUI will now be described in further detail. The investigation GUI enables users to more effectively navigate existing observations and cases,and to augment cases / observations with additional related telemetry as investigations are carried out.

[0141] The following description considers multiple observations that have been automatically generated and grouped into a case in the experience database 124, in the manner described above. However, all descriptions below pertaining to cases involving multiple observations apply equally to a single observation which has been raised in the experience database 124. Either way, the starting point for the knowledge graph is a subset of already ‘curated’ telemetry data that has triggered one or more detections within the platform.

[0142] As discussed, there may be many entities and entity types that are directly or indirectly associated with an observation. For example, an observation might pertain to a process, which in turn might be associated with a device and a parent process running on the device, or with a particular user identity. The device may, in turn, be associated with a network connection or network flow, and so on. The number of unique entities typically grows with the number of observations belonging to a case.

[0143] The investigation GUI generates and displays a knowledge graph representing the observations belonging to a case and an initial subset of the entities related to the case. The initial subset of entities is determined by the observation(s). Entities and observations are represented as nodes and relationships between observations and entities, and relationships between different entities, are represented as edges connecting the corresponding nodes. The GUI allows a user to modify the graph, by adding or removing nodes. For example, additional entity nodes and edges may be added by running structured queries on the telemetry database 116. A simplified query interface is provided to enable more efficient and precise telemetry querying, with reduced manual effort. Entity nodes and edges added in this way reflect entities and their relationships that have been captured in the structured telemetry data.

[0144] In addition to visualising entities and their relationships, the GUI supports a range of actions relating to entities represented in the graph.

[0145] Context is provided by the graph of interconnected information. All the information in relation to why an action was performed, what information was used to ask the action and what was the result of the action are known and captured within the knowledge graph thatrepresents the connected data. As the graph builds and the information it represents increases, the data will be stored along with the case to which it relates.

[0146] The knowledge graph is generated as a data structure in memory based on the observations belonging to the case. The node / edge structure of visualization is also reflected in the knowledge graph data structure. For example, nodes may be implemented as inmemory objects, and edges as references between objects. Any manual changes / additions to the knowledge graph are persistently stored in association with the case, enabling the modified knowledge graph to be reconstructed at a later time. Whilst the following description mainly focuses on the visualization of the knowledge graph, description pertaining to the structure and contents of the knowledge graph applies equally to the underlying data structure.

[0147] FIG. 6 shows a view within the GUI, in which an initial knowledge graph 60 is displayed. A list of cases 62 is displayed, in which cases are ordered by severity (e.g. by threat score assigned in the manner described above). A case 64 is shown selected in the GUI, and the depicted knowledge graph 60 pertains to the selected case 64. Four observations (“A”, “B”, “C” and “D”) belong to the selected case 64. Each observation is represented by an observation node in the knowledge graph, such as a first observation node ONI representing observation “A” and a second observation node ON2 representing observation “C”. In the present context, observations are treated as a specific entity type.

[0148] The observations of the case are separately represented on a timeline 68, by observation markers. Each observation marker (e.g., first and second observation markers OM1, OM2 representing observations “A” and “C” respectively) is placed at a location on the timeline 68 corresponding to a timestamp of the observation it represents.

[0149] The initial knowledge graph 60 summarizes the case 64 and, as noted, contains already curated data from the detectors running within the platform. As described above, detectors query the telemetry database 116 and submit activities of interest which become observations stored in the experience database 124.

[0150] To generate the knowledge graph 60, the case UI 126 queries the experience database 124, via an application programming interface (API), to obtain a list of all the observations in the case 64. The knowledge graph 60 is generated at the point the information from the API is loaded by the case UI 126 using the subset of telemetry curated from telemetry database 116.

[0151] Entities related to the selected case 64 are represented as entity nodes. Different types / categories of entity may be related to a case. In this example, the following entity types are represented as nodes in the graph 60: devices (represented by device nodes), network connections or network flows more generally (represented by network nodes), and processes (represented by process nodes). Other entities, such as users or files, may be similarly represented by additional nodes in the graph. Relationships between entities / observations are represented by edges between the corresponding nodes.

[0152] Both network entities (such as network connections) and endpoint entities (such as processes, users etc.) are included as nodes in the knowledge graph, and relationships between network and endpoint entities may be captured and visualized as edges.

[0153] Nodes may be directly connected (with an edge between them) or indirectly connected (e.g. both directly connected to a common node; for example, two nodes directly connected to the same observation node are said to be indirectly connected).

[0154] For example, in the graph 60, a first process node PN1 a represents a first process. First and second device nodes DN1, DN2 represent first and second devices respectively. A first network node NN1 represents a first network connection.

[0155] Visual attributes of the entity nodes may be used to convey information about the entities. For example, device nodes can have different icons, colours etc. depending on the type of device. For example, ‘suspicious’ devices (devices that have been flagged as having been involved in potentially suspicious activity) might be represented as red nodes that are visually distinct from other device nodes. In the graph 60, the second device node DN2 is rendered in a manner that visually identifies the second device as suspicious.

[0156] The graph 60 is dynamic in various respects.

[0157] A user can select a node within the graph 60 via a first selection input (e.g. hovering a cursor over the node) to visually highlight the selected node’s relationships to other nodes in the graph 60.

[0158] FIG. 6A shows a partial view of the GUI when the first network node NN1 is selected in this manner. Selecting the first network node NN1 causes all edges connected to that node to be highlighted: a first edge El between the first network node NN1 and the first device node DN1; a second edge E2 between the first network node NN1 and the second devicenode DN2; and a third edge E3 between the first network node NN1 and the first observation node ONI. Edges El and E2 imply that the first network node NN1 represents a connect! on / flow between the first and second devices, whilst edge E3 implies that observation A is associated with that connection / network flow.

[0159] FIG. 6B shows a partial view of the GUI when the first device node DN1 is selected. This also causes edge El between the first network node NN1 and the first device node DN1 to be highlighted, in addition to (among others) a fourth edge E4 between the first device node DN1 and the first process node PN1 (implying the first process is or was executed on the first device), and fifth and sixth edges E5, E6 between the first device node DN1 and the first and second observation nodes ONI, ON2. Edges E5 and E6 imply that observations A and C each are both associated with the first device.

[0160] FIG. 6C shows a partial view of the GUI when the first observation node ONI is selected. Among other things, this causes a seventh edge E7 between the first observation node ONI and the first process node PN1 to be highlighted, implying observation A is associated with the first process. Typically, a process would become associated with an observation because it has been associated with a telemetry record via endpoint monitoring (e.g. by the endpoint agent 316 or EPNS 616), which in turn gives rise to an observation.

[0161] Other types of relationships may be represented. For example, as described above, the platform is capable of collecting telemetry which links endpoint data (such as a process identifier, user identifier, file identifier etc.) with network data (such as a network connection or flow). For, an endpoint entity (such as a process, user, file etc.) that has been associated with a network entity (such as a network connection / flow etc.), an edge between the corresponding nodes may be generated and displayed.

[0162] FIG. 6D shows a partial view of the GUI when the second observation node ON2 is selected. Unlike the first observation, the second observation is not associated with any specific process(es).

[0163] FIG. 6E illustrates another dynamic aspect of the graph 60. The GUI is able to present a list of selectable node types, enabling a user to add or remove specific types of node from the graph. In the depicted example, the user has deleted the observation node type, causing the observation nodes to be removed from the graph. Each node is associated with a semantic label, with an option to display or hide the labels.Predetermined actions

[0164] FIG. 7 shows a GUI view, in which a second process node PN2 has been selected via a second selection action (e.g. clicking on the node). The second process node PN2 represents a second process associated with the second device, hence an edge between the second process node PN2 and the second device node DN2 is highlighted when the second process node PN2 is selected.

[0165] Details 700 of the second process are displayed in response to the second selection action on the second process node PN2.

[0166] In addition, one or more predetermined “action” indicators are shown, where an action refers to a user-initiated intervention in this context (system-initiated actions are also considered below, in the context of automated investigation / remediation). One type of an action is running a telemetry query. A query action may also be referred to as a ‘question’ herein. Other types of actions include, for example: removing a node in the knowledge graph; adding a comment node related to selected node(s); labelling or categorising a node e.g. labelling or categorising a node as "interesting", "irrelevant", etc.; instigating an intervention on a selected device node, e.g. isolating the corresponding device from one or more other devices, retrieving information from the selected device; adding the selected device to a list such as an exclusion list for a specified detection (to prevent the device from being flagged in future detection outputs), or finding all cases / observations which reference the device; instigating an intervention on a selected process node, such as terminating the corresponding process etc.

[0167] An action is defined by a condition or set of conditions that define applicability of the action. An action is typically associated with an entity type (or types) to which it is applicable (with additional applicability condition(s) in some cases). An indicator of each action applicable to the entity type of the selected node PN2 is displayed. A query action (question) is additionally associated with a query template.

[0168] In this example, a question indicator 702 is displayed, indicating a predetermined query type applicable to the selected node PN2, namely a “find parent process” query, which is applicable to process-type entity nodes. Once this question is selected, information is collected from all relevant nodes in order to be able to automatically build a structured query (by populating a query template associated with the selected question) and then run that queryagainst the telemetry database 116. The structured query contains the information required to find the parent process, which would include an identifier of the selected process (the second process represented by process node PN2 in this case), and an indication of the target relationship to the selected node (‘parent’ in this case).

[0169] More generally, a question requires one or more pieces of context data (such as an entity identifier) that is indicated via the selection of one or more existing nodes in the graph. The applicability criteria of a question are constructed to ensure that the question is only made available when the required context data has been indicated though the selection of an existing node or nodes in the graph. The context data used to generate a query is extracted from the information contained in the knowledge graph. That knowledge either forms part of the underlying observation(s), or has been previously added to the knowledge graph by the user. User-added knowledge can still be tracked back to the underlying observations, through a sequence of investigation step(s) starting from the underlying observation(s).

[0170] The ‘find parent process’ query obtains details of a parent process, which requires an identifier of a target process, and hence the selection of a process node. Thus, the ‘find parent process’ question is only displayed when a process-type node is selected (and which is not already connected to a parent process node in the knowledge graph). In more detail, applicability for the "find parent process" action requires: a selected PROCESS node with a "pid" (process identifier) value; that the selected PROCESS node is not the destination of any PROCESS P ARENT relationship; the existence of a neighbouring DEVICE node with a "host name" value (needed to query for the parent process); and an OBSERVATION node neighbouring the DEVICE node with a timestamp.

[0171] Whilst this example considers a question applicable to a single node, a question may be defined that is applicable to multiple nodes. Such questions are made available when multiple nodes satisfying the question’s applicability criteria are selected.

[0172] FIG. 8 shows information that is retrieved and displayed to the user in response to the user selecting the ‘find parent process’ question - details of the parent process in this case. This information is not added to the knowledge graph unless and until the user confirms the addition.

[0173] An acceptance window is then displayed to the user which contains the information found by the query. It can be cancelled (if it is considered to not be of interest to the case 62)or it can be accepted. Two selectable options are displayed, to confirm the addition of the parent process (“Accept”) or to disregard the results (“Cancel”). These results may or may not have any analytical value, and it is therefore desirable for the user to be able to review the located details before deciding whether to update the knowledge graph 60. This gives the user control over the ‘assimilation’ of knowledge into the knowledge graph.

[0174] Note that, in both cases, the running of the ‘find parent process’ query is recorded, along with an indication of whether or not the user added the retrieved information to the knowledge graph 60 (see FIG. 14, and the accompanying description below).

[0175] An additional intention for the acceptance step is to capture acceptance information such as user provided context and rationale for acceptance, e.g., label, description, categorisation (e.g., context, evidence), etc. The acceptance interface allows customisation per action. For example, the acceptance window allows a label and description field for the "add a comment" action.

[0176] Whilst this example considers processes and parent processes, the same techniques can be extended to other types of interrelated entities.

[0177] If the question results are accepted, a new node representing the parent process is generated. This results in an updated knowledge graph 90, shown in FIG. 9.

[0178] FIG. 9 shows a view of the GUI in which the updated knowledge graph 90 is displayed. As can be seen, the updated knowledge graph 90 includes a third process node PN3 representing the parent process located in the process depicted in FIGS. 7-8. The updated graph 90 also includes an edge between process node PN3 and process node PN2 representing the parent-child relationship between the second and third processes.

[0179] The updated knowledge graph 90 is persistently recorded in association with the case 62 to which it relates, to allow it to be accessed at a later time. In particular, new nodes / edges that are added to the knowledge graph are persistently stored in association with the case 62.

[0180] These features enable an interactive process to grow the knowledge graph, whilst minimizing extraneous information and maximizing analytical value. At any given point, the information that is currently represented within the graph determines what question or questions can be asked. As questions are asked (though querying on the telemetry database 116) and the answers are received, a user is able to build upon the information already withinthe graph in a targeted fashion. This creates a linkage between information or, in other words, an evidence chain, which is recorded in association with the case (see FIG. 14). The process begins with initial information and as questions are asked, the knowledge builds so that the graph grows.

[0181] Given a selected node, the applicability process determines which actions are relevant and possible given the selected node and any required context associated with the selected node. The ability to associate context criteria with an action is highly flexible. The applicability condition(s) may, for example, depend on attribute(s), of the selected node, its relationship(s), their relationship attribute(s), attribute(s) of one or more (direct or indirect) neighbouring nodes, or any combination thereof.Comment nodes

[0182] Comments can be included in the knowledge graph as node elements (comment nodes), allowing specific targeting, and targeting of multiple nodes, to imply relationships which may not be represented in telemetry (e.g., a user might add a comment such as "The report of unexpected location for these three devices is explainable by the owners’ attendance of a conference.")

[0183] FIG. 9 shows another aspect of the GUI, which is the ability to select one or multiple nodes and add a comment node associated with the selected node(s). In this example, three nodes (NN1, NN2, PN2) have been selected and an option 92 to add a comment node 92 is displayed.

[0184] FIG. 10A shows a set of comment fields (title, contents) that are displayed in response to the user selecting the comment option 92. The user can enter free text in these fields, and then generate a comment node by selecting an ‘Accept’ option.

[0185] Multiple nodes can be selected and a comment node can be added, which is connected to each of those nodes, and associated with user-entered text.

[0186] FIG. 10B shows a comment node CN1 created in this manner, with edges connecting it to the previously-selected nodes NN1, NN2, PN2. The comment node links together the selected nodes via edges in the graph. The list of the connected items can be seen below the comment on the right-hand side. When the comment node CN1 is selected, the previously- entered text is displayed.

[0187] Comment nodes and their associated text and edges are persistently recorded in association with the case 62.Network connections

[0188] A device can be selected and a question regarding what previous network connections were on this device can be asked. In the received response is a list of telemetry records. The records of interest within the list can be selected and then accepted. Upon accepting, a new node is created in the graph that represents the selected records. This allows the user to flag which items within the telemetry are of interest to the case.

[0189] Nodes representing network connections can also be added to the knowledge graph. Many network connections associated with a given device might have been recorded in the telemetry. The user is able to review network connections associated with a given device, and choose which of these (if any) to add to the graph.

[0190] FIG. 11 shows a view in which device node DN2 is selected. An option 1102 is displayed to view previous network connections associated with the corresponding device. Upon selection, the telemetry database 116 is queried to obtain a list of previous network connections (or flows) associated with the device.

[0191] FIG. 12 shows a list of network connections obtained in this manner. The user is able to select a subset of these network connections (which they consider to have analytical value) and add them to the graph.

[0192] FIG. 13 shows a network connection node NCN added to the graph, which represents the subset of previous network connections selected as relevant in FIG. 12. This node is selectable to view details of those network connections (three network connections, corresponding to three ‘rows’, were selected in this example). The network connection node NCN is persistently stored in association with the case 62. The NCN node may, for example, be a general DATA type node, with appropriate annotation(s) / label(s) added by the user.Evidence chain

[0193] To support explainability, the investigation history provides a log of each investigation step.

[0194] The recording and display of investigation history is provided for communication between the platform and users, and also between different teams; auditability of investigations; playback of history for tutorial; rich investigation process and timing telemetry; and for debugging purposes.

[0195] Each investigation step taken by the user (such as asking a question, accepting / rejecting the results of a question, adding a comment, querying previous connections etc.) is recorded in a persistent investigation history associated with the case, and time stamped.

[0196] FIG. 14 shows an investigation history view, in which a time sequence of recorded investigation steps is displayed. This includes the question, for example the parent process for a particular node. Whether it had been successful, what time it was performed, and whether it was accepted are all also recorded. A list of all investigative actions taken against the case can be built. This is useful both for investigative and auditing purposes, and to provide explainability in other contexts (such as automation of investigations based on machine learning).

[0197] In this case, the investigation history is shown to comprise three entries: a first entry 1402 recording the asking of the ‘find parent process question’ (in a ‘questionKey’ field), a timestamp of the query, the acceptance of the results (in a Boolean ‘accepted’ field), and an identifier of an existing node or nodes (at the time the question was asked) to which the question related (process node PN2 in this case, identified in an ‘inputNodes’ field).Automated investigation learning

[0198] Records of user interactions may be used as labelled examples of the investigation process. For example, such data may be used as training data for machine learning, providing a path to automating the investigation process.

[0199] For example, each investigation history (sequence of recorded investigation steps) can be used as a datapoint to train an automated investigation component via machine learning. For example, investigation patterns can be inferred from investigation histories collected from multiple users, and subsequently automated. For example, a model could be built which suggests potential actions a user could take, or automatically take investigative actionswithout user input. This is possible because the structured investigation approach facilitated by the GUI enables forensic data collection.

[0200] With DATA nodes more generally, a user may select individual entries (‘rows’) of interest for a telemetry dataset. The user’s row selections can be used as labelled data for machine learning purposes.Predetermined action types:

[0201] In one implementation, a library of predetermined action types is provided. Each action is associated with a set of applicability condition(s) and logic (e.g. a set of rules) for carrying out the action. For example, a query action (question) is associated with a structured query template that can be populated when the set of applicability condition(s) is satisfied.

[0202] Applicability conditions can be defined flexibly based on e.g. an attribute of the node itself, the existence of a particular type of neighbouring node or relationship, a neighbouring node attribute, and / or a relationship attribute etc.

[0203] An action builder interface is provided, which allows new questions and other types of actions to be added. The action builder interface enables, for example, a user to define a new question and associate any of the following with a new question: "question metadata", a context based "applicability" rule(s), a structured telemetry query, a result "acceptance" interface, and a knowledge graph "assimilation" procedure(s). Feedback is provided to users to validate their applicability, query and assimilation definition. The action builder interface can be similarly used to define other action types and their applicability condition(s), such as "Isolate device", "Add a comment", etc.

[0204] FIG. 15 shows an interface for applicability condition(s) associated with a new question type. Using this question builder interface, a user can create an ‘applicability’ rule set which determines whether the question is relevant and possible to ask in relation to a selected node and to the knowledge that currently exists in the graph. It starts with the selected node, for example whether this node is of the “process” type.

[0205] FIG. 16 shows a query template associated with a question type. The query template is related to the applicability rule set, in that data elements defined in the applicability conditions can be referenced in the query template. Therefore, when the question is asked,data elements extracted to evaluate the applicability conditions can also be used to populate the query template.

[0206] The GUI can also be extended to other query modalities, such as natural language queries performed on the telemetry database 116 using a large language model.

[0207] The GUI thus supports an investigation process as an iterative expansion of an empirical hierarchy of evidence, to provide relevant actions, maximise explainability and minimise the risk of premature conclusions.

[0208] It will be appreciated that the examples described above are illustrative rather than exhaustive. In general, the functional components described above can be implemented in one or more computing devices at one or more locations within a localized or distributed computer system. A computer system comprises computing hardware which may be configured to execute any of the steps or functions taught herein. The term computing hardware encompasses any form / combination of hardware configured to execute steps or functions taught herein. Such computing hardware may comprise one or more processors, which may be programmable or non-programmable, or a combination of programmable and non-programmable hardware may be used. Examples of suitable programmable processors include general purpose processors based on an instruction set architecture, such as CPUs, GPUs / accelerator processors etc. Such general-purpose processors typically execute computer readable instructions held in memory coupled to the processor and carry out the relevant steps in accordance with those instructions. Other forms of programmable processors include field programmable gate arrays (FPGAs) having a circuit configuration programmable through circuit description code. Examples of non-programmable processors include application specific integrated circuits (ASICs). Code, instructions etc. may be stored as appropriate on transitory or non-transitory media (examples of the latter including solid state, magnetic and optical storage device(s) and the like).

Claims

Claims1. A computer-implemented method of rendering a graphical user interface (GUI) for querying a collection of structured cybersecurity telemetry data, the method comprising: accessing in an experience database at least one observation, the at least one observation having been generated automatically by at least one detector operating on a collection of structured cybersecurity telemetry data; based on the at least one observation, generating a knowledge graph comprising (i) entity nodes representing entities associated with the at least one observation, and (ii) edges between the entity nodes, the edges representing relationships between the entities; generating a visualization of the knowledge graph in the GUI; receiving a first user input denoting selection of at least a first entity node of the knowledge graph, the first entity node representing a first entity; based on the first user input, extracting from the knowledge graph one or more pieces of context data and generating a structured query comprising the one or more pieces of context data; identifying, by running the structured query on the collection of structured cybersecurity telemetry data, a second entity having a relationship to the first entity; displaying in the GUI details of the second entity; receiving a second user input denoting acceptance of the second entity; and based on the second user input, generating within the knowledge graph: a second entity node representing the second entity and an additional edge between the first entity node and the second entity node, the additional edge representing the relationship to the first entity.

2. The method of claim 1, comprising determining that the first user input satisfies one or more applicability criteria associated with a predetermined query action, wherein the structured query is generated by accessing a query template associated with the predetermined query action and populating the query template based on the one or more pieces of context data.

3. The method of claim 2, comprising displaying, responsive to determining that the first user input satisfies the applicability criteria, an indication of the predetermined query action,wherein the structured query is run on the telemetry database responsive to a third user input denoting selection of the predetermined query action.

4. The method of claim 3, comprising persistently storing in an investigation history associated with the at least one observation: an indication of the third user input denoting selection of the predetermined query action, and an indication of the second user input denoting acceptance of the second entity.

5. The method of claim 2, 3 or 4, wherein the one or more pieces of context data are used to determine that the first user input satisfies the applicability criteria.

6. The method of any of claims 1 to 5, wherein the one or more pieces of context data comprise: an attribute of the first entity node, an attribute of an edge connected to the first entity node, or an attribute of another entity node in the knowledge graph directly or indirectly connected to the first entity node by one or more edges.

7. The method of any preceding claim, wherein the structured query returns a plurality of second entities each having a relationship to the first entity, and details of the plurality of second entities are displayed, wherein the second user input denotes a selection of the second entity from the plurality of second entities.

8. The method of any preceding claim, wherein the first entity is a device, and the plurality of second entities is a plurality of network connections associated with the first entity.

9. The method of any preceding claim, wherein the knowledge graph comprises a third entity node related to the first entity and an edge between the first entity node and the third entity node, wherein the second entity is related to the third entity, and a second additional edge is added to the knowledge graph between the second entity and the third entity.

10. The method of claim 9, wherein the first and second entities are processes, and the third entity is a device on which the processes are or have been executed.

11. The method of any preceding claim, wherein the knowledge graph is generated based on a case stored in the experience database, the case having multiple observations associated therewith, wherein the knowledge graph comprises an observation node for each observation, a plurality of other entity nodes, and an edge between the observation node and each other entity node related to that observation.

12. The method of any preceding claim, comprising: receiving a fourth user input denoting a selection of a plurality of nodes within the knowledge graph; displaying within the GUI a comment field for receiving user-input text; generating in the knowledge graph a comment node and a plurality of further edges between the comment node and the plurality of nodes, the comment node associated with user-entered text in the comment field; and persistently storing, in association with the at least observation, an indication of the comment node, the plurality of further edges, and the user-entered text.

13. The method of any preceding claim, comprising persistently storing, in association with the at least one observation, an indication of the second entity and the relationship to the first entity.

14. The method of any preceding claim, wherein the one or more pieces of context data comprise a first entity identifier of the first entity represented by the first entity node.

15. The method of any preceding claim, wherein the first user input denotes a selection of multiple first entity nodes, the one or more pieces of context data relating to the multiple first entity nodes.

16. A computer-implemented method of rendering a graphical user interface (GUI) for querying a collection of structured cybersecurity telemetry data, the method comprising:accessing in an experience database at least one observation, the at least one observation having been generated automatically by at least one detector operating on a collection of structured cybersecurity telemetry data; based on the at least one observation, generating a knowledge graph comprising (i) entity nodes representing entities associated with the at least one observation, and (ii) edges between the entity nodes, the edges representing relationships between the entities; generating a visualization of the knowledge graph in the GUI; receiving a first user input denoting selection of at least a first entity node of the knowledge graph, the first entity node representing a first entity; based on the first user input, extracting from the knowledge graph one or more pieces of context data; determining that a predetermined action is applicable to the first user input, by using the one or more pieces of context data to evaluate one or more predetermined applicability conditions associated with the predetermined action; displaying within the GUI an indication of the predetermined action applicable to the first user input; and responsive to a second user input denoting selection of the predetermined action, causing the predetermined action to be performed in relation to the first entity.

17. The method of claim 16, wherein the one or more pieces of context data comprise: an attribute of the first entity node, an attribute of an edge connected to the first entity node, or an attribute of another entity node in the knowledge graph directly or indirectly connected to the first entity node by one or more edges.

18. The method of any of claims 15 to 16, wherein the predetermined action is: a query action to obtain additional information relating to the first entity, a cybersecurity intervention action performed on the first entity, or an action to reconfigure the at least one detector based on the first entity.

19. The method of claim 18, wherein the first entity is a device, and the action is a cybersecurity intervention action, which comprises isolating the first device from one or more other devices.

20. The method of claim 18, wherein the predetermined action is an action to reconfigure the at least one detector, which comprises adding the first entity to an exclusion list associated with the at least one detector.

21. The method of claim 18, wherein the predetermined action is a query action, which is: performed on the collection of structured cybersecurity telemetry data to find one or more pieces of telemetry data related to the first entity, or performed on a database of observations or cases to find one or more further observations relating to the first entity to locate which have been generated automatically, a case being a group of related observations.

22. The method of any of claims 16 to 21, comprising persistently storing in an investigation history associated with the at least one observation an indication of the second user input denoting selection of the predetermined action.

23. The method of claim 4 or 22 or any claim dependent thereon, comprising using the investigation history to train a machine learning investigation model to perform automated threat investigation.

24. The method of any preceding claim, wherein the entity nodes comprise an endpoint node representing an endpoint entity and a network node representing a network entity, and the edges comprise an edge between the endpoint node and the network node representing a relationship between the endpoint entity and the network entity.

25. A computer system comprising one or more computers configured to implement the method of any preceding claim.

26. Non-transitory or transitory media embodying computer-readable instruction configured so as, upon execution by one or more computer processors, to cause the one or more computer processors to implement the method of any of claims 1 to 24.

Images