TECHNIQUES FOR DETERMINING CORRECTNESS AND / OR GENERATING AN ASSESSMENT OF THE RISK OF CYBER ATTACKS ON A SYSTEM

Abstract

One method proposed in this disclosure relates to a method for determining the correctness and / or generating an assessment of the risk of cyberattacks on a specific system. The method includes receiving a request to conduct one or more cyberattacks on the specific system and invoking a machine learning agent. The machine learning agent is designed to access a generative machine learning model trained to generate datasets and, based on a request, to generate and execute one or more cyberattacks on a system using the generative machine learning model.The procedure further includes conducting one or more cyberattacks on the system using the machine learning agent in response to the request, evaluating the results of the one or more cyberattacks conducted, and determining, based on a finding of the result evaluation step, whether a predetermined assessment for the risk of cyberattacks on the specific system is correct, or generating, based on a finding of the result evaluation step, an assessment for the risk of cyberattacks on the specific system.

Description

State of the art

[0001] In numerous fields of technology, it is desirable or even mandatory during product development to conduct a cyberattack risk assessment. For this purpose, formalized threat and risk analysis metrics are sometimes used. In the automotive sector, for example, a threat and risk analysis metric is defined in the ISO / SAE 21434 standard. As part of the threat and risk analysis, various cyberattacks on the system are simulated. The initial threat and risk assessment may need to be adjusted and refined during the product lifecycle.

[0002] Some conventional approaches to assessing the risk of cyberattacks on a system involve test engineers to a significant degree (so much so that they could even be described as "manual"). For example, a test engineer might run through various cyberattack scenarios in a test environment. Additionally or alternatively, a test engineer might assess the risk of cyberattacks based on criteria that are at least partially subjective. In some cases, this assessment might include classifying how sophisticated or complex a cyberattack on a system would need to be to succeed. This assessment is therefore subject to errors of human judgment (e.g., biases or random evaluation errors).

[0003] As a result, some state-of-the-art cyberattack risk assessment techniques can be very time-consuming. Additionally, these assessments can be subject to human error.

[0004] The present disclosure aims to provide solutions that can address these problems. Disclosure of the invention

[0005] The method proposed in this disclosure according to the first aspect relates to a method for determining the correctness and / or generating an assessment of the risk of cyberattacks on a specific system. The method includes receiving a request to carry out one or more cyberattacks on the specific system and invoking a machine learning agent. The machine learning agent is designed to access a generative machine learning model trained to generate data sets and, based on a request, to generate and carry out one or more cyberattacks on a system using the generative machine learning model.The procedure further includes conducting one or more cyberattacks on the system using the machine learning agent in response to the request, evaluating the results of the one or more cyberattacks conducted, and determining, based on the findings of the result evaluation step, whether a predetermined assessment for the risk of cyberattacks on the specific system is correct, or generating, based on the findings of the result evaluation step, an assessment for the risk of cyberattacks on the specific system.

[0006] The method proposed in this disclosure, according to a second aspect, is a method for training and / or configuring a machine learning agent to determine correctness and / or generate an assessment of the risk of cyberattacks on a given system. The method includes receiving a generative machine learning model trained to generate data sets. The method further includes configuring a machine learning agent to access the generative machine learning model trained to generate data sets and, based on a request, to generate and execute one or more cyberattacks on a system using the generative machine learning model.

[0007] According to a third aspect, the present disclosure comprises an environment designed to execute one of the methods described in the first and / or second aspects. The environment may be a test and / or development environment for a particular system.

[0008] According to a fourth aspect, the present disclosure comprises a computer program containing instructions which, when executed by a computing unit, cause the computing unit to execute a method for determining the correctness and / or generating a metric assessment for the risk of cyberattacks on a particular system according to one of the present disclosures.

[0009] According to a fifth aspect, the present disclosure comprises a computer-readable medium or signal that stores and / or contains the computer program according to the fourth aspect.

[0010] The techniques of the first to fifth aspects may exhibit one or more of the following advantages in some implementations.

[0011] First, the methods of this disclosure can reduce the time and / or resources required to determine correctness and / or generate an assessment of the risk of cyberattacks on a given system. In some examples, determination or generation can even be fully automated (i.e., without interaction with a human test engineer). Many prior art methods include significant non-automated components (these methods could even be described as "pen-and-paper" or "manual," even though computer tools are used). The machine learning agent of this disclosure can, in some examples, generate and execute the cyberattacks on the system (e.g., using a model of the system) and evaluate the results of these cyberattacks (e.g., by analyzing the impact of the attacks).The system can determine whether and, if so, which vulnerabilities were found, and accordingly assess or generate a risk assessment (e.g., based on a predetermined metric) for cyberattacks on the specific system. This can reduce the execution time from many hours or even days to just a few minutes in extreme cases. Partial automation of the aforementioned tasks can also result in a (potentially significant) time saving compared to some state-of-the-art methods.

[0012] Secondly, the methods described in this disclosure can reduce the influence of human error on the assessment of the risk of cyberattacks on the system. In some prior art methods, a human tester is responsible for evaluating the risk of cyberattacks according to a predetermined metric after performing more or less formalized tests (e.g., executing cyberattacks on the system). For example, a scale can be used to assess how sophisticated a cyberattack on a system must be to be successful or how complex the means required (e.g., access to specific system interfaces) are. These assessments are subject to all known (and unknown) errors of human judgment. Thus, different testers may have different biases (and, for example, rate the same scenario higher or lower on the scale).Automating the procedures according to the present disclosure can reduce the influence of human evaluation errors in some situations (since the machine learning agent can perform certain or even all steps of the procedure without human intervention). It is also possible to test a large number of systems with the same machine learning agent, which can further contribute to greater comparability of the individual results.

[0013] Thirdly, the use of a machine learning agent can increase the coverage of potential cyberattacks in the assessment in some cases. Machine learning agents can be equipped with knowledge of many or all known cyberattacks and can potentially execute and evaluate them automatically (as described above). This, too, can lead to a more robust assessment in some cases, as the risk of overlooking relevant cyberattacks can be reduced. Consequently, the security of the assessed systems can also be increased, since vulnerabilities can be found more reliably.

[0014] Fourthly, in some examples, vulnerabilities in the system that are discovered during the execution of cyberattacks can be at least partially automated and remedied.

[0015] Some terms used in the present revelation are explained below.

[0016] A “cyberattack” (hereafter occasionally abbreviated as “attack”) can be any attempt to obtain, disclose, alter, disable, or destroy data, properties, or other elements of a system by gaining access to the system (the attempt may be at least partially successful or unsuccessful). A cyberattack can be an unauthorized attempt with the aforementioned objectives. A cyberattack can use any means to achieve the above-mentioned objectives. In some examples, a cyberattack targets a computing unit and / or peripheral components of a computing unit (e.g., interfaces or memory). Cyberattacks can be directed against systems designed for or contributing to electronic data processing in any form.

[0017] A "machine learning agent" is a system trained to use a machine learning model (e.g., a generative machine learning model, in particular a Large Language Model - LLM) to find a solution to a problem according to a specific plan (in this disclosure, the generation and execution of cyberattacks and, optionally, the further steps to evaluate a result, as well as to determine its correctness and / or to generate an assessment of the risk of cyberattacks on a specific system). In addition to a generative machine learning model, the machine learning agent may be equipped with other modules (e.g., a planning module, a storage module, and / or tools for solving the problem). These other modules (e.g., the tools) may themselves include machine learning models, but this is not required.By integrating generative machine learning models into a machine learning agent, problems that might not be satisfactorily solvable using the generative model alone can be solved. For example, in many cases, a generative machine learning model in the form of a Large Language Model cannot provide a satisfactory answer to a prompt for solving a problem. A simple illustrative example is arithmetic problems. While a generative machine learning model in the form of a Large Language Model will provide an answer to a query, this answer may not be mathematically correct in many cases because it is generated using the general language generation mechanisms of the generative machine learning model. At this point, a machine learning agent can access a conventional computer program, for example.This is achieved by translating a computational task in a prompt into a request to an API of a computing program. The (correct) return value of the computing program can then be embedded in the text generated by the generative machine learning model in the form of a Large Language Model. Specific examples and possible configurations of machine learning agents for determining correctness and / or generating a metric assessment for the risk of cyberattacks on a specific system according to this disclosure are described below.

[0018] A “system” as defined in this disclosure can be any technical device designed to solve a specific technical problem. A system can include (or consist of) software and / or hardware components. A system can include a computing unit or be designed to run on a computing unit. A system contains at least one component that can be the target of a cyberattack. For example, a system can be a computing unit (e.g., a control unit). Further specific examples are given below. Brief description of the characters Fig. Figure 1 schematically illustrates in the middle column a procedure for determining correctness and / or generating an assessment for the risk of cyberattacks on a specific system according to the present disclosure. Fig. Figure 1 schematically illustrates in the left column a procedure for training and / or configuring a machine learning agent to determine correctness and / or to generate an assessment of the risk of cyberattacks on a specific system. Fig. Figure 1 schematically illustrates in the right column the procedure for applying a system examined using the procedures for determining correctness and / or generating a metric assessment for the risk of cyberattacks. Fig. Figure 2 schematically illustrates a structure of a machine learning agent according to the present disclosure. Fig. Figure 3 schematically illustrates a test and / or development environment in which the machine learning agent can be used according to the present disclosure. Detailed description

[0019] Fig. Figure 1 schematically illustrates in the middle column (II) a procedure for determining correctness and / or generating an assessment for the risk of cyberattacks on a specific system according to the present disclosure.

[0020] The procedure involves receiving a request to conduct one or more cyberattacks on the specified system. The request can be generated via a user interface. Additionally or alternatively, the request can be generated by an upstream computer system and transmitted to the environment that executes the procedure for determining correctness and / or generating a metric assessment for the risk of cyberattacks on a specific system, as disclosed herein. The request can specify the extent to which cyberattacks are to be carried out or the nature of the cyberattacks to be carried out. In other examples, the request may (e.g., only) contain the information that a procedure for determining correctness and / or generating an assessment for the risk of cyberattacks on a specific system should be initiated.The request may also, or alternatively, include information or a description of the specific system and / or its context (environment). In some examples, the request may include a model (e.g., a simulation model) of the specific system and / or its context (environment).

[0021] The procedure further includes calling 103 a machine learning agent 10, wherein the machine learning agent is designed to access a generative machine learning model trained to generate data sets and, based on a request, to generate and carry out one or more cyberattacks on a system using the generative machine learning model.

[0022] In some examples, the generative machine learning model may include a generative foundation model. In other examples, the generative machine learning model may include a language generation model. For example, the generative machine learning model may include a large language model.

[0023] The machine learning model can be based on an existing (trained) generative machine learning model and adapted for use in the methods of this disclosure using one or more of the techniques described below (e.g., through further training and / or fine-tuning). For example, the existing generative machine learning model can include one or more of CodeLlama, Llama, Mistral 7B, Ollama, CoPilot, and / or language models from the GPT family (e.g., ChatGPT). In other examples, multimodal models such as Gemini or GPT4o can be used. In other examples, the generative machine learning model can be designed and trained from scratch for use in the techniques of this disclosure (further details on adaptation and training will be presented below).

[0024] The datasets generated by the generative machine learning model can include text data. Alternatively or additionally, the datasets generated by the generative machine learning model can include image data (e.g., still images or video data). In some examples, the datasets generated by the generative machine learning model can include program code, command calls in a programming language, and / or database queries. Additionally or alternatively, carrying out one or more cyberattacks on the system can involve executing the program code, calling commands in a programming language, and / or database queries.

[0025] The generation of cyberattacks can take many different forms. In some examples, a path (an attack vector) for a potential cyberattack can first be identified by the generative machine learning model. For instance, the machine learning agent can query the generative machine learning model to identify a possible path for a cyberattack on the specific system (e.g., taking into account the properties and / or context of the specific system). A possible path might involve accessing one of the interfaces of the specific system or accessing another component of the specific system. In a further step, the generation of attacks can include developing a strategy for how the specific system can be attacked along the identified path. This strategy can include one or more types of cyberattacks and / or targets of cyberattacks.In the example of access via one of the interfaces of the specific system, the strategy can include injecting one or more messages via the interface (for example, by performing fuzzing of the interface) with the aim of changing a memory content of a memory of the specific system.

[0026] In some examples, generating the attack may involve creating executable code, commands, or requests to a tool or interface that can be used to carry out the cyberattack (or parts thereof). For example, the interface could be that of a simulation or other test environment. This could include, for instance, modifying data streams within or into the system, altering the contents of memory within the system, or changing the context (environment) of the system.

[0027] Additionally or alternatively, in some examples, generating the attack may involve producing executable code, commands, or requests to a tool or interface that can be used to monitor and / or record the effects of a cyberattack on a system. For example, monitoring or recording might involve detecting whether data, properties, or other elements of a system were obtained, disclosed, modified, disabled, or destroyed through access to the system. This could include monitoring data streams into or out of the system, monitoring the contents of memory within the system, or monitoring changes in the system's behavior or properties. Possible implementations of the attack generation steps described above (or the monitoring of their results) are discussed in the context of Fig. 2 described.

[0028] The procedure further includes carrying out one or more cyberattacks on the system using the machine learning agent in response to the request.

[0029] In some examples, the cyberattacks are carried out on a prototype of the system in question. Additionally or alternatively, the cyberattacks can be carried out on a model of the system in question. The model can be a simulation model of the system in question (i.e., the attacks can also be carried out in a simulated environment, e.g., by interfering with a simulation environment). Additionally or alternatively, the system in question can include at least some hardware components of the system under test, against which the cyberattacks are carried out (i.e., the attacks can be carried out at least partially in a test environment). In some examples, the cyberattacks can be carried out in a software-in-the-loop environment or in a hardware-in-the-loop environment.

[0030] In some examples, this may include carrying out all types of cyberattacks known to the machine learning agent. In other examples, the number and / or type of cyberattacks carried out may be limited according to specific criteria.

[0031] The execution of one or more cyberattacks can be iterative and / or carried out in multiple stages. The configuration of the machine learning agent can differ in the various iterations or stages (e.g., the configuration of the same machine learning agent can be modified, or multiple differently configured machine learning agents can be used in the different iterations or stages). For example, the machine learning agent can have access to different tools or datasets in different stages. Alternatively or additionally, the generative machine learning model can be trained and / or configured differently for different iterations or stages (e.g., further training or fine-tuning of the differently trained and / or configured machine learning models can be performed using different resources, such as databases with descriptions of known cyberattacks).Furthermore, or alternatively, the machine learning agent can structure requests to the machine learning model differently for different iterations or stages of carrying out the attacks (e.g., the request not to consider certain skills and knowledge for the attack).

[0032] The procedure further includes evaluating the results of one or more cyberattacks. This evaluation may include determining whether a cyberattack was successful (e.g., whether, during the cyberattack, data, properties, or other elements of the system were obtained, disclosed, altered, disabled, or destroyed through access to the system). Additionally or alternatively, the evaluation may include determining whether a cyberattack was unsuccessful (e.g., whether, during the cyberattack, data, properties, or other elements of the system were not obtained, disclosed, altered, disabled, and / or destroyed through access to the system). Furthermore, the evaluation may additionally or alternatively include determining whether, as a consequence of the cyberattack, certain properties or functionalities of the system are no longer present or no longer function at the desired level.

[0033] The procedure further includes Determine 109, based on a finding from the Evaluate the Outcome step, whether a predetermined assessment of the risk of cyberattacks on the specified system is correct. Alternatively or additionally, the procedure further includes Generate 109, based on a finding from the Evaluate the Outcome step, an assessment of the risk of cyberattacks on the specified system.

[0034] In some examples, the machine learning agent is further configured to perform the evaluation of a result. Additionally or alternatively, the machine learning agent can be further configured to determine and / or generate the evaluation. In these examples, the evaluation of a result, the determination and / or generation of the evaluation, or both, can be performed using the machine learning agent.

[0035] In some examples, determining whether a particular assessment of the risk of cyberattacks on the system accurately reflects the results of conducting one or more cyberattacks on the system may involve querying the generative machine learning model.

[0036] Alternatively, the generation process can involve a request to the machine learning model to generate an assessment of the risk of cyberattacks on the system according to a specific metric, depending on the results of carrying out one or more cyberattacks on the system.

[0037] The assessment can be based on a metric for evaluating the risk of cyberattacks on the system (e.g., a Threat and Risk Assessment - TARA metric). In some examples, the metric may provide a global assessment of the risk of cyberattacks on the system (for example, on a specific scale). The metric may include one or more criteria for evaluating the risk of cyberattacks. In some examples, the criteria include one or more aspects related to the sophistication or experience of the attack / attacker, an assessment of the quantity or complexity of the resources used in the attack, an assessment of the time required for an attack (e.g., for a successful attack), and / or the attacker's level of access to the system. Some examples of possible criteria are presented below.

[0038] The sophistication or experience of the attack / attacker can be assessed in the techniques of this disclosure by having the machine learning agent execute attacks with different configurations and / or use different machine learning models for different attacks. Thus, a machine learning model retrained and / or configured using more extensive resources can simulate a more experienced attacker than a machine learning model retrained and / or configured without using more extensive resources (or, for example, a generative machine learning model that has not been retrained at all and / or configured for the specific tasks of this disclosure, such as a Large Language Model trained for multimodal language generation).

[0039] The criterion for evaluating the quantity or complexity of the means used in the attack can be assessed in the techniques of this disclosure by having the machine learning agent conduct attacks using different tools. For example, for an initial attack, the machine learning agent might only have access to an API of the system. For a subsequent attack, the machine learning agent might have access to a diagnostic tool (e.g., in a repair shop).

[0040] The criterion for evaluating the time required for an attack (e.g., for a successful attack) can include the time needed until the attack reaches a predefined goal (is successful).

[0041] The criterion of the attacker's degree of access to the system can be assessed in the techniques of the present disclosure by granting the machine learning agent access to different interfaces or other components of the system for different attacks (e.g., for a first attack, access only to an over-the-air interface, and for a second attack, access to a physical interface).

[0042] The criteria presented above (or others) can be evaluated on a specific scale. The machine learning agent can be configured (e.g., through appropriate training or configuration of the generative machine learning model, or by using an additional machine learning model, or by using a module not based on machine learning) to generate an evaluation in response to the results of the cyberattacks. For example, a cyberattack carried out via a first interface, but not via a second interface, might successfully modify data in a memory and / or inject malware. This could lead to a specific risk assessment (e.g., a medium or high risk rating).

[0043] In some examples, the process may further include receiving the predetermined cyberattack risk assessment for the system and adjusting it if it is found to be incorrect. The predetermined assessment may be based on a cyberattack risk assessment metric (e.g., a Threat and Risk Assessment - TARA metric) as described above. The comparison may involve generating a new assessment as described above and comparing the generated and predetermined assessments. If the discrepancy between the generated and predetermined assessments exceeds a certain threshold, an inaccuracy of the predetermined assessment may be identified. An adjustment of the predetermined assessment may then be performed.Alternatively or additionally, further steps can be initiated (e.g., further tests to assess the risk of cyberattacks on the specific system).

[0044] The following sections will be based on… Fig. Two further aspects of the machine learning agent of the present disclosure are discussed. Fig. Figure 2 schematically illustrates a structure of a machine learning agent 10 according to the present disclosure.

[0045] The machine learning agent 10 can include an interface 22 for receiving requests (as described above). The requests can be user requests. In some examples, the interface 22 can be a network interface (which, for example, provides or contains an API or a web interface). In this way, it may be possible in some examples to offer the techniques of this disclosure as SaaS (Software-as-a-Service).

[0046] The machine learning agent 10 can include a core 24, which comprises a generative machine learning model 26. Additionally, the machine learning agent 10 can include a planning module 28 and memory 30. Additionally or alternatively, the machine learning agent 10 can access and / or include one or more tools and / or datasets 32.

[0047] In some examples, the machine learning agent 10 may be further configured to access one or more tools or datasets 32 designed to determine information regarding cyberattacks on the system and / or system properties. Alternatively or additionally, the one or more tools or datasets 32 may contribute to generating or executing the cyberattacks. The one or more cyberattacks on the system may also be generated and executed using the one or more tools or datasets.

[0048] For example, the machine learning agent may be designed to use information obtained through one or more tools or datasets 32 to query the generative machine learning model 26. In some examples, the generative machine learning model 26 may be adapted (e.g., through further learning and / or fine-tuning) to incorporate information obtained through one or more tools or datasets 32 into the generated datasets. For example, one of the one or more tools or datasets 32 may comprise a collection of descriptions of known attack types on the specific system (or parts thereof). In some examples, generating the one or more cyberattacks may involve selecting known attack patterns from the collection and generating a cyberattack according to the known attack patterns.For example, collections of attack types for specific systems may be available in various publications. These collections may be domain-specific in some cases. For instance, in the automotive sector, there is the book "The Car Hacker's Handbook: A Guide for the Penetration Tester" by Craig Smith, published in 2016. This book details cyberattacks on vehicles.

[0049] Planning Module 28 can be designed to identify, based on a query, one or more steps that must be performed to answer the query (as well as their sequence). Planning Module 28 can utilize the generative machine learning model 26 for this task. Planning Module 28 can be designed to employ one or more computer-based reasoning methods (e.g., chain-of-thought or tree-of-thought) for these tasks. Planning Module 28 can receive feedback from the other modules while performing the procedures of this disclosure.

[0050] Memory 30 can be designed to store internal logs of the machine learning agent 10, including past thoughts, actions and observations from the environment, including all interactions between the machine learning agent 10 and a user.

[0051] Core 24 is designed to send prompts to the generative machine learning model 26 based on the request and according to the steps specified by Planning Module 28. These prompts are used to generate and execute cyberattacks and, optionally, to evaluate the results and / or generate and / or determine the correctness of the evaluations. As part of this process, Core 24 can select or generate the prompts for the generative machine learning model 26. For example, a prompt might include one or more of the following: identifying a specific attack type or attack path for a cyberattack; developing a strategy for executing a specific attack type; providing resources (e.g., program code, interface commands, or tools) for generating and executing the attacks; or providing resources (e.g.,Program code, commands at interfaces or tools or similar) for evaluating the results of the attacks, assessing the risk of cyberattacks on the specific system (e.g. according to a predetermined metric) and / or determining whether a predetermined assessment of the risk of cyberattacks on the specific system is correct.

[0052] In a concrete example, a sequence of requests from kernel 24 could look like this. First, kernel 24 can send a request to generative machine learning model 26 to identify an attack path. In response, generative machine learning model 26 can identify an interface (e.g., a CAN interface) as a possible attack path. In a further step, kernel 24 can send a request to generative machine learning model 26 to design a strategy for a cyberattack along this attack path. In response, generative machine learning model 26 can suggest randomly generating interface messages and feeding them into the interface. Kernel 24 can request that generative machine learning model 26 write program code to generate the interface messages and to monitor a memory of the specific system (e.g.,(an executable script or other programming code in a specific programming language). Agent 24 can execute the program code in a simulation environment of the specific system to carry out the selected attack. In other examples, Core 24 can request the generative machine learning model 26 to write the interface messages itself. In still other examples, Core 24 can request commands from the generative machine learning model 26 to write commands for generating the interface messages. In a further step, Core 24 can execute the program code or commands to monitor the memory. In one example, Core 24 can receive feedback that it was possible to write to a memory of the specific system during the execution of a cyberattack (i.e., the evaluation indicates a successful attack).In response, the core 24 can send a query to the generative machine learning model 26 to determine whether a predetermined risk assessment for cyberattacks on the specific system is correct. For example, the generative machine learning model 26 might return that the successful attack indicates a higher risk assessment than a predetermined risk assessment. In some examples, the result (possibly with automatically generated logs and / or explanations) can be output via an output interface of the machine learning agent 10.

[0053] The tools and / or datasets 32 can include all means that the machine learning agent 10 can use to (possibly automatically) execute the steps described herein. In some examples, the tools include interfaces to a test environment of the specific system. Additionally or alternatively, the tools and / or datasets 32 can be devices or systems within a test environment of the specific system. For example, the tools 32 can include interfaces and / or devices that allow the machine learning agent 10 to interact with a simulation environment of the specific system or another test environment (e.g., a hardware-in-the-loop environment).

[0054] Additionally or alternatively, the tools can be databases and / or other data structures that the machine learning agent 10 can access while processing the steps described herein. For example, the databases and / or other data structures can contain information about one or more of the following: a security concept for the specific system, describing measures to defend against cyberattacks implemented in that system; security requirements for the specific system or a class of systems that includes the specific system, describing measures to defend against cyberattacks to be implemented in that system; assessments (e.g., according to predetermined metrics) of the risk of cyberattacks on the specific system or similar systems; and / or historical data (e.g., logs) relating to one or more of the aforementioned.A query of data from the databases and / or other data structures can, in turn, be generated by the generative machine learning model 26. The information from the databases and / or other data structures can be used in various ways in the techniques of this disclosure. For example, an attack path or attack type can be selected based on a security concept of the specific system or based on security requirements.

[0055] Techniques for training and / or configuring the machine learning agent 10 of the present disclosure are described below.

[0056] The present disclosure also relates to a method for training and / or configuring a machine learning agent 10 to determine correctness and / or to generate a risk of cyberattacks on a particular system. Fig. Figure 1 schematically illustrates in the left column (I) a procedure for training and / or configuring a machine learning agent to determine correctness and / or to generate a risk assessment for cyberattacks on a specific system.

[0057] The procedure comprises receiving an initial generative machine learning model trained to generate datasets. The initial generative machine learning model can be trained to generate the datasets without having been specifically adapted or configured for the procedures of this disclosure. In some examples, the initial generative machine learning model is a generative foundation model (e.g., a language model, in particular a large language model, such as the specific models mentioned above). In other words, the initial generative machine learning model may be trained on training data that includes the information described in this disclosure (e.g., on cyberattacks or assessments of the risk of cyberattacks on systems).However, training data containing this information was not treated differently in the training of the initial generative machine learning model than training data that did not contain this information. In other words, the initial generative machine learning model is not a task-specific trained and / or configured generative machine learning model with respect to the tasks of this disclosure.

[0058] The procedure further includes configuring a machine learning agent to access the generative machine learning model, which is trained to generate datasets and, based on a request, to generate and execute one or more cyberattacks against a system using the generative machine learning model. In some examples, this includes configuring the machine learning agent to access one or more tools and / or datasets designed to determine information regarding cyberattacks on the system and / or system properties, and / or contributing to the execution of the cyberattacks. The one or more cyberattacks against the system are additionally generated and executed using the one or more tools and / or datasets.The tools and / or data sets 32 can include one or more of the tools and / or data sets described above.

[0059] Configuring 113 may include further training or fine-tuning of the initial generative machine learning model to produce a generative machine learning model designed for the tasks of this disclosure (generating and executing the cyberattacks and optionally evaluating the results of the cyberattacks and / or determining and / or generating the evaluation). The training or fine-tuning may be performed using training data that includes historical or synthesized data pertaining to the respective tasks. In some examples, the further training or fine-tuning of the initial generative machine learning model may involve using a collection of descriptions of known attack types or attack paths against the specific system or a class of systems encompassing the specific system. For example, there may be domain-specific collections of attack types available for certain systems.The initial generative machine learning model can be further trained to generate attack types or attack paths from the collection.

[0060] In the course of further training or fine-tuning of the initial generative machine learning model, one or more pieces of information can be used relating to one or more of a security concept for the specific system, describing measures to defend against cyberattacks implemented in the specific system; security requirements for the specific system or a class of systems that includes the specific system, describing measures to defend against cyberattacks to be implemented in the specific system; assessments (e.g., according to predetermined metrics) for the risk of cyberattacks on the specific system or similar systems; and / or historical data (e.g., logs) relating to one or more of the aforementioned.

[0061] The preceding sections described measures for further training or fine-tuning an initial generative machine learning model. In other examples, the machine learning agent and the generative machine learning model it contains can also be trained from scratch to perform the tasks of this disclosure.

[0062] The following describes the embedding of the machine learning agent 10 into a test and / or development environment. Fig. Figure 3 schematically illustrates a test and / or development environment 300 in which the machine learning agent 10 can be used according to the present disclosure.

[0063] The procedures described in this disclosure can generally be carried out within the framework of a development process of the specific system.

[0064] This disclosure also relates to performing one of the techniques for determining the correctness and / or generating a cyberattack risk assessment for a specific system, and implementing the specific system after performing the techniques for determining correctness and / or generating a cyberattack risk assessment. Depending on the nature of the specific system, the implementation may involve creating an instance of the specific system, installing software to create an instance of the specific system, or similar implementation steps. Specific systems are described below.

[0065] As already mentioned, one of the techniques for determining correctness and / or generating a cyberattack risk assessment can be performed on models or prototypes of the specific system. Therefore, in this disclosure, the term "specific system" refers to both these prototypes or models and subsequent development stages up to a marketable product. It is understood that both the form and configuration of the specific system can change during this process (e.g., from a model in a simulation to a marketable control unit).

[0066] The procedures of this disclosure can be carried out at any stage of the product lifecycle of a product containing the specified system. In some examples, the procedures of this disclosure can be carried out iteratively (i.e., the procedures are executed multiple times in succession, with, for example, an assessment for the risk of cyberattacks being modified at the end of each execution).

[0067] One test and / or development environment 300 according to Fig. Figure 3 shows 10 different elements besides the machine learning agent.

[0068] For example, a security concept 302 may be defined for the specific system. Additionally and alternatively, an assessment of the risk of cyberattacks on the specific system 304 may be defined. The latter may, in some examples, be generated and / or adapted according to the techniques of this disclosure. An initial assessment of the risk of cyberattacks on the specific system 304 may, in some examples, be generated by other means (e.g., manually or by another machine learning model).

[0069] Based on the security concept 302 and / or the assessment of the risk of cyberattacks on the specific system 304, a specific system 312 can be developed for carrying out the techniques of this disclosure 306. The specific system created in this step can be a model or prototype (e.g., a "Minimum Viable Product - MVP"). The specific system thus created can be evaluated by the machine learning agent 10 using the techniques of this disclosure. Alternatively or additionally, the specific system thus created can be subjected to one or more steps of a verification and validation procedure 310 (at the end of which the release 308 of the specific system for implementation may be granted). The results of the evaluations of this disclosure can be incorporated into the verification and validation procedures.For example, a configuration and / or a security concept can be adapted for the specific system (e.g., if it turns out that the specific system has certain vulnerabilities to cyberattacks, e.g., by implementing measures to close these vulnerabilities).

[0070] In some examples, the machine learning agent 10 may be designed to suggest or implement changes to the security concept and / or the specific system in order to close discovered vulnerabilities (e.g., by modifying program code of the predetermined system).

[0071] The present disclosure also relates to an environment 300 designed to execute one of the methods according to the present disclosure. In some examples, the environment may be a test and / or development environment for the specific system.

[0072] The present disclosure also relates to a method of using 121 a certain system according to the steps of the techniques of the present disclosure.

[0073] In examples, the specific system may be designed for the regulation and / or control and / or monitoring of a technical system.

[0074] Examples of the method include using the specified system to control, regulate and / or monitor a vehicle function, a robot function, a building automation function, a power tool automation function, and / or a household appliance automation function.

[0075] In one example, the specific system may be designed for installation in a vehicle and / or for controlling a vehicle function (particularly a driving function). For example, the vehicle function may be an autonomous and / or assisted driving function. In some examples, the specific system may be designed to run on a vehicle's computer system (e.g., in an autonomous, highly automated, or assisted driving vehicle). For example, the computer system may be implemented locally in the vehicle or (at least partially) in a backend that communicates with the vehicle. For example, the specific system may include or be a control unit. In some examples, the vehicle may include a computer system with a communication interface that enables communication with a backend.For example, the specific system can be executed in this backend. In one example, the specific system could be a system for lateral and / or longitudinal guidance of the vehicle. In other examples, the specific system could receive speed information or distance information as input data. Alternatively or additionally, the input data could include a relative speed and / or a distance between a first vehicle, a second vehicle, a person, and / or a stationary object. Alternatively or additionally, the input data could include variables based on at least one of a steering angle, an attitude angle, a yaw rate, a slip angle, and / or a lateral error. Alternatively or additionally, the input data could include information from a network, such as motion and / or direction information from other vehicles.In examples, this information can be provided via vehicle-to-vehicle communication (V2V communication) or via a backend (V2X communication). Alternatively or additionally, the input data can include steering speed or target values ​​for acceleration and / or braking.

[0076] In some examples, the specific system may be designed for integration into a drive control system or drive unit and / or serve to control a motor-related function (particularly motor control). In other examples, the specific system may be integrated into the drive control system of an electric machine. For instance, the state vector of the state-space model may contain variables that are based on at least one control signal, an operating mode, or a power setting of the electric machine.

[0077] The present disclosure also relates to the use of the specified system for controlling and / or regulating and / or monitoring a robot.

[0078] In other examples, the specific system may be located within a robot and / or designed to control a robot function (particularly a robot motion function). For example, the specific system may be a system for lateral and / or longitudinal guidance of the robot. In some examples, the specific system may run on a robot's computer system. For example, the computer system may be implemented locally within the robot or (at least partially) in a backend that communicates with the robot. In some examples, the specific system may run in a backend. In some examples, the specific system may receive velocity or distance information as input data.Alternatively or additionally, the input data can include a relative velocity and / or distance between a first robot, a human, another mobile device, and / or a stationary object. Alternatively or additionally, the input data can include variables based on at least one steering angle, orientation angle, yaw rate, slip angle, and / or lateral error. Alternatively or additionally, the input data can include information from a network, such as motion and / or direction information from other robots, mobile devices, and / or humans. In examples, this information can be provided via direct communication or via a backend. In one example, an input vector can include a steering velocity or target values ​​for acceleration and / or braking.

[0079] The present disclosure also relates to the use of the specified system for controlling and / or regulating and / or monitoring functions in building automation.

[0080] In one example, the specific system may be designed for installation within a building and / or serve to control, regulate, and / or monitor building functions (especially to control and / or regulate building automation functions). For example, the building function may be a function for regulating room temperature, lighting, and / or security equipment.

[0081] The techniques described in this disclosure can be automated in some examples.

[0082] Disclosure further relates to a computer system designed to execute the procedures for determining correctness and / or generating a risk assessment for cyberattacks on a given system. Alternatively or additionally, the computer system may be designed to execute the procedures for training and / or configuring a machine learning agent to determine correctness and / or generate a risk assessment for cyberattacks on a given system as disclosed herein. The computer system may include a processor and / or memory. The computer system may be network-based and / or distributed.

[0083] Disclosure further includes a computer program containing instructions that, when executed by a computer system, cause the computer system to perform the procedure for determining correctness and / or generating a risk assessment for cyberattacks on a specific system. Alternatively or additionally, the computer program may contain instructions that, when executed by a computer system, cause the computer system to perform the procedures for training and / or configuring a machine learning agent to determine correctness and / or generate a risk assessment for cyberattacks on a specific system, as disclosed herein. The computer program may be in interpretable or compiled form, for example. It may be loaded (even partially) into a computer's RAM for execution, for example, as a sequence of bits or bytes. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited non-patent literature

[0000] "The car hacker's handbook: a guide for the penetration tester" by Craig Smith from 2016

[0048]

Claims

[1] Method for determining the correctness and / or generating an assessment of the risk of cyberattacks on a particular system (312), wherein the method comprises: Receiving (101) a request to carry out one or more cyberattacks on the specified system (312); Calling (103) a machine learning agent (10), wherein the machine learning agent (10) is designed to: to access a generative machine learning model (26) trained to generate data sets, and to generate and carry out one or more cyberattacks on a system based on a request using the generative machine learning model; Carrying out (105) one or more cyberattacks on the specified system (312) using the machine learning agent (10) in response to the request; Evaluate (107) the results of the one or more cyberattacks carried out; and Determine (100), based on a finding of the step of evaluating the results, whether a predetermined assessment (304) for the risk of cyberattacks on the specified system (312) is correct, or generate (109), based on a finding of the step of evaluating the results, an assessment for the risk of cyberattacks on the specified system (312). [2] Method according to claim 1, wherein the machine learning agent (10) is further designed to: to access one or more tools or data sets (32) designed to obtain information relating to cyberattacks on the system and / or system features and / or to contribute to carrying out the cyberattacks, and wherein the one or more cyberattacks on the specified system (312) are additionally created and carried out using the one or more tools or data sets (32). [3] Method according to claim 2, wherein the machine learning agent (10) is designed to use information obtained by means of one or more tools or datasets (32) to query the generative machine learning model (26) and / or wherein the generative machine learning model (26) is adapted to incorporate information obtained by means of the one or more tools or datasets (32) into the generated datasets. [4] Method according to one of claims 2 to 3, wherein one of the one or more tools or data sets (32) comprises a collection of descriptions of known attack patterns on the system. [5] Method according to claim 4, wherein the generation of one or more cyberattacks comprises selecting known attack patterns from the collection and generating a cyberattack according to the known attack patterns. [6] Method according to any one of claims 1 to 5, wherein the data sets generated by the generative machine learning model (26) comprise text data and / or image data. [7] Method according to any one of claims 1 to 6, wherein the data sets generated by the generative machine learning model (26) comprise calls to commands in a programming language and / or database queries, wherein carrying out one or more cyberattacks on the specific system (312) comprises calling the commands in a programming language and / or database queries. [8] Method according to any one of claims 1 to 7, further comprising: Receiving the predetermined assessment (304) for the risk of cyberattacks on the specified system (312); and Adjusting the predetermined assessment (304) for the risk of cyberattacks on the specified system (312) if the determination reveals that the predetermined assessment (304) for the risk of cyberattacks on the specified system (312) is incorrect. [9] Method according to any one of claims 1 to 8, wherein the machine learning agent (10) is further designed to perform the evaluation of a result, the detection and / or generation of the evaluation, or both, and wherein the evaluation of a result, the detection and / or generation of the evaluation, or both, are performed using the machine learning agent (10). [10] Method according to claim 9, wherein the determination comprises a query to the generative machine learning model (26) to determine whether a particular assessment for the risk of cyberattacks on the system correctly reflects the results of carrying out one or more cyberattacks on the particular system (312). [11] Method according to any one of claims 1 to 6, wherein the cyberattacks are carried out on a prototype of the specific system (312) or a model of the specific system (312). [12] Methods for training and / or configuring a machine learning agent (10) to determine correctness and / or to generate a risk of cyberattacks on a particular system (312), comprising: Receiving (111) a generative machine learning model (26) trained to generate data sets; Configuring (113) a machine learning agent to: Accessing the generative machine learning model (26) which is trained to generate data sets, and Based on a request, using the generative machine learning model, to generate and carry out one or more cyberattacks on a system. [13] Method for training and / or configuring according to claim 12, wherein configuring the machine learning agent comprises: Designing the machine learning agent (10) to access one or more tools or datasets (32) designed to identify information regarding cyberattacks on the system and / or system properties and / or contributing to the generation or execution of the cyberattacks, and wherein the one or more cyberattacks on the system are additionally generated and carried out using the one or more tools or data sets (32), optionally wherein the design includes further training and / or fine-tuning of the received generative machine learning model. [14] Environment (300) designed to perform one of the methods of claims 1 to 13, optionally wherein the environment is a test and / or development environment for the specific system (312). [15] Computer program containing instructions which, when executed by a computing unit, cause the computing unit to execute a method for determining correctness and / or generating an assessment of the risk of cyberattacks on a particular system (312) according to any one of claims 1 to 13.

Images