Techniques for validation and / or generation of determinations regarding risk of cyberattacks to a system

Abstract

To specify a solution capable of coping with a problem related to a risk of cyberattacks.SOLUTION: The method proposed herein relates to a method for correctness checking and / or determination with respect to the risk of cyberattacks. The method includes receiving a request to perform cyberattacks and invoking a machine-learning agent. Machine-learning agents are designed for generating and performing cyberattacks using generative machine-learning models. The method further includes performing, using the machine-learning agent, a cyberattacks in response to the request, evaluating a result of the performed cyberattacks, and confirming, based on a finding of the evaluating of the result, whether the predetermined determination regarding the risk of the cyberattacks is correct or determining the risk of the cyberattacks based on the finding of the evaluating of the result.SELECTED DRAWING: Figure 3

Description

[Background technology]

[0001] In many fields of technology, it is desirable, or even necessary, to carry out a determination of the risk of cyberattacks on a system during product development. For this purpose, some formalized metrics for threat and risk analysis are used. In the automotive sector, for example, metrics for threat and risk analysis are defined in the standard ISO / SAE 21434. Within the framework of threat and risk analysis, various cyberattacks on a system are run through from start to finish. This initial determination of the threat and risk situation must then be adapted and refined as necessary during the product lifecycle.

[0002] Many traditional approaches to making a determination regarding the risk of a cyberattack on a system involve a test engineer to a significant extent (thus, these approaches could even be described as "manual"). For example, various cyberattack scenarios may be run through by a test engineer in a test environment. Additionally or alternatively, the test engineer may determine the risk of a cyberattack based, at least in part, on subjective criteria. In many cases, part of the determination may be a ranking of how sophisticated or laborious a cyberattack on the system would have to be to be successful. This determination is therefore subject to human judgment error (e.g., distortion / bias or random judgment errors).

[0003] As a result, many of the current state-of-the-art techniques for determining the risk of cyber attacks can be very time-consuming, and the determination can be subject to human error. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] "The car hacker's handbook: a guide for the penetration tester," Craig Smith, 2016 Summary of the Invention [Problem to be solved by the invention]

[0005] This disclosure demonstrates solutions that may address these problems. [Means for solving the problem]

[0006] A method according to a first aspect proposed in the present disclosure relates to a method for verifying the accuracy and / or generating a determination regarding the risk of a cyber-attack on a specific system. The method includes receiving a request to conduct one or more cyber-attacks on the specific system and invoking a machine learning agent. The machine learning agent is designed to access a generative machine learning model trained on a generation of a dataset and, based on the request, to generate and conduct one or more cyber-attacks on the system using the generative machine learning model. The method further includes, in response to the request, conducting one or more cyber-attacks on the system using the machine learning agent, evaluating results of the one or more conducted cyber-attacks, and, based on findings from the step of evaluating the results, verifying whether a predetermined determination regarding the risk of a cyber-attack on the specific system is accurate or generating a determination regarding the risk of a cyber-attack on the specific system based on findings from the step of evaluating the results.

[0007] A method according to a second aspect of the present disclosure is a method for training and / or configuring a machine learning agent to confirm and / or generate a determination regarding the risk of a cyber-attack on a particular system. The method includes receiving a generative machine learning model trained on a generation of a dataset. The method further includes configuring the machine learning agent to access the generative machine learning model trained on the generation of the dataset and, upon request, to generate and execute one or more cyber-attacks on the system using the generative machine learning model.

[0008] The present disclosure, in accordance with a third aspect, includes an environment designed to perform one of the methods according to the first and / or second aspects, which may be a test and / or development environment for a particular system.

[0009] The present disclosure, according to a fourth aspect, includes a computer program comprising instructions which, when executed by a computing unit, cause the computing unit to perform a method for correctness verification and / or generation of a metric determination regarding risk of cyber-attack to a particular system according to the present disclosure.

[0010] The present disclosure, according to a fifth aspect, comprises a computer-readable medium or signal storing and / or embodying a computer program according to the fourth aspect. The techniques of the first to fifth aspects may have one or more of the following advantages in a significant number of implementations.

[0011] First, the disclosed method may reduce the time and / or resource costs for verifying and / or generating a verdict regarding the risk of a cyberattack on a particular system. In a significant number of instances, the verification or generation may even be performed fully automated (i.e., without interaction with a human test engineer). Many state-of-the-art methods involve significant non-automated portions (these methods may even be described as "pen-and-paper" or "manual," even when using computer tools). The disclosed machine learning agent, in a significant number of instances, may generate and conduct cyberattacks on a system (e.g., using a model of the system), evaluate the results of these cyberattacks (e.g., whether, when, and which weaknesses were found), and, accordingly, inspect or generate a verdict (e.g., based on predetermined metrics) regarding the risk of a cyberattack on a particular system. This may reduce the time to perform the method from hours or even days in extreme cases to minutes. Partial automation of the above tasks may also result in time savings (sometimes significant) compared to a significant number of state-of-the-art methods.

[0012] Second, the method of the present disclosure can reduce the impact of human judgment error on determining the risk of a cyberattack on a system. In some current methods, after conducting a more or less formalized test (e.g., executing a cyberattack on a system), a human tester is tasked with determining the risk of the cyberattack based on a predetermined metric. For example, the cyberattack on the system may be determined based on a scale of how sophisticated it must be to succeed, or based on a scale of how complex the means required for it (e.g., accessing a specific interface of the system) are. These determinations are subject to all known (and unknown) errors of human judgment. That is, different testers may have different biases (and, for example, the same scenario may be rated higher or lower based on the scale). Automation of the method based on the present disclosure can significantly reduce the impact of this human judgment error in many situations (because a machine learning agent can perform some or even all steps of the method without human involvement). It is also possible to test multiple systems with the same machine learning agent, which can also contribute to greater comparability of individual results.

[0013] Third, the use of machine learning agents can significantly increase the coverage of possible cyber-attacks in a verdict. The machine learning agent can be equipped with knowledge of many or all known cyber-attacks and, in some circumstances, can automatically conduct and evaluate these cyber-attacks (as described above). This, too, can significantly reduce the risk of skipping important cyber-attacks in the verdict, resulting in a more robust verdict. This process can also increase the security of the verdicted system, as weaknesses can be found more reliably.

[0014] Fourth, in a significant number of instances, weaknesses in the system that become apparent in the course of conducting a cyber attack can be eliminated, at least in part, automatically. Some terms used in this disclosure are explained below.

[0015] A "cyber-attack" (hereinafter sometimes abbreviated to "attack") may be any attempt (which may be at least partially successful or unsuccessful) to obtain, disclose, modify, deactivate, or destroy data, properties, or other elements of a system by gaining access to this system. A cyber-attack may be an unauthorized attempt with the above goals. A cyber-attack may use any auxiliary means to achieve the above goals. In a significant number of instances, a cyber-attack targets a computing unit and / or peripheral components of a computing unit (e.g., an interface or memory). A cyber-attack may be directed at a system designed for or contributing to any form of electronic data processing.

[0016] A "machine learning agent" is a system trained to use a machine learning model (e.g., a generative machine learning model, in particular a large-scale language model - LLM) to find a solution to a problem based on a specific plan (in this disclosure, the generation and execution of a cyber-attack, optionally with further steps for evaluating the results and verifying correctness and / or generating a decision regarding the risk of the cyber-attack to a specific system). To this end, the machine learning agent may comprise not only the generative machine learning model but also further modules (e.g., a planning module, a memory module, and / or tools for solving the problem). The further modules (e.g., tools) may themselves include a machine learning model, but this is not necessary. Problems that may not be satisfactorily solved by a generative machine learning model alone can be solved using a generative machine learning model by integrating it into the machine learning agent. For example, in many cases, a generative machine learning model in the form of a large-scale language model cannot provide a satisfactory answer in response to a request (prompt) to solve a problem. A simple illustrative example is a computational task. A generative machine learning model in the form of a large-scale language model does indeed provide an answer to the requested computational task. However, this answer may often be mathematically incorrect because it is generated by the general language generation mechanism of the generative machine learning model. Here, the machine learning agent can access a conventional computing program, for example, by translating a computing task in a prompt into a request to the computing program's API. The (correct) return value of the computing program can then be incorporated into the text generated by the generative machine learning model in the form of a large-scale language model. Specific examples and possible forms of machine learning agents for correctness verification and / or generation of metric determinations regarding the risk of cyber-attacks to a particular system based on the present disclosure are described further below.

[0017] A "system" in this disclosure may be any technical device designed to solve a specific technical task. A system may include software and / or hardware components (or may consist of one or more of these components). A system may include a computing unit or may be designed to run on a computing unit. A system contains at least one component that may be the target of a cyber-attack. For example, a system may be a computing unit (e.g., a control unit). Further specific examples are described below. [Brief explanation of the drawings]

[0018] [Figure 1] The middle column is a schematic diagram of a method for corrective validation and / or generating a metric judgment regarding the risk of a cyber-attack to a particular system based on the present disclosure, the left column is a schematic diagram of a method for training and / or configuring a machine learning agent for corrective validation and / or generating a metric judgment regarding the risk of a cyber-attack to a particular system, and the right column is a schematic diagram of a method for applying a system tested by the method for corrective validation and / or generating a metric judgment regarding the risk of a cyber-attack. [Figure 2] FIG. 1 is a schematic diagram of the structure of a machine learning agent according to the present disclosure. [Figure 3] FIG. 1 is a schematic diagram of a test and / or development environment in which a machine learning agent according to the present disclosure may be used. DETAILED DESCRIPTION OF THE INVENTION

[0019] FIG. 1, in the middle column (II), schematically illustrates a method for validating and / or generating a judgment regarding the risk of cyber-attack on a particular system based on the present disclosure.

[0020] The method includes step 101 of receiving a request to conduct one or more cyberattacks on a specific system. The request may be generated via a user interface. Additionally or alternatively, the request may be generated by a pre-installed computer system and transmitted to an environment that executes a method for validating and / or generating a metric determination regarding the risk of a cyberattack on a specific system according to the present disclosure. The request may specify the scale of the cyberattack to be conducted or the type of cyberattack to be conducted. In other examples, the request may contain information (for example, only) that a method for validating and / or generating a metric determination regarding the risk of a cyberattack on a specific system should be initiated. The request may additionally or alternatively include information or a description of the specific system and / or the context (environment) of this system. In many examples, the request may include a model (e.g., a simulation model) of the specific system and / or its context (environment).

[0021] The method further includes step 103 of invoking a machine learning agent 10, the machine learning agent designed to access the generative machine learning model trained on the generation of the dataset and, upon request, to generate and execute one or more cyber-attacks on the system using the generative machine learning model.

[0022] In many instances, the generative machine learning model can include a generative infrastructure model. In many instances, the generative machine learning model can include a model for language generation. For example, the generative machine learning model can include a large-scale language model.

[0023] The machine learning model can be built on the basis of an existing (trained) generative machine learning model and can be adapted (e.g., by further training and / or fine-tuning) for use in the methods of the present disclosure using one or more of the techniques described further below. For example, the existing generative machine learning model can include one or more of CodeLlama, Llama, Mistral 7B, Ollama, CoPilot, and / or a language model from the GPT family (e.g., ChatGPT). In other examples, multimodal models such as Gemini and GPT4o can be used. In other examples, a generative machine learning model can be designed and trained from scratch for use in the techniques of the present disclosure (further details about adaptation and training are provided further below).

[0024] The dataset generated by the generative machine learning model may include text data. Alternatively or additionally, the dataset generated by the generative machine learning model may include image data (e.g., individual images or video data). In numerous examples, the dataset generated by the generative machine learning model may include program code, invocation of commands in a programming language, and / or database queries. Additionally or alternatively, conducting one or more cyber-attacks on a system may include execution of program code, invocation of commands in a programming language, and / or database queries.

[0025] Generating a cyber-attack can be performed in a variety of different ways. In numerous examples, a path (attack vector) for a possible cyber-attack can first be identified by a generative machine learning model. For example, a machine learning agent can request the generative machine learning model to identify possible paths for a cyber-attack on a particular system (e.g., considering the characteristics and / or context of the particular system). A possible path can be, for example, access through one of the particular system's interfaces or access to another component of the particular system. In a further step, generating an attack can include designing a strategy for how to attack the particular system along the identified path. This strategy can include one or more types of cyber-attack and / or a goal of the cyber-attack. In the example of access through one of the particular system's interfaces, the strategy can include sending one or more messages through the interface (e.g., by fuzzing the interface) with the goal of modifying the memory contents of the particular system's memory.

[0026] Generating an attack may, in many instances, include generating executable code, commands, or requests to a tool or interface to perform a cyber attack (or portion thereof). This interface may be, for example, an interface to a simulation environment or other test environment. For example, data streams within or into the system may be altered, the contents of memory within the system may be altered, or changes to the system's context (environment) may be performed.

[0027] Additionally or alternatively, attack generation may, in numerous examples, include generating executable code, commands, or requests to tools or interfaces to monitor and / or record the effects of a cyber attack on a system. In one example, it may be monitored or recorded whether data, properties, or other elements of a system are obtained, disclosed, modified, deactivated, or destroyed through access to the system. For example, data streams in or from the system may be monitored, the contents of memory in the system may be monitored, or changes in the system's behavior or properties may be monitored. Possible implementations of the above-described steps of attack generation (or monitoring the success of the attack) are described in conjunction with FIG. 2.

[0028] The method further includes a step 105 of using the machine learning agent to conduct one or more cyber-attacks on the system in response to the request. In many instances, the cyber-attack is performed on a prototype of the particular system. Additionally or alternatively, the cyber-attack may be performed on a model of the particular system. The model may be a simulation model of the particular system (i.e., the attack may also be performed using a simulation, e.g., by intervention in a simulation environment). Additionally or alternatively, the particular system may include, at least in part, hardware components of the system to be tested against which the cyber-attack is performed (i.e., the attack may be performed at least in part within a test bench). The cyber-attack may, in many instances, be performed in a software-in-the-loop environment or a hardware-in-the-loop environment.

[0029] In a number of examples, this may include the execution of all types of cyberattacks known to the machine learning agent, while in other examples, the amount and / or type of cyberattacks executed may be limited based on certain criteria.

[0030] Step 105 of conducting one or more cyber-attacks may be performed iteratively and / or in multiple stages. In this case, the settings of the machine learning agent may be selected differently for different segments or stages (e.g., one setting of the same machine learning agent may be varied, or multiple machine learning agents with different settings may be used for different segments or stages). For example, the machine learning agent may use different tools or datasets for different stages. Alternatively or additionally, the generative machine learning model may be trained and / or configured differently for different segments or stages (e.g., further training or fine-tuning of a differently trained and / or configured machine learning model may be performed using various resources, for example, using a database with descriptions of known cyber-attacks). Furthermore, additionally or alternatively, the machine learning agent may make different requests to the machine learning model for different segments or stages of conducting the attack (e.g., to not consider the requirements, certain capabilities, and knowledge for the attack).

[0031] The method further includes a step 107 of evaluating the results of the one or more conducted cyber-attacks. The evaluation may include determining whether the cyber-attack was successful (e.g., whether data, properties, or other elements of the system were obtained, disclosed, modified, deactivated, or destroyed as a result of the access to the system during the cyber-attack). Additionally or alternatively, the evaluation may include determining whether the cyber-attack was unsuccessful (e.g., whether data, properties, or other elements of the system were not obtained, disclosed, modified, deactivated, and / or destroyed as a result of the access to the system during the cyber-attack). Furthermore, additionally or alternatively, the evaluation may include determining whether certain properties or functionality of the system are no longer provided at all or with the desired quality as a result of the cyber-attack.

[0032] The method further comprises a step 109 of verifying whether a predetermined decision regarding the risk of a cyber-attack on a particular system is correct based on the findings of the step of assessing the results. Alternatively or additionally, the method further comprises a step 109 of generating a decision regarding the risk of a cyber-attack on a particular system based on the findings of the step of assessing the results.

[0033] In a number of examples, the machine learning agent is further designed to perform evaluation of the results. Additionally or alternatively, the machine learning agent may be further designed for verification and / or generation of a verdict. In these examples, evaluation of the results, verification and / or generation of a verdict, or both may be performed using the machine learning agent.

[0034] In a number of examples, the validation may include querying the generative machine learning model whether the determined judgment regarding the risk of a cyber attack on the system accurately reflects the outcome of conducting one or more cyber attacks on the system.

[0035] Alternatively, generating may include requesting the machine learning model to generate a determination of the risk of a cyber-attack on the system based on particular metrics, dependent on the results of conducting one or more cyber-attacks on the system.

[0036] The determination may be based on a metric for determining the risk of a cyber-attack on the system (e.g., "Threat and Risk Assessment - TARA-metric"). In many examples, the metric may include a comprehensive assessment (e.g., based on a particular scale) of the risk of a cyber-attack on the system. The metric may include one or more criteria for determining the risk of a cyber-attack. In many examples, the criteria include one or more of the cunning or experience of the attack / attacker and / or a determination of the amount or complexity of the means used in the attack and / or a determination of the time required for the attack (e.g., for the attack to be successful) and / or the reach of the attack on the system. Some examples of possible criteria are provided below.

[0037] The measure of attack / attacker cunning or experience may be determined in the techniques of this disclosure by a machine learning agent performing attacks in various settings and / or using different machine learning models for different attacks, i.e., a machine learning model retrained and / or configured using greater resources may simulate a more experienced attacker than a machine learning model retrained and / or configured without greater resources (or, for example, a generative machine learning model that was not retrained at all and / or configured for the specific task of this disclosure, e.g., a large-scale language model trained for multimodal language generation).

[0038] In the disclosed technique, the criterion of determining the amount or complexity of means used within an attack can be determined by having a machine learning agent execute the attack using various tools. For example, for a first attack, the machine learning agent may only have access to a system's API. For a further attack, the machine learning agent may have access to diagnostic tools (e.g., at a factory).

[0039] The criteria for determining the time required for an attack (for example, for the attack to be successful) may include the time required for the attack to achieve a predetermined goal (to be successful). In the techniques of the present disclosure, the criterion of the reach of an attack into a system may be determined by the machine learning agent gaining access to different interfaces or other components of the system for different attacks (e.g., access to only the over-the-air interface for a first attack, and access to the physical interface for a second attack).

[0040] The above-introduced (or additional) criteria may be judged based on a specific scale. The machine learning agent may be designed to generate a verdict in response to an evaluation of the results of a cyber-attack (e.g., by appropriately training or configuring the generative machine learning model, by using an additional machine learning model, or by using a module not based on machine learning). For example, a cyber-attack may be performed through a first interface but not through a second interface, successfully modifying data in memory and / or injecting malware. This may result in a specific risk verdict (e.g., a medium or high risk verdict).

[0041] In quite a number of examples, the method may further include receiving a predetermined determination regarding the risk of a cyber-attack on the system, and adapting the predetermined determination regarding the risk of a cyber-attack on the system if the verification reveals that the predetermined determination regarding the risk of a cyber-attack on the particular system is incorrect. The predetermined determination may be based on a metric for determining the risk of a cyber-attack on the system (e.g., a "Threat and Risk Assessment - TARA-metric") as described above. The comparison may include generating a determination as described above and comparing the generated determination with the predetermined determination. If the difference between the generated determination and the predetermined determination exceeds a certain threshold, the predetermined determination may be confirmed to be incorrect. Subsequently, adaptation of the predetermined determination may be performed. Alternatively or additionally, further steps (e.g., further testing for the determination regarding the risk of a cyber-attack on the particular system) may be initiated.

[0042] Further aspects of the machine learning agent of the present disclosure will be discussed below with reference to Figure 2. Figure 2 illustrates schematically the structure of a machine learning agent 10 according to the present disclosure. The machine learning agent 10 may include an interface 22 for receiving requests (as described above). The requests may be user requests. In many instances, the interface 22 may be a network interface (e.g., providing or including an API or web interface). Thus, in many instances, the techniques of this disclosure may be provided as Software-as-a-Service (SaaS).

[0043] The machine learning agent 10 may include a core 24 that includes a generative machine learning model 26. Additionally, the machine learning agent 10 may include a planning module 28 and a memory 30. Additionally or alternatively, the machine learning agent 10 may have access to and / or include one or more tools and / or datasets 32.

[0044] In quite a number of examples, the machine learning agent 10 may further be designed to access one or more tools or datasets 32 that are designed to determine information regarding a cyber-attack on the system and / or characteristics of the system. Alternatively or additionally, the one or more tools or datasets 32 may contribute to the generation or execution of a cyber-attack. One or more cyber-attacks on the system may be generated and executed using one or more tools or datasets in addition thereto.

[0045] The machine learning agent may be designed to use, for example, information acquired using one or more tools or datasets 32 to request the generative machine learning model 26. In many instances, the generative machine learning model 26 may be adapted (e.g., by further training and / or fine-tuning) to add the information acquired using one or more tools or datasets 32 to the generated dataset. One of the one or more tools or datasets 32 may include, for example, a collection (or a portion thereof) of descriptions of known attack types for a particular system. In many instances, generating one or more cyberattacks may include selecting known attack patterns from the collection and generating the cyberattack based on the known attack patterns. For example, various publications may contain collections of attack types that can be used for a particular system. In many instances, the collection of attack types may be domain-specific. For example, in the automotive domain, there is the book "The Car Hacker's Handbook: A Guide for the Penetration Tester" by Craig Smith, published in 2016. This book provides a detailed introduction to cyberattacks on vehicles.

[0046] Based on the request, planning module 28 may be designed to identify one or more steps (and the order in which they must be performed) to fulfill the request. Planning module 28 may employ generative machine learning model 26 for this task. Planning module 28 may be designed to use one or more methods of computer-based reasoning (e.g., chain-of-thoughts or tree-of-thoughts) for these tasks. Planning module 28 may obtain feedback from other modules during the implementation of the methods of the present disclosure.

[0047] The memory 30 may be designed to store the internal protocols of the machine learning agent 10, including all interactions between the machine learning agent 10 and the user, including past ideas, actions, and observations from the surroundings.

[0048] Based on the request and based on steps predetermined by planning module 28, core 24 is designed to perform the tasks of generating and conducting a cyber-attack, and optionally evaluating the results and / or generating and / or determining the correctness of a verdict, by directing requests (prompts) to generative machine learning model 26. As part of this process, core 24 may select or generate requests (prompts) to generative machine learning model 26. The requests (prompts) may include, for example, one or more of identifying a particular attack type or attack path for a cyber-attack, elaborating a strategy for conducting a particular attack type, providing aids (e.g., program code, commands to an interface or tool, or the like) for generating and conducting the attack, providing aids (e.g., program code, commands to an interface or tool, or the like) for evaluating the results of the attack, determining the risk of a cyber-attack to a particular system (e.g., based on predetermined metrics), and / or confirming whether a predetermined verdict regarding the risk of a cyber-attack to a particular system is correct.

[0049] In one specific example, the sequence of requests from the core 24 may look like this: First, the core 24 may send a request to the generative machine learning model 26 to identify an attack path. In response, the generative machine learning model 26 may identify an interface (e.g., a CAN interface) as a possible attack path. In a further step, the core 24 may send a request to the generative machine learning model 26 to develop a strategy for a cyber attack along this attack path. In response, the generative machine learning model 26 may suggest randomly generating interface messages to send to the interface. The core 24 may request the generative machine learning model 26 to write program code (e.g., executable scripts or other programming code in a particular programming language) for generating interface messages and for monitoring the memory of a particular system. The agent 24 may execute the program code in a simulation environment of the particular system to perform the selected attack. In another example, the core 24 may request the generative machine learning model 26 to write the interface messages themselves. In yet another example, the core 24 may request the generative machine learning model 26 to write commands for generating the interface messages. In a further step, core 24 may execute program code or commands to monitor memory. In one example, core 24 may receive feedback that it was able to write to the memory of a particular system during a cyber-attack (i.e., the assessment reveals a successful attack). In response, core 24 may send a query to generative machine learning model 26 asking whether a predetermined assessment regarding the risk of a cyber-attack on a particular system is correct. Generative machine learning model 26 may respond, for example, that a successful attack indicates a higher risk assessment than the predetermined risk assessment. In many instances, the results (possibly along with an automatically generated protocol and / or commentary) may be output via an output interface of machine learning agent 10.

[0050] The tools and / or datasets 32 may include any means that the machine learning agent 10 may use to accomplish (possibly automatically) the steps described herein. In many instances, the tools include interfaces to a test environment for a particular system. Additionally or alternatively, the tools and / or datasets 32 may be devices or systems within the test environment for a particular system. The tools 32 may include, for example, interfaces and / or devices that allow the machine learning agent 10 to intervene in a simulation environment or other test environment (e.g., a hardware-in-the-loop environment) for a particular system.

[0051] Additionally or alternatively, the tool may be a database and / or other data structure that the machine learning agent 10 can access in the course of accomplishing the steps described herein. The database and / or other data structure may contain, for example, information about one or more of: a safety concept for a particular system describing cyber-attack defense measures implemented within the particular system; safety requirements for a particular system or for a class of systems that includes the particular system describing cyber-attack defense measures to be implemented within the particular system; a determination (e.g., based on predetermined metrics) regarding the risk of a cyber-attack on the particular system or similar systems; and / or historical data (e.g., protocols) regarding one or more of the above. Data read from the database and / or other data structure may again be generated by the generative machine learning model 26. Information from the database and / or other data structure may be used in various ways in the techniques of the present disclosure. For example, an attack path or attack type may be selected based on the safety concept or safety requirements of the particular system.

[0052] Below, techniques for training and / or configuring the machine learning agent 10 of the present disclosure are described. The present disclosure also relates to a method for training and / or configuring a machine learning agent 10 to ascertain and / or generate a verdict regarding the risk of a cyber-attack to a particular system. Figure 1, in the left column (I), schematically illustrates a method for training and / or configuring a machine learning agent to ascertain and / or generate a verdict regarding the risk of a cyber-attack to a particular system.

[0053] The method includes step 111 of receiving an initial generative machine learning model trained for the generation of a dataset. The initial generative machine learning model may be trained for the generation of a dataset without being specifically adapted or configured for the method of the present disclosure. In many instances, the initial generative machine learning model is a generative foundation model (e.g., a language model, particularly a large-scale language model, such as the specific models shown above). In other words, the initial generative machine learning model may indeed be trained for training data that includes information described in this disclosure (e.g., about cyberattacks or about determining the risk of cyberattacks to a system). However, in training the initial generative machine learning model, training data that contains this information is not treated differently from training data that does not contain this information. In other words, the initial generative machine learning model is not a generative machine learning model that is task-specifically trained and / or configured for the task of the present disclosure.

[0054] The method further includes configuring 113 the machine learning agent 10 to access the generative machine learning model trained on the generation of the dataset and, based on the request, to generate and conduct one or more cyber-attacks against the system using the generative machine learning model. In many examples, configuring the machine learning agent includes designing the machine learning agent to access one or more tools or datasets 32, the one or more tools or datasets 32 designed to determine information related to and / or characteristics of the cyber-attack against the system and / or contribute to conducting the cyber-attack. The one or more cyber-attacks against the system are generated and conducted using the one or more tools and / or datasets 32 in addition thereto. The tools and / or datasets 32 may include one or more of the tools and / or datasets described above.

[0055] The configuring step 113 may involve further training or fine-tuning of the initial generative machine learning model to generate a generative machine learning model designed for the tasks of the present disclosure (generating and conducting cyber-attacks, and optionally evaluating the results and / or verifying and / or generating verdicts of the cyber-attacks). This training or fine-tuning may be done using training data that includes historical or synthetic data for the respective task. In many instances, the initial generative machine learning model may be further trained or fine-tuned using a collection of descriptions of known attack types or attack vectors for a particular system or a class of systems that includes the particular system. For example, there may be a collection of domain-specific attack types available for a particular system. The initial generative machine learning model may be further trained to generate attack types or attack vectors from this collection.

[0056] Further used in the further training or fine-tuning of the initial generative machine learning model may be one or more of the following information: a safety concept for a particular system describing the cyber-attack defensive measures implemented within the particular system; a safety requirement for a particular system or for a class of systems that includes the particular system describing the cyber-attack defensive measures to be implemented within the particular system; a determination (e.g., based on predetermined metrics) regarding the risk of cyber-attack to the particular system or to similar systems; and / or historical data (e.g., protocols) regarding one or more of the above.

[0057] The previous paragraphs have described steps for further training or fine-tuning the initial generative machine learning model. In other examples, a machine learning agent and its embedded generative machine learning model may be trained from scratch to perform the tasks of the present disclosure.

[0058] The following describes the integration of machine learning agent 10 into a test and / or development environment. Figure 3 illustrates a schematic diagram of a test and / or development environment 300 in which machine learning agent 10 according to the present disclosure may be used.

[0059] The methods of the present disclosure may generally be implemented within the context of a particular system's development process. The present disclosure also relates to an implementation of a technique for determining the accuracy and / or generating a verdict regarding the risk of a cyber-attack on a particular system, as well as to the implementation of a particular system after the implementation of a technique for determining the accuracy and / or generating a verdict regarding the risk of a cyber-attack. In this regard, implementation may involve creating an instance of the particular system, installing software to create an instance of the particular system, or similar implementation steps, depending on the type of the particular system. Specific systems are described further below.

[0060] As previously mentioned, one of the techniques for validating and / or generating a determination regarding the risk of a cyber-attack may be implemented on a model or prototype of a particular system. Accordingly, in this disclosure, these prototypes or models, as well as their subsequent development into a commercially available product, are referred to as a "particular system." It is understood that the form and configuration of the particular system may change during this process (e.g., from a model in a simulation to a commercially available control device).

[0061] The disclosed method may be implemented at any time during the product lifecycle of a product incorporating a particular system, and in many instances may be implemented iteratively (i.e., the method is run multiple times in succession, with a revised determination of, for example, cyber-attack risk at the end of each run).

[0062] The test and / or development environment 300 according to FIG. 3 illustrates not only the machine learning agent 10 but also various other elements. For example, a safety concept 302 for a particular system may be defined. Additionally or alternatively, a determination 304 regarding the risk of a cyber-attack on the particular system may be defined. The latter may, in many instances, be generated and / or adapted based on the techniques of the present disclosure. The initial determination 304 regarding the risk of a cyber-attack on the particular system may, in many instances, be generated in other ways (e.g., manually or by further machine learning models).

[0063] Based on the safety concept 302 and / or the determination 304 regarding the risk of cyber-attack on the specific system, a specific system 312 for implementing the techniques of the present disclosure can be developed 306. The specific system generated in this step can be a model or a prototype (e.g., a "minimum viable product - MVP"). The specific system thus generated can be evaluated by the machine learning agent 10 using the techniques of the present disclosure. Alternatively or additionally, the specific system thus generated can be subjected to one or more steps of a verification and validation method 310 (at the end of which the specific system can be released 308 for implementation). The results of the determination of the present disclosure can be incorporated into the verification and validation method. For example, the configuration and / or safety concept of the specific system can be adapted (e.g., if the specific system is found to have certain vulnerabilities to cyber-attacks, for example by implementing measures to close these vulnerabilities).

[0064] In a significant number of instances, the machine learning agent 10 may be designed to suggest or implement modifications to the safety concept and / or specific systems to close the identified weaknesses (e.g., by modifying the program code of a given system).

[0065] The present disclosure also relates to an environment 300 designed to perform one of the methods according to the present disclosure. In many instances, this environment may be a test and / or development environment for a particular system.

[0066] The present disclosure also relates to methods of use 121 of certain systems following the steps of the techniques of the present disclosure. In the example, a particular system may be designed for the regulation and / or control and / or monitoring of a technical system.

[0067] In examples, the method may include using a particular system to control, regulate, and / or monitor a vehicle function, a robot function, a building automation function, a power tool automation function, and / or a home appliance automation function.

[0068] In one example, a particular system can be designed for placement within a vehicle and / or for controlling vehicle functions (especially driving functions). The vehicle functions can be, for example, functions for autonomous and / or assisted driving. In numerous examples, the particular system can be designed to run on a computer system of a vehicle (e.g., of a vehicle that drives autonomously, highly automated, or assisted). The computer system can be implemented, for example, locally within the vehicle or (at least in part) within a backend communicatively connected to the vehicle. The particular system can include, or be, a control device, for example. In numerous examples, the vehicle can include a computer system with a communication interface that enables communication with the backend. For example, the particular system can run within the backend. In one example, the particular system can be a system for lateral and / or longitudinal steering of the vehicle. In an example, the particular system can receive speed or distance information as input data. Alternatively or additionally, the input data can include relative speed and / or distance between a first vehicle, a second vehicle, a person, and / or a stationary object. Alternatively or additionally, the input data may include variables based on at least one of steering angle, direction angle, yaw rate, sideslip angle, and / or lateral error. Alternatively or additionally, the input data may include information from a network, such as movement information and / or direction information of other vehicles. In examples, this information may be provided by vehicle-to-vehicle communication (V2V communication) or by a backend (V2X communication). Alternatively or additionally, the input data may include steering speed or target setpoints for acceleration and / or braking operations.

[0069] In an example, a particular system may be designed for placement within a drive control or drive unit and / or may be used for regulating motor-related functions (e.g., for motor regulation). In an example, a particular system may be designed for placement within a drive regulation of an electric machine. For example, a state vector of a state-space model may include variables based on at least one of a control signal, an operating mode, or an output regulation of an electric machine.

[0070] The present disclosure also relates to the use of certain systems for the control and / or regulation and / or monitoring of robots. In other examples, the particular system may be located within a robot and / or designed to control robot functions (especially for controlling the robot's locomotion functions). The particular system may be, for example, a system for lateral and / or longitudinal steering of the robot. In numerous examples, the particular system may execute on the robot's computer system. This computer system may be implemented, for example, locally within the robot or (at least in part) within a backend communicatively connected to the robot. In numerous examples, the particular system may execute within the backend. In examples, the particular system may obtain speed or distance information as input data. Alternatively or additionally, the input data may include relative speed and / or distance between the first robot, a person, an additional mobile device, and / or a stationary object. Alternatively or additionally, the input data may include variables based on at least one of steering angle, direction angle, yaw rate, sideslip angle, and / or lateral error. Alternatively or additionally, the input data may include information from a network, for example, movement and / or direction information of other robots, mobile devices, and / or people. In examples, this information may be provided by direct communication or by a backend. In one example, the input vector may include steering speed or target set points for acceleration and / or braking maneuvers.

[0071] The present disclosure also relates to the use of the particular system for controlling and / or regulating and / or monitoring functions in building automation. In one example, a particular system may be designed for placement within a building and / or may be used to control and / or regulate and / or monitor building functions (e.g., for controlling and / or regulating building automation functions), such as functions for regulating room temperature, lighting, and / or safety mechanisms.

[0072] The techniques of this disclosure may be performed automatically in many instances. Further, a computer system designed to execute the method for verifying and / or generating a verdict regarding the risk of a cyber-attack on a particular system is disclosed. Alternatively or additionally, the computer system may be designed to execute the method for training and / or configuring a machine learning agent for verifying and / or generating a verdict regarding the risk of a cyber-attack on a particular system based on the present disclosure. The computer system may include a processor and / or main memory. The computer system may be network-based and / or distributed.

[0073] Further disclosed is a computer program containing instructions that, when executed by a computer system, cause the computer system to perform a method for verifying accuracy and / or generating a verdict regarding the risk of a cyber-attack on a particular system. Alternatively or additionally, the computer program may contain instructions that, when executed by a computer system, cause the computer system to perform a method for training and / or configuring a machine learning agent for verifying accuracy and / or generating a verdict regarding the risk of a cyber-attack on a particular system based on the present disclosure. The computer program may exist, for example, in an interpretable form or a compiled form. The computer program (even in part) may be downloaded into a computer's RAM for execution, for example, as a bit sequence or a byte sequence. [Explanation of symbols]

[0074] 10 Machine Learning Agents 22 Interface 24 cores 26 Generative Machine Learning Models 28 Planning Module 30 memory 32 One or more tools or datasets 100 Step to verify whether a predetermined judgment regarding the risk of cyber-attack on a particular system is correct. 101 receiving a request to conduct one or more cyber attacks on a particular system. 103 Steps to invoke machine learning agent 10 105 Steps for conducting one or more cyber attacks 107 Step of assessing the results of one or more conducted cyber attacks 109. A step of generating a judgment regarding the risk of cyber-attack to a particular system based on the findings of the step of evaluating the results. 111. Receiving an initial generative machine learning model trained on a generation of datasets. 113 configuring the generative machine learning model to generate and conduct one or more cyber-attacks against the system; 121 Use of specific systems 300 Environment 302 Safety Concept 304 Determining the risk of cyber attacks on a particular system 306 Development of specific systems for implementing the techniques of this disclosure 308 Specific System Releases 310 Verification and Validation Methods 312 Specific Systems

Claims

1. 1. A method for validating and / or generating a determination regarding the risk of a cyber attack on a particular system (312), comprising: receiving (101) a request to conduct one or more cyber-attacks on the particular system (312); A step (103) of invoking a machine learning agent (10), wherein the machine learning agent (10) To access a generative machine learning model (26) trained on the generation of the dataset; and a step (103) designed to generate and execute one or more cyber-attacks against the system based on the request using the generative machine learning model; In response to the request, using the machine learning agent (10) to conduct one or more cyber-attacks (105) against the particular system (312); assessing (107) the results of the one or more conducted cyber-attacks; a step (100) of verifying whether a predetermined judgment (304) regarding the risk of a cyber-attack on the specific system (312) is correct based on the findings of the step of evaluating the results, or a step (109) of generating a judgment regarding the risk of a cyber-attack on the specific system (312) based on the findings of the step of evaluating the results; A method comprising:

2. The machine learning agent (10) further comprises: designed to access one or more tools or datasets (32) that are designed to determine information related to and / or characteristics of a cyber-attack on said system and / or that contribute to said execution of said cyber-attack; The method of claim 1 , wherein the one or more cyber attacks on the particular system (312) are generated and carried out using the one or more tools or datasets (32) in addition thereto.

3. the machine learning agent (10) is designed to use information acquired using the one or more tools or datasets (32) to request the generative machine learning model (26); and / or 3. The method of claim 2, wherein the generative machine learning model (26) is adapted to add information acquired using the one or more tools or datasets (32) to the generated dataset.

4. The method of claim 2 or 3, wherein one of the one or more tools or datasets (32) comprises a collection of descriptions of known attack patterns on the system.

5. The method of claim 4 , wherein the generating the one or more cyber-attacks comprises selecting known attack patterns from the collection and generating cyber-attacks based on the known attack patterns.

6. The method of claim 1 , wherein the dataset generated by the generative machine learning model (26) comprises text data and / or image data.

7. the dataset generated by the generative machine learning model (26) comprises command invocations in a programming language and / or database queries; The method of any one of claims 1 to 6, wherein the execution of one or more cyber attacks on the specific system (312) includes invoking the commands in a programming language and / or database queries.

8. receiving the predetermined determination (304) regarding a risk of cyber-attack to the particular system (312); 8. The method of claim 1, further comprising: if the verification reveals that the predetermined judgment (304) regarding the risk of a cyber-attack on the particular system (312) is incorrect, adapting the predetermined judgment (304) regarding the risk of a cyber-attack on the particular system (312).

9. 9. The method of claim 1, wherein the machine learning agent (10) is further designed to perform evaluation of the results, confirmation and / or generation of the judgment, or both, and wherein the evaluation of the results, confirmation and / or generation of the judgment, or both, are performed using the machine learning agent (10).

10. 10. The method of claim 9, wherein the validation includes querying the generative machine learning model (26) whether a determined judgment regarding the risk of a cyber-attack on the system accurately reflects the outcome of the execution of one or more cyber-attacks on the particular system (312).

11. The method of claim 1 , wherein the cyber-attack is carried out on a prototype of the particular system (312) or a model of the particular system (312).

12. 1. A method for training and / or configuring a machine learning agent (10) for validating and / or generating a determination regarding the risk of a cyber attack on a particular system (312), comprising: receiving (111) a generative machine learning model (26) trained for the generation of a dataset; Machine learning agents, accessing the generative machine learning model (26) trained on the generation of the dataset; and configuring (113) the generative machine learning model to generate and execute one or more cyber-attacks against the system based on the request; A method for training and / or configuring, comprising:

13. The configuration of the machine learning agent: designing the machine learning agent (10) to access one or more tools or datasets (32) designed to determine information about and / or characteristics of a cyber-attack on the system and / or that contribute to the creation or execution of the cyber-attack; 13. The method for training and / or configuring of claim 12, comprising the step of: the one or more cyber-attacks on the system are generated and implemented using the one or more tools or datasets (32) in addition thereto, and optionally the design includes further training and / or fine-tuning of the received generative machine learning model.

14. An environment (300) designed to perform one of the methods of claims 1 to 13, optionally being a test and / or development environment for said particular system (312).

15. A computer program containing instructions that, when executed by a computing unit, cause the computing unit to perform a method for verifying correctness and / or generating a determination regarding the risk of a cyber-attack on a particular system (312) according to any one of claims 1 to 13.

Images