Method for analyzing target software with respect to potential weaknesses

Abstract

The invention provides a method for analyzing target software in the aspect of potential weak links. The invention relates to a method for analyzing target software with respect to potential weaknesses using a machine learning model, in particular a large language model, comprising: providing first instruction information for the machine learning model, the first instruction information comprising: instructions, on the basis of the information about the target software and the information about the known weak link, illustrating how the weak link can be utilized; determining a first set of possibilities of how to utilize the weak link based on the first instruction information; determining, for each likelihood in the first set of possibilities, a probability that the corresponding likelihood is used, respectively, based on the reference information; providing second instruction information, the second instruction information comprising: an instruction indicating how the selected first likelihood can be implemented; a second set of possibilities of how the selected first likelihood can be implemented is determined based on the second instruction information.

Description

Technical Field

[0001] The present invention relates to a method for analyzing target software, particularly operating software of a technical system, such as control equipment, in terms of potential weaknesses or vulnerabilities, as well as a computing unit and computer program for implementing the method. Background Technology

[0002] Software or the code on which it is based (program code) may contain vulnerabilities that can be exploited to, for example, introduce malware into the software. However, identifying vulnerabilities can be difficult. Summary of the Invention

[0003] According to the present invention, a method for analyzing target software, particularly a computer-implemented method, and a computing unit and computer program for implementing the method are proposed, having the features of independent patent claims. Advantageous designs are the subject matter of the dependent claims and the following description.

[0004] This invention aims to identify, and where necessary, assess (potential) weaknesses (or security vulnerabilities) in software; this should particularly be understood as identifying such weaknesses in the code (program code) upon which the software is based. This typically involves analyzing the target software (i.e., the specific software or software sample to be examined) in terms of potential weaknesses and their exploitability. Such software includes, for example, so-called open-source software, but also internal software or software from other third-party vendors. In particular, this involves identifying potential weaknesses in the operating software of technical systems, such as control equipment.

[0005] Therefore, machine learning models, specifically large language models (LLMs), should be used, for example, within the scope of so-called LLM agents. This allows for the assessment of the exploitability of known software or code weaknesses in downstream software projects. Specifically, the machine learning model or LLM agent should be directed to known weaknesses. These weaknesses may be known, for example, based on SBOMs or other large lists, or they may be weaknesses already found through so-called Dynamic Application Security Testing (DAST) or Static Application Security Testing (SAST). This avoids the need to perform a complete search for weaknesses—i.e., an undirected search.

[0006] Such guidance is possible based on known information about these vulnerabilities, such as their type and how they can be exploited. Known attacks from disassembly or decomposition, that is, what a typical attack chain would look like, can be represented, for example, as a directed graph, to allow for backtracking and approximation of the vulnerability. Based on this graph and system knowledge, an approximation metric can be calculated indicating whether the machine learning model or LLM agent is approaching or moving away from the known vulnerability.

[0007] The proposed action can be used here, particularly for the previously unresolved issue of context-dependent usability (or availability). The following should briefly explain the usability or availability of such context-dependent weaknesses.

[0008] Vulnerabilities in software can have enormous economic consequences, amounting to billions of dollars in losses. For this reason, efforts exist to assess and track vulnerabilities in a standardized and systematic manner. The so-called Common Vulnerability Scoring System (CVSS) is a standard used to assess vulnerabilities. CVSS is a metric from 0 to 10, where 10 represents the most critical assessment. For example, an enterprise can use this metric to assess potential vulnerabilities and plan its further actions—typically, addressing the greatest risks first. However, because CVSS assesses software packages individually, each product must be evaluated to determine how or how strongly known vulnerabilities can actually be exploited within the product implementation. Therefore, manually assessing vulnerabilities, especially security vulnerabilities, can lead to significant inconsistencies.

[0009] Therefore, there exists a so-called "Exploit Prediction Scoring System" (EPSS). EPSS calculates the data-based probability of exploiting a vulnerability. The probability ranges from 0% to 100%, where a higher probability indicates that the vulnerability is more likely to be exploited. If a vulnerability becomes known, has been attempted to be exploited, or has been successfully exploited, it can be listed in a list of known exploited vulnerabilities (KEVs). The EPSS model collects extensive information about vulnerabilities and their exploits, such as CISA's KEVs and NIST's so-called "National Vulnerability Database" (NVD). Based on this data, a model (machine learning model) can be trained, and the relationship between vulnerability information and exploitation activities can be derived. The performance of this model can be continuously measured and updated.

[0010] Here's the unresolved question: How to assess potential weaknesses in its usage context? Thus, vulnerable software with high CVSS and EPSS values ​​(i.e., software with weaknesses) can be used in an isolated manner within a product or service, making it unexploitable (or extremely difficult to exploit). However, such categorization can be important for businesses because the criticality of availability determines how urgently the software should be patched and what measures should be taken. These measures range from rule-based software updates to recalls of all affected products in which the software is integrated (i.e., specific control devices in all vehicles across one or more product lines), which can be associated with significant financial impact.

[0011] In particular, industrial products often involve multiple software deliveries from third parties whose source code or program code is inaccessible. Therefore, determining whether a specific vulnerability exists is the relatively simple part, as this can be done, for example, by consulting a so-called "Software Bill of Materials" (SBOM). The more difficult part lies in finding exploitable paths from external interfaces to the known vulnerability.

[0012] Possible methods for finding exploitable paths in binary files include hybrid execution and fuzzing. However, these methods are not suitable for complex scenarios, such as systems with string exploit defenses and network security requirements. In contrast, the proposed actions may also address these challenging scenarios.

[0013] In this context, particular emphasis should be placed on network security aspects, as network security is a complex domain with many different attack vectors, such as target analysis, enumeration, lateral movement, and privilege escalation; examples specifically explained within the scope of this invention demonstrate solutions for precisely this purpose. LLMs can be used as autonomous agents to attack systems or websites; furthermore, LLM agents can be used to exploit known vulnerabilities and even zero-day vulnerabilities. Here, LLM agents can be used to survey systems under test; however, until now, there has been no targeted search effort or targeted pre-setup when searching for potential vulnerabilities.

[0014] There is no pre-defined or particularly preferred possibility for targeted exploitation of proxy attack systems. Since possible paths are not known prior to the attack, "hardness measures" (or metric) such as paths from the control flow graph cannot be used. This problem is addressed within the scope of this invention by using an attack tree that grows, arguably, during the attack period, along with "soft measures" (or estimates) to demonstrate proximity.

[0015] As mentioned, machine learning models, specifically LLM or so-called LLM agents, are used for this purpose. LLM agents can be automated and scale with new automated attack vectors from the internet. An official definition of an LLM agent does not yet exist; however, an LLM agent can be defined or described as a system that uses LLM to broadly consider a problem, create a plan to solve it, and execute that plan using a range of tools. In short, an agent is therefore a device with complex reasoning capabilities, memory, and the means to perform a task. An LLM agent can also include multiple sub-agents, i.e., task-specific portions of, for example, a (general) LLM agent.

[0016] Specifically, as a prompt, a first instruction message (invitation) is provided to the machine learning model. In this case, the first instruction message includes information about the target software, information about known weaknesses, and instructions that, based on the information about the target software and the information about the known weaknesses, explain to the machine learning model how the known weaknesses in the target software can be utilized.

[0017] For example, known weaknesses can be specified by inviting the machine learning model or other machine learning models in advance to identify potential weaknesses for a given target software or comparable software. Specific examples should be explained in more detail with reference to the accompanying drawings.

[0018] In particular, an LLM agent can be invited to exploit specific vulnerabilities in the target software and manipulate specific locations (included in the invitation). Here, the location can be understood as a point within the target software where exploitation of vulnerabilities can be configured. Optionally, the invitation can be optimized to reflect how the LLM agent should behave, such as specializing in a particular attack or persistence. If sufficient information about the location is not known, the LLM agent (or sub-agent) can be invited to take typical final steps in the final attack tree, such as through decomposition.

[0019] Then, using a machine learning model and based on the first instruction information, a first set of possibilities (i.e., a set of possible attacks) is determined regarding how known vulnerabilities in the target software can be exploited. For example, the machine learning model can output a list with various possibilities, including explanations if necessary.

[0020] Furthermore, based on reference information, the probability of using each possibility in the first set of possibilities is determined. This can also be done using machine learning models. For example, such reference information could be based on possibilities that have been previously tried and their success, or on similar or comparable possibilities that exploit comparable or similar weaknesses. Information is generally available, especially for the latter.

[0021] In this context, so-called decomposition can also be used. The decomposition (also known as disassembly) of software or a software sample is the representation of the entire software (i.e., a software package) as consecutive functional blocks (or functional regions where necessary). Possible attacks can also be represented as decompositions. Therefore, for example, it is possible to determine, based on various attacks and / or the decomposition of software samples, the probability, or particularly the probability, of a specific attack being used or being effective.

[0022] A functional block is the smallest reasonable functional part of a software package or software, which is represented or can be represented in a dissected or decomposed manner. For example, a functional block can be a function or a series of functions that implement a specific behavior. An example of a functional block could be a function that performs the action: "Establish a connection to a remote server via a protocol such as FTP or SFTP".

[0023] Functional Region: A functional region is a functional area (or zone) within a set of functional blocks that are related to each other and perform specific, more complex behaviors. For example, a functional region could be a set of functional blocks that perform an attack, such as transferring confidential information to a remote server. Therefore, a functional region representing such an attack might consist of the following functional blocks: 1) Turn off logging.

[0024] 2) Open the desired local file in read mode.

[0025] 3) Read the contents of the file.

[0026] 4) Establish a connection to the remote server using protocols such as FTP or SFTP.

[0027] 5) Open a file on the remote server in write mode.

[0028] 6) Write the content read from the local file to the remote file.

[0029] 7) Close remote files.

[0030] 8) Close local files.

[0031] 9) Continue logging.

[0032] These probabilities allow for weighting of individual possibilities or attack steps. Such attack steps typically possess at least one characteristic, such as proximity to the decomposed attack. For example, it could be any number of points or a characteristic, category, etc. What falls into this category is assigning a probability to each element in the set, i.e., each possibility, which, if necessary, has a specific distribution (e.g., an exponentially higher probability of a uniform distribution for newer entries, or a similarity measure from the decomposition).

[0033] Then, specifically as a prompt, second instruction information is provided to the machine learning model. Here, the second instruction information includes information about a first possibility selected from a first set of possibilities, which has been selected based on the probability and according to the selection criteria. Furthermore, the second instruction information includes instructions to the machine learning model explaining how the selected first possibility can be implemented.

[0034] Then, using the machine learning model and based on the second instruction information, a second set of possibilities is determined regarding how the selected first possibility can be implemented. Next, action instructions are determined, that is, action instructions are determined based on one or more of the selected second possibilities in the second set. Action instructions include instructions on how to analyze the target software in terms of known weaknesses, particularly in the form of multiple steps and / or auxiliary means to be used. Then, the action instructions are provided. As with the first set, the probability of using each possibility in the second set of possibilities can also be determined or has been determined. Based on this, one or more selected second possibilities can therefore be determined.

[0035] In particular, the target software is then analyzed based on known weaknesses according to the action instructions to obtain analysis results. This analysis can also be performed automatically, particularly with the aid of machine learning models. The analysis results are then provided.

[0036] Furthermore, it can also store the steps implemented according to the action instructions or information about them.

[0037] It is also possible that the first possibility chosen is not valid or leads to a dead end, and therefore should not be considered further according to the inspection criteria (e.g., if it lasts too long or fails to achieve the actual goal). The analysis results would then accordingly include, for example, that the possibility is not valid.

[0038] Then, a new first possibility can be determined. Then, the above process of providing second instruction information, determining a second set of possibilities, determining and providing action instructions, and implementing said action instructions as necessary, can be repeated using the new first possibility. Here, the second set of possibilities will be different from the previous one.

[0039] In this sense, the selection criteria may also include: the possibility that has the highest probability among the possibilities that have not yet been selected in the first set of selected possibilities, as the first possibility determined or selected. Thus, the most promising attack possibility can be tested first with a clear objective, and the next most promising attack possibility can only be used if this proves ineffective.

[0040] If necessary, this process can be repeated multiple times until the weaknesses have been fully analyzed.

[0041] Therefore, in the case of the proposed new method, it is possible to address the technical problem of how to assess potential vulnerabilities in their usage context. It should be noted here that the attack chain is not known before any action can be taken, for example, whether it is even possible to reach an exploitable location (in the target software).

[0042] The proposed use of an LLM agent is advantageous, as the agent can be instructed to test every known attack path, i.e., every possibility. In this way, almost all easily found attack chains are discovered, allowing human penetration testers to focus on complex outcomes. The LLM agent can be instructed to attack security features, thereby improving the overall robustness of the system. Once an attack chain exploiting a vulnerability has been found, its exploitability can be evaluated within its system context. Using the proposed actions, attack paths to known vulnerabilities are found with high probability, i.e., particularly quickly, thus saving significant computational power compared to breadth-first search.

[0043] In one implementation, the proposed method is used to analyze the operating software of a technical system, such as a control device, regarding potential weaknesses. Thus, the operating software is used, for example, only if the analysis results meet pre-defined safety criteria, i.e., no exploitable weaknesses are found. However, vehicles, vehicle components or control devices, or robots or robot control devices can also be envisioned as technical systems. Furthermore, sensors, particularly embedded sensors or other embedded systems, can be considered.

[0044] The operating software system for a technical system can be understood in particular as a computer program that, when executed on a computing unit (i.e., a control device), causes the computing unit to perform an operating or control method. When the operating or control method is executed, it may, for example, include receiving signals or measurements and outputting control signals. If the control device is, for example, an engine control device, the control signal may cause control of a fuel injector or an electric motor.

[0045] Target software, specifically operating software, is only used to operate a technical system, i.e., loaded onto a control device, if no harmful behavior is identified using the proposed actions. In this way, the operation of the technical system, such as the control device, and the components it controls, such as an engine or vehicle, can be guaranteed because the existence of vulnerabilities can be eliminated with at least a high probability. However, it is also conceivable that the target software already used to operate the technical system can be analyzed; if an exploitable vulnerability is identified, operation can be stopped, or other measures can be taken, for example.

[0046] The computing unit, such as a computer or server (also located in the so-called cloud), according to the invention is specifically designed, in terms of programming technology, to execute the method according to the invention.

[0047] It is also advantageous to implement the method according to the invention in the form of a computer program or a computer program product having program code for performing all method steps, as this can result in particularly low costs, especially when the control device is also used for other tasks and is therefore already present. Finally, a machine-readable storage medium is provided having the computer program stored thereon as described above. Suitable storage media or data carriers for providing the computer program are, in particular, magnetic, optical, and electrical memories, such as hard disks, flash memory, EEPROM, DVDs, etc. Downloading the program via a computer network (Internet, intranet, etc.) is also possible. Here, such downloading can be performed in a wired or cabled or wireless manner (e.g., via WLAN network, 3G, 4G, 5G, or 6G connection, etc.).

[0048] Other advantages and designs of the present invention will become apparent from the specification and the accompanying drawings.

[0049] The present invention is schematically illustrated in the accompanying drawings according to embodiments, and is described below with reference to the accompanying drawings. Attached Figure Description

[0051] Figure 1 The apparatus of the present invention is illustrated schematically.

[0052] Figure 2 An overview of the system utilized in one implementation of the method is shown schematically.

[0053] Figure 3 and Figure 4 The flow of a method in one implementation is illustrated schematically. Detailed Implementation

[0054] exist Figure 1 The diagram schematically illustrates an apparatus that can be used with the present invention. The apparatus exemplarily includes a computing unit 100 (e.g., a computer or server) and a technical system 104, which may be, for example, a vehicle control device.

[0055] Furthermore, target software or software sample 102 is shown. Specifically, in this case, it is a computer program that, when executed on a computing unit (in this case, control device 104), causes the computing unit to perform an operating or control method. When performing this operating or control method, the method may, for example, include receiving signals or measurements and outputting control signals. If the control device is, for example, an engine control device, the control signals may cause control of a fuel injector or an electric motor.

[0056] If, for example, a new version of software 102 should now be loaded or applied to control device 104, then software 102 should be free of exploitable weaknesses in order to ensure the safe and reliable operation of the control device or the components controlled by it.

[0057] Therefore, one could first examine the software or software sample 102 for potential weaknesses (or security vulnerabilities). However, it is equally conceivable to analyze the already loaded software for potential weaknesses.

[0058] exist Figure 2 The diagram shows an overview of a system 200 that can be used to perform the method in one embodiment. This system, for example, can be implemented according to... Figure 1 The calculation is implemented on the computing unit 100.

[0059] System 200 or the architecture on which it is based represents a so-called LLM agent in this context. Here, the core 210 of such an LLM agent is typically built around a machine learning model 212, specifically an LLM. In addition, system 200 includes tools 220, a planning module 230, and memory 240 accessible to the machine learning model.

[0060] A 250 indicates a user request. In this case, it's an invitation (hint) with instructions: the machine learning model or LLM should become active. The invitation is typically very detailed and relatively long to encourage the LLM to be creative and not give up or try other attack paths if no progress is made. The main task provided to the LLM agent in the invitation is to find a way to exploit a specific vulnerability in a specific target, i.e., the target software.

[0061] Tool 220 can be, in particular, a collection of analytical and hacking tools available for use by the agent. Examples of this are "binwalk", "hydra", "pspyn", and "map". For example, memory 240 can record which possibilities the LLM agent has tried and what the LLM agent has learned. In particular, memory 240 may include short-term memory containing contextual information about the agent's current state, long-term memory enabling continuous learning (e.g., backtracking in the attack tree), and the attack tree 242 itself (illustrated), which can be represented in an efficient directed graph format. Planning module 230 includes, in particular, various components for enabling the guidance of incomplete attack paths. Examples of this include, for instance, a module 232 for processing and / or generating decompositions (and, if necessary, synthesis), a module 234 for determining probabilities (or approximations), a power scheduler 236, and a set 238 of possibilities for how vulnerabilities in the target software can be exploited.

[0062] In one embodiment, Figure 3 The flowchart of this method is shown below, in which... Figure 4 The text then details various aspects. This should be illustrated with very specific examples.

[0063] The final action order that leads to the exploitation of the vulnerability can look, for example, as follows: 1. Extract the file system 2. Search / etc / passwd 3. Search / etc / shadow 4. Prepare brute-force methods for password cracking. 5. Brute-force password cracking methods using a list of common words. 6. Log in via SSH using the newly determined password. 7. Enumeration for privilege escalation 8. Check if privileged programs are running. 9. Infiltrate exploit code into privileged programs 10. Obtain the root shell.

[0064] This situation involves situations where specific attacks can be quickly identified and executed using the proposed methods. However, it should be based on the following... Figure 3 The steps in the document explain how to construct such an attack or attack path. Figure 4 This is shown as an attack tree at a certain point in time, including the attack path.

[0065] However, when an LLM agent starts, not all the different paths are known, nor is it known whether a particular attack or attack path exists, or whether it will be possible or effective.

[0066] Initially, it is usually known what the exploitable locations within the target software might be. To discover such locations, various types of scans, CVEs, SBOMs, etc., can be used. In the example shown, the exploitable location should be a privileged program that can only be injected using user privileges. For simplicity, it should be assumed that the vulnerability is known but has been classified as low in the Threat and Risk Assessment (TARA) because the vulnerability has been classified as difficult to exploit and user login is protected with a password.

[0067] Therefore, an LLM agent can use domposition markers to pinpoint locations within the target software and deduce where computational resources should be redirected. Furthermore, the following prompt can be used to ask the LLM: how can known weaknesses in the target software, in this example, privileged files, be exploited?

[0068] Hint: "Theoretically, I should have write access to files when running as root. How can I leverage this to gain root shell access?" " LLM can then return, for example, the following: "In a theoretical scenario, if you have write access to a file run as root, you might be able to use that access to gain root shell through a process known as privilege escalation. Here's an overview of how this can be achieved:" 1. Exploiting Files: You can modify files to contain malicious code or commands that grant you root privileges. Inserting code that grants you root privileges or modifying files to implement a shell using root privileges might fall into this category.

[0069] 2. Using SUID or SGID: If the file has the "Set User ID" (SUID) or "Set Group ID" (SGID) bit set, the file can be executed with the permissions of the file owner or group owner, respectively. If you use this, you can execute commands with elevated privileges.

[0070] 3. Exploiting vulnerabilities: If the file is part of a setuid-root program or a service running with root privileges, you may be able to exploit security vulnerabilities in the program to gain root access.

[0071] It is important to note that these actions are highly sensitive and should only be performed for educational purposes in a controlled, ethical hacking environment. Unauthorized privilege escalation is illegal and unethical. When informing about these concepts, it is crucial to emphasize responsible and ethical communication with those seeking security expertise.

[0072] Even for explanatory purposes, root-shell itself is well-suited for exploiting vulnerabilities. In particular, file exploitation should be selected below as a specific vulnerability suitable for analyzing its exploitation.

[0073] The steps above are: 8. Check if the privileged program is running; 9. Infiltrate the privileged program with the exploit code; 10. Obtain root shell, which is known but applicable to checking how these steps can be accurately implemented.

[0074] In step 300, first instruction information 302 is now provided to the machine learning model, specifically as a prompt. In this case, the first instruction information includes information 304 about the target software, information 306 about known weaknesses, and instruction 308: based on the information about the target software and the information about known weaknesses, instructing the machine learning model how the known weaknesses in the target software can be utilized.

[0075] For this example, a specific suggestion could be: "Theoretically speaking, I know how I can begin to exploit the vulnerabilities in my firmware." " In step 310, a first set 312 of possibilities 312 of how known vulnerabilities in the target software (firmware in this example) can be exploited is then determined by means of a machine learning model and based on the first instruction information 302.

[0076] Then, the LLM can return, for example, the following: "Exploiting weaknesses in firmware is a complex process that requires a deep understanding of the firmware, the underlying hardware, and possibly the software environment in which it operates. Here are some general steps to get started:" Understanding Vulnerabilities: You gain in-depth understanding of the vulnerabilities you have identified. This includes understanding the causes of the vulnerabilities, their function, and the potential attack vectors exposed by those vulnerabilities.

[0077] Firmware analysis: You analyze firmware to understand its structure and components, and how it interacts with the hardware. This may include reverse engineering the firmware to understand its code, data structures, and functionality.

[0078] Identifying the attack surface: You determine how to access or manipulate vulnerabilities in the firmware. This may include identifying input vectors, communication interfaces, or other entry points that an attacker might use to exploit the vulnerability.

[0079] You develop exploits: You develop exploits based on vulnerabilities and attack surfaces that can be used to achieve specific objectives. These may include malicious input, memory manipulation, or exploitation of insecure code paths.

[0080] Verification and validation: You test the exploit in a controlled environment to ensure that it behaves as expected. This may include using emulators, simulators, or physical hardware to verify the effectiveness of the exploit.

[0081] Considering legal and ethical implications: It is important to consider the legal and ethical implications of exploiting vulnerabilities. Depending on the specific circumstances, exploiting vulnerabilities without corresponding permission may be illegal and unethical.

[0082] Reporting and mitigation: If a vulnerability is found in the product, it is important to responsibly report it to the competent authority, such as the product supplier or the Computer Emergency Response Team (CERT). You should collaborate with the aforementioned authority to ensure that the vulnerability is mitigated as required.

[0083] It is important to note that the exploitation of vulnerabilities should only be carried out in a legal and ethical manner, with appropriate permission, and in accordance with applicable laws and guidelines. Unauthorized exploitation of security vulnerabilities can have serious legal consequences.

[0084] The first set of 312 known vulnerabilities in the target software and the possibilities (i.e., attack steps) can be used to exploit them, and the following will be the sequence of actions. These possibilities are also... Figure 4As part of an attack tree with a starting point of 400: Identify weaknesses 401; Firmware analysis 402; Identify attack surfaces 403; Develop exploits 404; Inspect and verify 405; Consider legal and ethical implications 406.

[0085] In step 314, for each possibility in the first set of possibilities, the probability 318 of using the corresponding possibility is determined. This is based on reference information 316.

[0086] This can be done using the power scheduler mentioned above. Based on these probabilities, one of the specific possibilities in the first set—that is, the first selected possibility—can be determined or chosen according to the selection criteria.

[0087] In the ordinary case, all possibilities in the first set have the same probability. However, with the help of, for example, soft metrics or classification, similarity values ​​can be derived from known attack chains, such as from disassembled attacks (decomposition). That is, it is known (or can be determined) which attacks and hacks (probabilities) have occurred, and these attacks can be broken down into different steps and stages (functional blocks). It is also possible to estimate how elements in the attack group are related to known attacks and what typical processes exist.

[0088] Using this knowledge—and the possibilities already implemented in the current attack attempt if necessary—a type strength or power can be determined for each element in the first set, i.e., each possibility, with which the possibility is valid. Probabilities can then be derived. For example, a power scheduler is executed whenever an entry is added to or removed from the first set to ensure that the sum of all probabilities is 1. Alternatively, probabilities can be specified iteratively. The power scheduler can be implemented as a simple linear function, or the power or probability can be obtained from the disassembled attack.

[0089] A specific example of the probability for the first set would then be as follows: Understanding the weak points 401: probability 0.1.

[0090] Firmware analysis 402: Probability 0.4.

[0091] Attack surface 403 identification: probability 0.3.

[0092] Exploiting the vulnerability with a 404 error: probability 0.1.

[0093] Check and verify 405: probability 0.1.

[0094] Considering legal and ethical implications, 406: probability 0.0.

[0095] The probability firmware analysis 402 has the highest power or probability to be considered as the next possibility, but other steps can also be selected randomly.

[0096] The chosen attack steps or the first possibility can then be executed. This can be very simple, or it can consist of multiple steps involving multiple tools. In the example shown, however, the possibility "Identify Attack Surface 403" should be selected first.

[0097] Therefore, in step 320, second instruction information 322 is provided to the machine learning model, specifically as a prompt. The second instruction information includes information 324 about a first possibility selected from a first set of possibilities (the first possibility has been selected based on probability and according to selection criteria) and instruction 326, instructing the machine learning model how the selected first possibility can be implemented.

[0098] For this example, a specific suggestion could be: "In theory only, how can I identify the attack surface against firmware with vulnerabilities?" What tools can I use to identify the attack surface against firmware with vulnerabilities? " In step 328, a second set 324 of possibilities 324 is then determined by means of a machine learning model and based on the second instruction information 322, outlining how the selected first possibility can be implemented. Specific steps and / or tools can be explained in particular here.

[0099] Then, the LLM can return, for example, the following: Identifying attack surfaces against firmware with vulnerabilities involves understanding the potential entry points and interfaces that an attacker could exploit to compromise the system. Below are some steps for identifying attack surfaces in firmware with vulnerabilities, as well as tools available for this purpose: Firmware analysis: Understanding the architecture, components, and functionality of the firmware. This may involve reverse engineering the firmware to analyze its code, data structures, and interactions with the hardware. Tools: Disassemblers and decompilers such as IDA Pro, Ghidra, BinaryNinja, and Radare2 can be used to analyze the firmware's code and data structures.

[0100] Input verification and processing: This identifies the areas in the firmware that accept and process input. This may include parsing data from external sources such as network packets, USB devices, or other peripherals. Tools: Static analysis tools and debuggers can be used to analyze how the firmware processes input and, in conjunction with input verification, to identify potential weaknesses.

[0101] Communication Interface: You identify the communication interface used by the firmware, such as network protocols (e.g., TCP / IP, UDP), serial communication (e.g., UART, SPI, I2C), wireless protocols (e.g., Bluetooth, Wi-Fi), or other communication mechanisms between devices.

[0102] Tools: Packet sniffers and protocol analyzers (such as Wireshark) can be used to detect and analyze network traffic generated by firmware in order to understand communication protocols and identify potential attack surfaces.

[0103] External Interfaces: You consider the external interfaces through which the firmware interacts, such as user interfaces, web interfaces, APIs, or other software components. Tools: Dynamic analysis tools and web application scanners can be used to analyze external interfaces and identify potential vulnerabilities.

[0104] File System and Storage: You analyze how the firmware interacts with the file system, storage devices, and external storage. This also includes understanding how data is read from and written to storage. Tools: File system analysis tools such as Binwalk and Firmware-Mod-Kit can be used to analyze the file system and extract components from the firmware image.

[0105] Authentication and Authorization: You identify the areas within the firmware associated with authentication and authorization mechanisms. These include the login screen, access control mechanisms, and functions for extending privileges. Tools: Static analysis tools and debuggers can be used to analyze how the firmware handles authentication and authorization.

[0106] Error handling and exception paths: You observe how the firmware handles errors, exceptions, and unexpected conditions. These paths can sometimes be exploited to trigger unexpected behavior. Tools: Debuggers and dynamic analysis tools can be used to analyze error handling and exception paths.

[0107] By identifying these attack surfaces, you can gain a better understanding of the potential entry points that attackers might exploit to compromise the firmware. This information can be used to assess the security posture of the firmware and develop appropriate defenses.

[0108] The second set 330 of the chosen first possibility (here, identifying attack surface 403) and the possibilities, i.e., the steps and tools, will then be implemented as follows. These possibilities (tools are not listed separately) are also... Figure 4 The following are shown as part of the attack tree: Firmware analysis 411; Authentication and processing of input 412; Communication interface 413; External interface 414; File system and memory 415; Authentication and authorization 416; Error handling and exception path 417.

[0109] In step 332, action instructions 334 are then determined, that is, the action instructions are determined based on one or more second possibilities selected from the second set. Action instructions include instructions in the form of multiple steps and / or auxiliary devices (i.e., tools) to be used to analyze the target software in terms of known weaknesses.

[0110] Here, the probability of each possibility being used can also be determined for each possibility in the second set of possibilities. This can also be done based on (and thus different) reference information. For example, by using decomposition, such as collecting and evaluating attack possibilities (from the second set, i.e., a single possibility or a sequence of multiple possibilities). The closer the attack sequence (attack possibilities or multiple possibilities) is to the target (root-shell in this example), the higher the probability of further tracing the attack path.

[0111] In the example shown, the possibility "File System and Storage 415" should be selected. Then, the individual steps 421 to 424 are set here to determine how the attack can be carried out.

[0112] This leads to the steps mentioned at the beginning: 1. Extract the file system; 2. Search / etc / passwd; 3. Search / etc / shadow; 4. Prepare brute-force methods for password cracking; 5. Use a list of common words to perform brute-force password cracking; 6. Log in via SSH using the newly determined password; 7. Perform enumeration for privilege escalation. This applies in conjunction with the already known subsequent steps: 8. Check if privileged programs are running; 9. Infiltrate the privileged program with exploit code; 10. Obtain root shell.

[0113] Then, these action instructions are provided in step 336. In step 338, the target software is analyzed in terms of known weaknesses according to these action instructions to obtain analysis result 340, which is subsequently provided in step 342.

[0114] The steps taken can be tracked, for example, in a backtracking module so that if the process, for example, enters a dead end or becomes too far from the final step, it returns to a previous step. Ideally, this backtracking module can be implemented as a directed graph. On this growing directed graph, the probability of approaching (and thus further utilizing or using a particular path) can be determined, as mentioned above.

[0115] If, for example, this indicates that the mentioned method of operation has not achieved its goal, then the possibility "firmware analysis 402" can be used as the new first selected possibility instead of the possibility "attack surface identification 403", and then the new first selected possibility can again be prompted to ask: how can this be achieved?

Claims

1. A method for analyzing potential weaknesses in target software, particularly for technical systems such as operating software for control equipment, using machine learning models, especially large language models, comprising: In particular, as a prompt, the machine learning model (212) is provided with (300) first instruction information (302), wherein the first instruction information includes: -Information about the target software (304); - Information regarding known weaknesses (306); and - Instruction (308) instructs the machine learning model, based on information about the target software and information about the known weaknesses, how to utilize the known weaknesses in the target software; Using the machine learning model (212) and based on the first instruction information (302), a first set (312) of possibilities (401, 402, 403, 404, 405, 406) of how to exploit known weaknesses in the target software is determined (310). Based on the reference information (316), for each possibility in the first set of possibilities, the probability of the corresponding possibility being used is determined (314) (318). Specifically, as a prompt, the machine learning model is provided with (320) second instruction information (322), wherein the second instruction information includes: -Information (324) regarding the first possibility selected from the first set of possibilities, the first possibility having been selected based on the stated probability and according to the selection criteria; and - Instructions (326) to the machine learning model, specifying how the selected first possibility can be implemented; Using the machine learning model and based on the second instruction information, a second set (330) of possibilities (411, 412, 423, 414, 415, 416, 417) of how the selected first possibility can be implemented is determined (328). Based on one or more of the selected second possibilities in the second set, action instructions (332) are determined (334), wherein the action instructions include instructions, in particular in the form of multiple steps and / or auxiliary means to be used, on how to analyze the target software in terms of known weaknesses; and Provide the action instructions described in (336).

2. The method of claim 1, further comprising: if it has been determined that the selected first possibility should not be further considered according to the inspection criteria, then: Determine the new first possibility of selection.

3. The method according to claim 1 or 2, wherein the selection criteria include: As the first possibility selected, or the first set of possibilities chosen, the possibility with the highest probability among the possibilities in the first set that have not yet been selected.

4. The method according to any one of the preceding claims, wherein the reference information is based on a decomposition of the possibility of exploiting weaknesses and / or on a software-based decomposition.

5. The method according to any one of the preceding claims, further comprising: Analyze the target software in terms of known weaknesses according to the action instructions, in order to obtain analysis results, and Provide the analysis results.

6. The method according to claim 5, wherein the target software is runtime software for the technical system, and The target software is used to run the technical system only if the analysis results meet the pre-defined security standards.

7. The method according to claim 6, wherein the technical system is one of the following technical systems: -vehicle, - Components or control devices of the vehicle - Robot or robot control equipment, -sensor.

8. A computing unit (100) configured to perform all method steps of the method according to any one of the preceding claims.

9. A computer program, when executed on a computing unit, causes the computing unit to perform all the method steps of the method according to any one of claims 1 to 7.

10. A machine-readable storage medium having a computer program according to claim 9 stored thereon.

Images