Methods for controlling technical systems

Abstract

Methods for controlling a technical system are disclosed according to various different embodiments. [Solution] The method provides a textual description of the behavior of a technical system to a large language model (LLM) by instruction, the instruction to produce an output comprising a presentation of multiple states of the technical system and a presentation of conditions for multiple states relating to the input parameters of the technical system, the presentation presents which of the multiple states the technical system is in under what configuration of input quantities; check whether the output of the LLM meets criteria for completeness and / or integrity, and if the output meets the criteria, control the technical system according to the output; and if the output of the LLM does not meet the criteria, instruct the LLM once or more to correct the output until the corrected output meets the criteria, and control the technical system according to the corrected output.

Description

Technical Field

[0001] The present disclosure relates to a method for controlling a technical system, in particular a robot or a vehicle (or a component thereof). Background Art

[0002] Control of technical systems can be very complex. Accordingly, there are extensive requirements for control software for controlling technical systems, so that this control software accurately reproduces the desired behavior. In particular, the control software must completely and consistently reflect the desired behavior, which may only be described at least partially in text form (and possibly incompletely). However, it is difficult to implement a text-based description completely and consistently. This is because a precise algorithm typically requires a precise formal model. Therefore, an approach that ensures that a technical system is accurately controlled as desired, even if the desired properties are described in an unstructured form, is desirable. Summary of Invention Means for Solving the Problems

[0003] Methods for controlling a technical system are disclosed according to various different embodiments, the method comprising supplying a textual description of the behavior of the technical system to a large language model (LLM) by instruction, the instruction to produce an output comprising presentations of several states of the technical system and presentations of conditions for several states relating to input parameters of the technical system, the presentations of which of the several states the technical system is in under what configuration of input quantities; checking whether the output of the LLM satisfies criteria for completeness and / or integrity, and if the output satisfies the criteria, controlling the technical system according to the output; and if the output of the LLM does not satisfy the criteria, instructing the LLM once or more to correct the output until the corrected output satisfies the criteria, and controlling the technical system according to the corrected output.

[0004] The method described above makes it possible to generate precise control rules for technical systems (e.g., vehicles or robots) using a textual description as a starting point. In particular, discrete decision models, such as state machines or decision trees, can be generated from the output of the LLM, enabling effective checks for consistency and / or completeness, as well as precise control. Therefore, the LLM is used as an interface between a natural language description of a desired behavior and a precise mathematical description of that behavior.

[0005] This method may first involve, for example, one or more human users creating a text-based description.

[0006] The following shows various different examples.

[0007] Example 1 is a method for controlling the technical system described above.

[0008] Example 2 is the method of Example 1, wherein instructing the LLM to correct the output includes supplying the Large Language Model with supplements or modifications to a textual description of the behavior of the technical system (e.g., by a human user).

[0009] The inspection may also lead to the recognition that the textual description is incomplete or incorrect, for example, if the LLM fails to provide complete or consistent output even after multiple instructions. In that case, the textual description can be supplemented and / or modified to obtain a complete and consistent model for control.

[0010] Example 3 is the method of Example 2, wherein at least one additional behavior rule is added as a supplement to the text-based description.

[0011] For example, the inspection can reveal that the textual description is incomplete, such as failing to describe the behavior (i.e., state, e.g., operating mode) for one or more combinations of input parameter configurations (i.e., choices). By supplementing with corresponding behavioral rules, the LLM can adequately complement its own output.

[0012] Example 4 is a method according to any one of Examples 1 to 3, comprising: generating a discrete decision model describing the behavior of a technical system from the output of a large language model or a corrected output of a large language model; checking the discrete decision model to determine whether the output or corrected output meets a criterion; and, if the output or corrected output meets the criterion, controlling the technical system according to the discrete decision model.

[0013] The output of an LLM is more structured than a text-based description. From this LLM output, a formal (or mathematical) discrete decision model can be generated, such as a decision tree (e.g., a BDD (binary decision tree)) or a state machine. Such decision models can be efficiently checked for completeness and consistency using mathematical methods. Furthermore, efficient control is possible based on such decision models; that is, it is possible to generate an accurate control model from a text-based description.

[0014] Example 5 is a method according to any one of Examples 1 to 4, wherein instructing the LLM to correct the output includes instructing the LLM to supplement one or more such conditions in the output of the LLM with respect to one or more configurations of the input quantities, if the conditions in which the technical system exists in one or more configurations of the input quantities are not presented.

[0015] LLM may not exhaust all possible combinations of input parameter choices in its output, for example. This is identified when checking the criteria (particularly with respect to completeness), and LLM is appropriately instructed to supplement its output in this regard.

[0016] Example 6 is a method of any one of Examples 1 to 5, wherein instructing the LLM to correct the output includes instructing the LLM to supplement possible configurations when the presentation of possible configurations is not fully presented in the output of the LLM with respect to one or more input quantities.

[0017] For example, LLM may not be able to present, or present all, possible configurations (e.g., values ​​or ranges of values) for certain input quantities. This, too, is identified when checking the criteria (particularly with respect to completeness), and LLM is appropriately instructed to supplement its output in this regard.

[0018] Example 7 is a method according to any one of Examples 1 to 6, wherein controlling the technical system according to an output or a corrected output includes generating control software from the output or a corrected output and executing the control software by a control device of the technical system.

[0019] Example 8 is a data processing system (in particular, a control device or control means) configured to carry out the method described in any one of Examples 1 to 7.

[0020] Example 9 is a computer program comprising instructions, wherein when the instructions are executed by the processor, the computer program causes the processor to perform the method described in any one of Examples 1 to 7.

[0021] Example 10 is a computer-readable medium for storing instructions, wherein the instructions cause the processor to perform the method described in any one of Examples 1 to 7 when the instructions are executed by the processor.

[0022] Similar reference numerals in each figure generally refer to the same part in all different drawings. The drawings are not necessarily to scale, and instead are primarily intended to illustrate the principles of the present invention. Various different embodiments will be described in the following description with reference to the following drawings. [Brief explanation of the drawing]

[0023] [Figure 1] This is a diagram of a vehicle. [Figure 2] This figure shows the identification and inspection of a formal description of the behavior of a technical system according to one embodiment. [Figure 3] This is a flowchart showing a method for controlling a technical system according to one embodiment. [Modes for carrying out the invention]

[0024] The following detailed description relates to the accompanying drawings, which are shown for illustrative purposes to depict specific details and aspects of the present disclosure by which the present invention may be implemented. Other aspects may be utilized, and structural, logical and electrical changes may be made without departing from the protective scope of the present invention. The various different aspects of the present disclosure are not necessarily mutually exclusive. This is because several aspects of the present disclosure may be combined with one or more other aspects of the present disclosure to form new aspects.

[0025] In the following, various different examples will be described in more detail.

[0026] Figure 1 shows a vehicle 101.

[0027] In the example of Figure 1, the vehicle 101, for example a motor vehicle such as a passenger car or a truck, is provided with a vehicle control device (for example, an electronic control unit (ECU)) 102.

[0028] The vehicle control device 102 has a data processing component, for example a processor (for example, a CPU (central processing unit)) 103, and a memory 104 for storing control software 107 (that is, one or more computer programs) that the vehicle control device 102 complies with to operate, and data to be processed by the processor 103. The processor 103 executes the control software 107 (therefore, the control software 107 is illustrated as part of the processor 103 in Figure 1).

[0029] The control software 107 comprises instructions that, when executed by the processor, cause the processor 103 to control one or more components of the vehicle 101.

[0030] The control software 107 is transmitted, for example, from the computer system 105 to the vehicle 101, for example, via the network 106 (or using a storage medium such as a memory card). This can also be done during operation (or at least when the vehicle 101 is with the user), because the control software 107 is updated to a new version over time, for example.

[0031] Requirements for software such as control software 107 (i.e., desired behavior, such as how a robotic device is controlled) are the starting point for each software development project (at least for software of a relevant scale and complexity). Based on these requirements, the software architecture is defined, the code is developed, and test cases are written. To achieve this, the requirements must be complete and consistent. Complete means that all relevant use cases are described. Consistent means that there are no inconsistencies within the given requirements.

[0032] This challenge becomes more complex as the number of requirements increases. For example, in the case of a fuel cell control system (for a fuel cell in vehicle 101), the set of requirements includes more than 4,000 requirements for each control software. Manually checking these requirements (and by extension the behavior they represent) for completeness and integrity is practically impossible. However, failing to check for completeness and integrity entails a significant risk of error.

[0033] One way to address the problem of requirements errors (and consequently, errors in describing behavior) is to use a formal specification methodology such as Model-Based Systems Engineering (MBSE). In MBSE, domain models—rather than requirements documents in natural language (NL)—are used as the primary means of communication among engineers. Because the specification language is substantially mathematical, these domain models can be tested and verified. There is little room for ambiguity, and the probability of requirements errors going undetected is significantly lower.

[0034] However, these models must be created and maintained manually by system experts, which is time-consuming. A newer approach is to directly use Large-Language-Models (LLMs) for requirements analysis. This approach lacks transparency and has a fundamental problem: LLMs are vulnerable to classical mathematical logic and, consequently, to the formal mathematical models (such as decision trees) that software should adhere to when controlling robotic devices. This is because formal mathematical models typically differ significantly from natural language.

[0035] According to various embodiments, an approach is provided that enables the inspection of a set of requirements for software (and / or control of a robotic device) with respect to completeness and integrity by a highly automated, fast, and reliable method. According to various embodiments, this includes, in particular, the automatic formulation of requirements—in this case, a textual description of the desired behavior of a technical system and, consequently, the software controlling the technical system—so that the requirements thus automatically formulated can be used as a basis for inspection and further steps such as optimization or test case generation.

[0036] For this purpose, the ability of LLMs to operate using natural language is combined with the mathematical capabilities of formal (decision) models, thereby checking requirements for mathematical properties such as completeness or consistency, as will be explained below with reference to Figure 2, for example. For example, computer system 105 implements LLM 108.

[0037] Figure 2 schematically shows the identification and examination of a formal description of the behavior of a technical system according to one embodiment.

[0038] SCODE (Systematic Co-Design) Essential Analysis (based on Binary Decision Diagrams (BDDs)) enables the formulation of system know-how and its inspection for characteristics such as completeness and consistency. This is carried out, for example, through question-and-answer sessions with system experts. According to various embodiments, a specification of requirements for the desired behavior of a technical system, here in text form 201, is sent to a large-scale language model 202 (i.e., an LLM such as AskBosch, ChatGPT, Llama, etc.), and the know-how contained in the specification is formulated through a virtual "question-and-answer session" between the language model 202 and the SCODE tool, which then generates an SCODE model, in this example a BDD 203 representing the behavior. In this case, for this "question-answer session," a question or instruction 204 (for example, a question asking about elements missing for the completeness of model 203, or a question about duplication (and thus inconsistency) within decision model 203) is derived from SCODE model 203 and forwarded to language model 202. The SCODE model 203 is updated by LLM202's response, and this procedure is repeated until there are no more duplications and omissions (i.e., possible combinations of input parameters that have not been addressed).

[0039] In other words, for example, the following will be carried out:

[0040] 1) A request 201 (regarding the behavior of a technical system) existing in natural language is passed to LLM202 via input instructions, and the context is defined.

[0041] 2) The user defines the relevant technical system by presenting the relevant system interface to LLM202. This information consists of the relevant dimensions (i.e., input parameters) in SCODE Model 203. In addition, various possible system states (modes) are defined. Optionally, LLM202 itself can be used to extract suggestions from Request 201 regarding relevant dimensions and / or system states as a basis for manual selection and / or completion by the user.

[0042] 3) All possible discrete values ​​(choices) for all dimensions are identified by corresponding questions to the LLM (in the form of one or more prompts) based on the requirements given in natural language form (e.g., temperature = "low", "high", or "very high", joint position = "between 0 and 30 degrees", "greater than 30 degrees"). The information obtained consequently from the responses defines the choices for each dimension used in the formal model 203. Now the combinatorial space of the system in question is defined (i.e., a combinatorial space is a set of modes, and each combination of input parameters yields one specific mode, provided that the set is complete and each mode is uniquely determined when the requirements or model are consistent). All of this can be done automatically by the SCODE tool (i.e., a tool that generally generates and / or checks formal models), and in particular, communication with the LLM202 via automatically generated prompts can be done automatically.

[0043] 4) For a given input combination, if the system response (i.e., the mode (i.e., state) to which the technical system should transition is not defined) is formulated and forwarded to the LLM. The response is used to update the SCODE model. This process (formulating the prompt using the question and updating the formal model 203) can be performed automatically by an SCODE tool (i.e., a tool that generally generates and / or checks the formal model).

[0044] 5) Step 4 is repeated until all combinations of dimensions (i.e., different choices of input parameters, which may also be ranges of values) are covered.

[0045] 6) The result is a complete (SCODE) model 203 (with respect to the input parameters defined so far) based on the given requirement 201. Now, this model 203 can be further analyzed with respect to consistency 205 and completeness 206 (for example, again by the SCODE tool or a general inspection tool).

[0046] 7) If consistency 205 or integrity 206 is not provided, the model 203 will be adjusted, or requirement 201 will also be adjusted. This adjustment may require a manual process, such manual process requiring system know-how (particularly for adjusting requirement 201) and therefore not performed automatically (or at least not entirely automatically).

[0047] 8) If the specifications of requirement 201 and SCODE model 203 are ultimately complete and consistent, SCODE model 203 may be used for control purposes, after optimizing the structure or the number of requirements, or for the purpose of implementing test cases.

[0048] In the following, as a simple example, we will show an example of input instructions (prompts) for LLM202 to create SCODE model 203 from natural language request 201, and explain how those input instructions (prompts) are used.

[0049] System prompt: "You are an AI assistant that analyzes natural language requests. Please respond using a JSON object."

[0050] An example of the first prompt, i.e., a prompt with a request in the form of natural language. (Step 1 in the above flow): "The following describes the requirements of the software system. The control unit should determine whether the app is permitted to run in the vehicle. If it cannot determine this, it should switch to fallback mode. If the vehicle's hood is open, the app should be disabled. The app should only be permitted to run if the wipers are not manually switched on, and it is not raining or the vehicle is stopped."

[0051] A prompt to determine the system mode (step 2a in the above flow). What are some system modes that are distinguished by differences in behavior? Please use concise and convincing names for the modes. Please present your response in the following JSON format: {"System-Mode":[{"Mode 1":"Mode Name"},{"Mode 2":"Mode Name"},···]}"

[0052] In the above example request, the LLM202's response to this prompt would be, for example, as follows: {'System-Mode':[{'Mode 1':'Execute Allowed'},{'Mode 2':'Execute Disabled'},{'Mode 3':'Fallback Mode'}]} This means that there are three system modes, and these three system modes can be directly inherited by SCODE Model 203.

[0053] Prompts for the input signal (steps 2a and 3 in the above flow) to prompt the LLM202 to output the input parameters and their possible options if the LLM202 has not already done so: What are the relevant input signals? And what are their possible values? Please provide your response in the following JSON format: {"Input signal":[{"Signal":"Signal name", "Value":[List of possible values]},...}"

[0054] The LLM202's response to this prompt might be, for example, as follows: {"Input Signal":[{"Signal":"Hood Status", "Value":["Open", "Closed"]},{"Signal":"Wiper Status", "Value":["On", "Off"]},{"Signal":"Rain Status", "Value":["Rain", "No Rain"]},{"Signal":"Vehicle Status", "Value":["Driving", "Stopped"]}]}

[0055] This defines the dimensions (input signals or input parameters) and their choices (list of values ​​per input signal) for the SCODE model 203. If only a single value exists for one input parameter, LLM202 is instructed to refine its choices (for example, by the SCODE tool) as follows: "Based on the relevant values ​​or ranges mentioned in the text, refine the possible values ​​for signal {signal name}."

[0056] If LLM202 only presents input parameters without offering any options, you will be asked questions about these input parameters, for example, as follows (step 3 in the above flow): "What are the possible values ​​of the input signal {signal name} based on the relevant values ​​or ranges mentioned in the text? Please provide your response in the following JSON format: {"Input signal":[{"Signal":"Signal name", "Value":[List of possible values]},...}"

[0057] For each mode, the corresponding conditions (i.e., the combination of input parameters required to achieve that mode) are invoked by the LLM202 as follows (steps 4 and 5 in the flow above): Under what conditions is mode {mode-ID} "{mode name}" valid? Please use predefined signals and values ​​whenever possible. Please present your response in JSON format as in this example: Please provide multiple options for the {"Condition":[{"Rainfall Condition":"No Rain", "Vehicle Condition":"Stopped"}]} selection.

[0058] For mode 2, in this example, the following results can be obtained: {"Condition":[{"Hood Status":"Open"},{"Wiper Status":"On"},{"Rainfall Status":"Rain", "Vehicle Status":"Driving"}]}

[0059] This means that Mode 2 is active when a) the hood is open, b) the wipers are switched on, or c) it is raining and the vehicle is in motion. The SCCODE tool translates this information into three rules regarding the mode. If an option not included in the previously detected signal list is used in a mode condition, the SCCODE tool prompts the LLM203 to improve this list based on the missing options, for example, using a prompt: "Improve the input signals and input values ​​by additionally considering the following input signals and their possible values:" In this prompt, the signal and option values ​​are presented in JSON format.

[0060] The result (i.e., the LLM output as a response to this prompt) is used to define a new dimension / choice. If the choice is changed as a result, the mode condition (and therefore the rule for this mode) is newly identified using the input instruction as described above.

[0061] For inspection (SCODE analysis) and feedback, the collected dimensions, choices, modes, and rules are exported in an appropriate file format (e.g., BDD file format), and the SCODE tool performs the SCODE analysis.

[0062] In summary, various different embodiments provide a method as shown in Figure 3.

[0063] Figure 3 shows a flowchart 300 illustrating a method for controlling a technical system according to one embodiment.

[0064] In 301, a textual description of the (desired) behavior of a technical system is supplied to a large language model by instructions (in the form of one or more prompts), which instruct the model to produce an output comprising a presentation of several (possible) states of the technical system and a presentation of conditions for these states relating to the input parameters of the technical system (i.e., input quantities or input signals, also referred to here as dimensions), which present the states in which of the several (possible) states the technical system is in for any configuration of input quantities (i.e., possible choices or combinations thereof of input quantities, e.g., "the hood is open," "the temperature is above zero degrees," etc.).

[0065] In 302, the output of the LLM (which the LLM produces in accordance with the textual description and instructions) is checked to see if it meets the criteria for completeness and / or integrity (of states and / or conditions), and if the output meets the criteria, the technical system is controlled according to this output.

[0066] In 303, the LLM is instructed once or more times to correct the output until the corrected output meets the criterion, depending on whether the LLM's output does not meet the criterion (i.e., output supplements and / or corrections to the output, i.e., supplement and / or correct the output), and in 304, the technical system is controlled according to the corrected output (i.e., if the output meets the criterion, the technical system is controlled according to the output; if the output does not meet the criterion, the system is controlled according to the corrected output).

[0067] The method in Figure 3 may be implemented by a data processing system (e.g., one or more computers or microcontrollers) comprising one or more data processing units. The term “data processing unit” may be understood as any type of entity that enables the processing of data or signals. The data or signals may be processed, for example, according to at least one (i.e., one or more) special function performed by the data processing unit. The data processing unit may include, or consist of, integrated circuits such as analog circuits, digital circuits, logic circuits, microprocessors, microcontrollers, central units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable gate arrays (FPGAs), or any combination thereof. Any other method for implementing each of the functions described in more detail herein may also be understood as a data processing unit or logic circuit device. One or more of the method steps described individually herein may be implemented by the data processing unit through one or more special functions performed by the data processing unit.

[0068] In other words, this method is implemented in various different embodiments, particularly in computer form.

[0069] The approach in Figure 3 is used to control technical systems, particularly robotic devices. The term “robotic device” can be understood to refer to any technical system (with mechanical parts whose movement is controlled), such as computer-controlled machinery, vehicles, home appliances, power tools, manufacturing machinery, personal assistants, or access control systems. For example, control software for a technical system is generated from the output of the LLM (e.g., by computer system 105), and then the technical system (e.g., vehicle 101) is controlled accordingly (e.g., by vehicle control device 102).

[0070] Various embodiments may use sensor signals from various sensors as input quantities, such as video, radar, LiDAR, ultrasound, motion, and thermal imaging (in particular, detecting these sensor signals during control and responding to them according to the LLM's output or a corrected output).

Claims

1. A method for controlling a technical system (101), The aforementioned method, (301) provides a textual description (201) of the behavior of a technical system (101) to a large-scale language model (108, 202) by instruction, wherein the instruction instructs the model to output an output comprising a presentation of multiple states of the technical system (101) and a presentation of conditions for the multiple states relating to the input parameters of the technical system (101), wherein the presentation indicates what configuration of input quantities the technical system (101) is in each of the multiple states (301), The process involves checking whether the output of the large-scale language model (108, 202) meets the criteria for completeness and / or consistency, and if the output meets the criteria, controlling the technical system (101) according to the output (302), In response to the fact that the output of the large-scale language model (108, 202) does not meet the criteria, the large-scale language model (108, 202) is instructed once or more times to correct the output until the corrected output meets the criteria (303), Controlling the technical system (101) according to the corrected output (304), Methods that include...

2. Instructing the large-scale language model (108, 202) to correct the output is: To supply the large-scale language model (108, 202) with supplements or modifications to the textual description (201) relating to the behavior of the technical system (101). including, The method according to claim 1.

3. As a supplement to the textual description (201) above, at least one additional behavioral rule is added: The method according to claim 2.

4. A discrete decision model (203) describing the behavior of the technical system (101) is generated from the output of the large-scale language model (108, 202) or from the corrected output of the large-scale language model (108, 202), By examining the discrete decision model (203), it is possible to check whether the output or the corrected output satisfies the criteria, If the output or the corrected output satisfies the criteria, the technical system (101) is controlled according to the discrete decision model (203), including, The method according to any one of claims 1 to 3.

5. Instructing the large-scale language model (108, 202) to correct the output is: In the output of the large-scale language model (108, 202), if the state in which the technical system (101) exists in one or more configurations of the input quantity is not presented with respect to one or more of the configurations of the input quantity, instruct the large-scale language model (108, 202) to supplement one or more such states. including, The method according to any one of claims 1 to 4.

6. Instructing the large-scale language model (108, 202) to correct the output is: In the output of the large-scale language model (108,202), if the presentation of possible configurations for one or more input quantities is not fully presented, instruct the large-scale language model (108,202) to supplement the presentation of possible configurations. including, The method according to any one of claims 1 to 5.

7. Controlling the technical system (101) according to the output or the corrected output is: The process of generating control software from the output or the corrected output, The control software is executed by the control device of the aforementioned technical system (101), including, The method according to any one of claims 1 to 6.

8. A data processing system (102, 105) configured to carry out the method described in any one of claims 1 to 7.

9. A computer program comprising instructions, wherein, when the instructions are executed by the processor, the instructions cause the processor to carry out the method according to any one of claims 1 to 7.

10. A computer-readable medium for storing instructions, wherein the instructions cause the processor to perform the method according to any one of claims 1 to 7 when the instructions are executed by the processor.

Images