Method for testing a function of a technical system using an ai-assisted test-case automation process

Abstract

The invention relates to a computer-implemented method for testing a function of a technical system (130) in a test case, the method having a first step (S1) of receiving, in a request input part (311), one or more text-based requests which define at least one test case, said requests at least partly comprising natural language; a second step (52) of generating at least one regular expression for each text-based request (350) in a request checking part (312); a third step (53) of generating a test-case database (350) from the regular expressions for each request in a data structuring part (340); a fourth step (S4) of receiving one or more pieces of test-case-specific parameter information in a parameter information part (313); a fifth step (S5) of combining at least one regular expression of the test-case database (350) with at least one piece of test-case-specific parameter information in order to form one or more test-case schemes in a selection part (331); a sixth step (S6) of generating a test-case-specific algorithm from the one or more test-case schemes in a test-generation part (333); and a seventh step (S7) of transmitting the test-case-specific algorithm into a test environment (150); testing the function of the technical system (130) by executing the test-case-specific algorithm, by means of a test-case peripheral device (140) within the test environment (150); returning the test results to the computer system (200); and carrying out an evaluation by comparing at least one test result with a test-case-specific parameter.

Description

[0001] Method and system for testing a function of a technical system by AI-supported test case automation

[0002] Description

[0003] Technical field

[0004] The present invention lies in the technical field of equipment for software development, in particular requirements analysis and test case automation.

[0005] background

[0006] In both software development and requirements analysis and test case automation in general, the prevailing problem is obtaining complete requirements formulated according to predefined rules and using these requirements to automatically or even autonomously test the software, products, or technical systems to be developed. Within software development, there is currently a trend away from linear development methods, such as the classic "V-model," towards continuous development. Using machine learning or artificial intelligence methods, continuous software development can be made more efficient, for example, by supporting requirements analysis, the generation of software and program code, autonomous testing, and data and error analysis.To translate these potential benefits into real time and resource savings, two key elements are required. First, requirements should contain predefined properties according to established rules, enabling the automated or autonomous generation and execution of test cases. Second, the potential test environment for the software, product, or technical system should be designed to allow for the automated or autonomous testing of incoming requirements. However, a conflict can arise if requirements are formulated in a variety of different formats, for example, in different languages ​​or with different structures within the same language, necessitating their standardization before implementation within the test environment.Furthermore, it may be necessary to convert the defined requirements into a machine-readable language or program code. A further problem can arise here: a potential user may have a mental concept of a test case and its associated requirements, but lacks the necessary knowledge or resources to convert them into a suitable format. This creates the need for a technical solution that enables experts to transform natural language input into requirements that follow predefined rules and then automatically or autonomously transfer these requirements into test environments and execute test cases. This is the current state of the art.

[0007] European patent application EP 3 508 981 A1 discloses a touchless testing platform system that generates automated test scripts, sequences test cases, and implements solutions for detected defects. The system utilizes data mining, artificial intelligence, and machine learning to create enterprise test suites. It comprises several modules, including a test scenario and process map extractor, an automation accelerator, and a defect analyzer. It enables the analysis of usage and error patterns, the prioritization of test cases, and the dynamic determination of test case sequencing, with the goal of increasing the efficiency and quality of software testing.

[0008] Patent application US 2020 / 0409827 A1 discloses a system and method for the automated generation of test data and associated configuration data for tests, wherein an artificial neural network is used to translate test requirement data into vectors and identify a context, based on an automation module. The method includes executing the selected automation module to generate test data and configuration data. The system monitors the execution time of test scenarios and validates performance by comparison with a reference value.

[0009] European patent EP 3 588 303 B1 discloses a system and method for identifying test cases in software development, whereby requirements in the form of natural language are received as input and processed according to predefined rules. Specifically, this involves tokenization and the output of a set of tokens, which are further processed using a quantification metric to associate individual tokens with a test case data set store.

[0010] European patent EP 3 968 148 B1 discloses control code generation through active machine learning, wherein a text input, comprising control aspects, is searched and compared with a similarly formatted text in order to identify similar control aspects and derive a suitable control program from them.

[0011] European patent EP 3 903 182 B1 discloses a method for receiving natural language and automatically converting it into machine-readable language, enabling a user without knowledge of a specific programming language to generate a solution to a technical problem. Summary

[0012] The current state of the art lacks a holistic approach to test case automation, which generates the result of a test case for software, a product, or a technical system as output from an input text containing one or more requirements in different forms of natural language, optionally automatically or semi-automatically, taking user acceptance into account, executes individual intermediate steps, and optionally visualizes one or more of the intermediate steps.

[0013] The object of the present invention is to provide a method and a computer-based system for test case automation by AI-supported requirements analysis, which receives an input text comprising one or more requirements in different forms of natural language as input, automatically or autonomously generates a test case from it, executes it and produces the result of the test case as output.

[0014] This problem is solved by a computer-implemented method according to claim 1 and a system according to claim 15 for testing a function of a technical system in a test case by executing individual method steps that are at least partially executed by a processor of a computer system, such as receiving one or more text-based requirements, comprising at least a portion of natural language, defining at least one test case, in a requirement input section; generating at least one regular expression for each text-based requirement, in a requirement verification section; generating a test case database from the regular expressions of the respective requirements, in a data structuring section; and receiving one or more test case-specific parameter information.in a parameter information section; combining at least one regular expression from the test case database with at least one test case-specific parameter piece of information to form one or more test case schemas, in a selection section; generating a test case-specific algorithm from the one or more test case schemas, in a test generation section; transmitting the test case-specific algorithm to a test environment; testing the function of the technical system by executing the test case-specific algorithm through a test case peripheral within the test environment and returning the test results to the computer system and evaluating them by comparing at least one test result with a test case-specific parameter. A regular expression within the meaning of the invention is a text expression or a text segment that follows certain rules of syntax, grammar,The language and vocabulary are followed. These rules are preferably predefined and provided before the execution of the inventive method. In one embodiment, the generation of at least one regular expression for each text-based requirement is performed by a first machine learning language processing model (LLM), which is created using training data based on regular, text-based requirements that comply with predefined rules, including one or more of the syntax, grammar, atomicity, conciseness, uniqueness, consistent language, schema following, design independence, and purpose independence. The language model can thus be adapted to the task of generating a regular expression in the context of the invention.

[0015] Alternatively, a basic language model trained with general language data can be used. These basic models are available without specific training. To make them usable for the task of generating a regular expression in the context of the invention, they are used with customized inputs or instructions and / or parameters passed during input. This so-called "prompt engineering" is used to pass context information to the basic model, which improves the generation of regular expressions. In one possible embodiment, examples of the desired inputs / outputs can be passed to the language processing model (LLM) as part of the prompt, so that it then continues the specific task multiple times according to this pattern. Furthermore, different prompts with subtasks can be used to translate requirements into regular expressions.For example, the LLM natural language processing model can first evaluate criteria for the regular expressions, then generate improvement suggestions, and then re-evaluate the criteria based on these improvements. The compilation of these prompts and their sequence is part of the automated program logic according to the invention for converting the requirements into regular expressions.

[0016] In one embodiment, the generation of a test case database (350) from the regular expressions of the respective requirements is performed by a second language processing model (LLM), which is located within a data processing section (360). The second LLM is created using training data based on regular, text-based requirements that conform to predefined rules. The second LLM is configured to receive text-based commands in natural language and, in response, process data from the test case database (350), including adding or removing, reading and displaying, and merging data. As an alternative to the specially trained language processing model, the second language processing model can also consist of a basic language model, with the output being generated by prompt engineering.As previously described, contextual information is passed to the base model to improve the generation of regular expressions. Examples of the desired input / output can be passed to the natural language model (LLM) as part of the prompt, or different prompts with subtasks can be used to convert requirements into regular expressions. These subtasks might, for example, first evaluate the criteria for the regular expressions using the LLM, then generate improvement suggestions, and finally re-evaluate the criteria based on these improvements. The second natural language model can then be used to capture the requirements converted into regular expressions by the first model and represent them in a knowledge graph using information about the architecture of the system under test.The knowledge graph, which can be created for the entire technical system, is used to derive the individual test cases from its structure and the information stored for the individual elements. In one implementation, the knowledge graph is generated by the second language processing model (LLM), which is created using training data comprising sample data from similarly structured requirement texts ("few-shot prompting") and entity-relationship data ("entity-relationship extraction prompting").

[0017] In one embodiment, receiving one or more test case-specific parameter information includes receiving individual numerical values ​​or at least partial structural information of the technical system (130), which are arranged within the data structuring section (340).

[0018] In one embodiment, combining at least one regular expression of the test case database (350) with at least one test case-specific parameter information to form one or more test case schemas includes extracting at least one structure-relevant piece of information, such as at least one function, one signal and one message, from the at least one parameter information and their connections from the at least one regular expression.

[0019] In one embodiment, generating a test-case-specific algorithm from one or more test-case schemas in a test generation section (333) comprises extracting information about at least one function, one signal, and one message from the one or more test-case schemas and inserting that information into a test-case template, wherein the test-case-specific algorithm comprises a process flow in machine-readable language. In contrast to the prior art described above, the present invention analyzes, unifies, and corrects various forms of natural language, such as mixed language, syntactically and according to other established rules.The corrected speech input is further converted into machine-readable language via various intermediate steps and transmitted to a test environment or an interface to a software development, a product to be developed or a technical system, whereby the machine-readable language is executed to carry out a test case and the result of the test case is fed back and optionally visualized.

[0020] The objective technical effects of the present invention result in a reduction of the programming and formulation effort required to establish requirements according to predefined criteria; an improvement of software, a product, or a technical system through the automated or autonomous execution of a test case; a saving of time in the testing of software, products, or technical systems; and a scaling of the time saving with the number of tests to be performed.

[0021] Brief description of the drawings

[0022] Fig. 1 shows a system for testing a function of a technical system by AI-supported test case automation.

[0023] Figs. 2A - 2B show a computer system for executing the procedure for testing a function of a technical system by AI-supported test case automation.

[0024] Fig. 3A shows the representation of a test case database within a graphical user interface.

[0025] Fig. 3B shows the process flow of the method according to the invention.

[0026] Figures 4A - 4D show visualized embodiments of a requirements verification section.

[0027] Figs. 5A - 5B show visualized process steps of generating regular expressions.

[0028] Detailed description

[0029] Fig. 1 shows a system (100) for testing a function of a technical system (130) by AI-supported test case automation. The method according to the invention is executed at least partially by a computer system (200). The at least partial execution of the method by the computer system (200) results from the possibilities of semi-automated or fully automated execution of the method. Semi-automated execution is achieved, for example, by integrated user acceptance queries or user intervention options.

[0030] User acceptance checks can be implemented in one or more process steps. A user acceptance check can be implemented by completely or partially interrupting the execution of the process steps, with the process resuming after predefined criteria are met. A criterion for a user acceptance check can be the confirmation of the current process step by a user (120). A user intervention option can be implemented by interrupting the execution of the process steps, with the process resuming after the user intervention is completed. User intervention can be the addition or removal of additional information by the user (120). For example, user intervention can include changing settings within the computer system (200) by the user (120).For example, user intervention may include the user entering, deleting or changing one or more parameters or other data (120).

[0031] The system (100) can comprise one or more databases (110). A database (110) within the meaning of the invention can be configured as an external database (110), for example, as a cloud application or a server-based backend. Alternatively, a database (110) within the meaning of the invention can be arranged wholly or partially within the computer system (200). In any case, the system (100) is configured to provide data-related communication between one or more databases (110) and the computer system (200), so that the computer system (200) can exchange data and information with the one or more databases (110) via a potential transmission path. The system (100) further comprises at least one technical system (130) to be tested, a test peripheral (140), and a test environment (150).For the purposes of the invention, a technical system (130) is any technical system (130) to which at least one input variable is supplied and which generates an output variable from that input variable using technical means. The technical system (130) comprises at least one function that describes the underlying process or algorithm for generating an output variable from at least one input variable, such that this function can be tested within the method according to the invention. The technical system (130) can, for example, comprise any type of machine, in particular motor vehicles. For the purposes of the invention, a test peripheral (140) is any technical means that is not necessarily part of the technical system (130) but is configured to test a function of the technical system (130) that exists according to a defined test case.In one embodiment, the test peripheral (140) can also include a computer (200) which establishes communication with the technical system (130) via conventional transmission paths and, for example, influences the control of the technical system (130) via data and information communication. In some embodiments, the computer (200) of the test peripheral can be fully or partially integrated into the computer (200) for executing the method according to the invention. In an alternative embodiment, the test peripheral (140) can include one or more technical means for acquiring measurement information, or measuring instruments, in order to test the functionality of the technical system (130).For the purposes of the invention, a test environment (150) is understood to be any space that is equipped to accommodate the corresponding technical system (130) to be tested, as well as the test peripherals (140) required for the associated test case, and to provide further supporting peripherals. In an embodiment in which the technical system (130) comprises a motor vehicle, a test environment may, for example, include a motor vehicle test bench or a component test bench.

[0032] Fig. 2A shows a computer system (200) for carrying out the inventive method for testing a function of a technical system (130) by AI-supported test case automation. The computer system (200) can comprise one or more of the following elements, such as an internal communication device (210), a computing device or processor (220), an operating device (230), a storage device (240), an information transmission device (250), a network interface (260), and a database (270). The internal communication device (210) can comprise a bus or a bus system, such as a peripheral component interconnect (PCI architecture bus). The internal communication device (210) is configured to exchange data and information between the respective elements of the computer system (200).The computing device (220) can comprise a processor, for example a CPU (central processing unit), or any other type of processor, such as a microprocessor, a digital signal processor, or a microcontroller, suitable for performing calculations related to the computer program according to the invention. The operating device (230) can comprise haptic devices such as a computer mouse, a computer keyboard, a control lever, or other optical or wireless peripherals, or further devices that serve as an interface between the computer system (200) and a user (120). The operating device (230) is configured to allow the user (120) to input commands or other input information into the computer system (200).The storage device (240) can include one or more random access memory (RAM), data storage devices such as read-only memory (ROM), and mass storage devices. For example, instructions such as arithmetic operations, general functional sequences, or algorithms contained in the computer program can be loaded into the RAM of the storage device (240) of the computer system (200), which are then executed by a processor of the computing device (220). The storage device (240) can also include any type of mass storage device intended for storing all kinds of information and data, such as multiple hard disks, floppy disks, optical disks, semiconductor memory, non-volatile memory, EPROM, EEPROM, flash memory, magnetic disks, removable storage media, or other storage media.The information transmission device (250) is generally designed to transmit information to the user (120) in a variety of ways. The information transmission device (250) may include one or more of the following elements, such as a screen or other display for the visual presentation of information and data, or a loudspeaker or other device for the acoustic presentation of information and data. The information transmission device (250) may be fully or partially integrated with the operating device (230) and / or functionally coupled to it. For example, elements of the operating device (230) and the information device (250) may be designed in a graphical user interface (GUI).The network interface (260) can include various communication protocols, such as Transmission Control Protocol / Internet Protocol (TCP / IP) or Hypertext Transfer Protocol (HTTP), or others suitable for providing external communication links to the computer system (200) via remote access and / or computer networks. The database (270) can include computer program-specific data and information, such as model data, process flow data, configuration data, algorithm data, parameter data, design data, but also program tools, such as design environments like CAD (computer-aided design) applications, analysis environments, or others.

[0033] Fig. 2B shows a possible configuration of the storage device (240) of the computer system (200). In one aspect of the invention, a computer program (300) is arranged within the storage device (240). The computer program comprises different memory areas or sections, such as a data input section (310), a database section (320), and a test section (330). The individual sections can form their own subroutines or subsections. Each section can be designed as hardware- or software-implemented means of the computer system (200). In general, all process steps of the computer-implemented method can be carried out by means that are fully or partially implemented in hardware or software. Specific embodiments are described by the embodiments listed here.The data input section (310) is essentially configured to receive data of various types and introduce them into the method according to the invention. The database section (320) is essentially configured to store a test case database (350) and to provide possibilities for data exchange between that database and other sections of the computer program (300), the computer system (200), or the entire system (100). The test section (330) is essentially configured to receive test cases from the test case database (350), process them, and generate and execute the actual tests from them.

[0034] The data input section (310) comprises a request input section (311), a request verification section (312), and a parameter information section (313). The request input section (311) is primarily designed to receive text-based requirements in natural language, or request texts. The input of these request texts can be automated by loading files and documents through the request input section (311). The request texts can be in text-based or table-based formats, such as Excel files, CSV files, or any editor-generated files, as well as other file formats like CAN dbc, A2I, or hex files. Alternatively, request texts can be loaded manually by the user.For this purpose, one or more parts of the computer program can be displayed completely or partially via a graphical user interface (GUI) and provide interaction between the user (120) and the computer system (200). The imported data preferably comprises a tabular format or can be converted into one. The tabular format includes, in particular, at least one column for identifying the respective entry and at least one column for the associated data content. The requirements validation section (312) is essentially configured to generate at least one regular expression for each requirement of the test case database (350). For the purposes of this invention, a regular expression is to be understood as a text expression or text segment that follows certain rules of syntax, grammar, language, and vocabulary.These rules are preferably predefined and provided before the execution of the method according to the invention. In one aspect of the invention, these rules are loaded as parameters into the computer program (300), in particular into the data input section (310), and more preferably into the parameter information section (313). The parameter information section (313) is essentially configured to receive additional parameters that interact with the test case database (350) during the execution of the method or at least influence the execution of the method at one or more points.

[0035] The database section (320) comprises a data structuring section (340), a test case database (350), and a data processing section (360). The data structuring section (340) is essentially configured to arrange the data received in the data input section (310) in a predefined structure and optionally also to display it, so that further sections, such as the test section (330), can process the corresponding data in a structured manner. In one aspect of the invention, the data structuring section (340) includes the GUI and its display within the computer system (200), as well as providing corresponding interaction elements for the user (120). The test case database (350) is essentially configured to store all preprocessed requirements, along with the associated test case information and other information.The data processing section (360) is essentially configured to process data from the data structuring section (340) and the test case database (350) within the database section (320). In one aspect of the invention, the data processing by the data processing section (360) includes, but is not limited to, modifying database entries, reading information from the database entries, and preparing and initializing test cases.

[0036] The test section (330) comprises a selection section (331), a knowledge graph (332), and a test generation section (333). The selection section (331) is essentially configured to select and display requirement texts and possible, associated test cases. The knowledge graph (332) is essentially configured to link the regular expressions generated from the requirement texts with the structural information of the technical system under test, thereby structuring the information for the test cases, taking into account associated parameter information. The test generation section (333) is essentially configured to generate one or more tests, taking into account parameter information and the knowledge graph (332), and to transmit or execute these tests in the test environment (150).

[0037] Figure 3A shows the representation of the test case database 350 within a graphical user interface (GUI) within the computer program (300). The individual sections of the computer program can be arranged as separate parts within the GUI. For example, the data input section (310) can form its own area within the GUI, which is labeled "Load Data". For example, the database section (320) can form its own area within the GUI, which is labeled "Requirements". For example, the test section (330) can form its own area within the GUI, which is labeled "Test". For example, the data structuring section (340) can form its own area within the GUI, which is labeled "Structure".For example, the data processing section (360) can form its own area within the GUI, which is labelled "Tasks". The test case database (350) comprises tabularly arranged requirement information, including at least one identification column (351), one description column (352), one type column (353), one status column (354), and one test column (355). The identification column (351) is essentially configured to store index-specific information about the respective table entry. In one embodiment, the identification column includes a number or numbered names to identify the respective table entries. The description column (352) is essentially configured to store the preprocessed requirement text.The requirement texts comprise the preprocessed regular expressions that define the requirements for each test case. The type column (353) is primarily configured to store the type of each table entry. The type of a table entry includes an assigned classification to indicate the state of that entry. The status column (354) is primarily configured to store the status of each table entry. The status of a table entry indicates whether the entry has been released. The test column (355) is primarily configured to store the test status of each table entry. The test status indicates whether the test for the corresponding table entry has already been performed.

[0038] Fig. 3B shows the process flow of the method according to the invention. In one aspect, the present invention comprises a computer-implemented method for testing a function of a technical system (130) in a test case, wherein the method comprises the following steps, which are at least partially executed by a processor (220) of a computer system (200), such as receiving one or more text-based requirements defining at least one test case, comprising at least a portion of natural language, in a requirement input section (311), in a first step (S1); generating at least one regular expression for each text-based requirement (350), in a requirement verification section (312), in a second step (S2); and generating a test case database (350) from the regular expressions of the respective requirements, in a data structuring section (340), in a third step (S3).Receiving one or more test case-specific parameter information pieces, in a parameter information section (313), in a fourth step (S4); Merging at least one regular expression of the test case database (350) with at least one test case-specific parameter information piece to form one or more test case schemas, in a selection section (331), in a fifth step (S5); Generating a test case-specific algorithm from the one or more test case schemas, in a test generation section (333), in a sixth step (S6); Transmitting the test case-specific algorithm to a test environment (150);Testing the function of the technical system (130) by executing the test case-specific algorithm through a test case peripheral (140) within the test environment (150) and returning the test results to the computer system (200) and evaluating them by comparing at least one test result with a test case-specific parameter in a seventh step (S7).

[0039] In one aspect, the first step (S1) of the method according to the invention comprises receiving one or more text-based requirements, at least one of which defines a test case and includes at least a portion of natural language, in a requirements input section (311). Text-based requirements in natural language can differ in a multitude of properties according to the invention, such as the language itself, but also the syntax, grammar, vocabulary, and spelling of the same languages, and even further region-specific influences, such as dialects, in contrast to conventional requirements analyses, where the input of requirements texts is already linked to certain conditions of those aforementioned properties in order to simplify subsequent processing of the input, for example, by tokenization. However, corresponding methods fail precisely because they do not obtain the inputs in the desired form.In the method according to the invention, therefore, no requirements are placed on the input of the requirement text, so that anyone can enter data without, for example, encountering problems with an input mask or obtaining unsatisfactory results. Input in any form of natural language offers the advantage that anyone, regardless of language proficiency and with limited technical knowledge of the technical system under test, can enter data. In an alternative embodiment, input is achieved by loading corresponding files, as already mentioned in the description section for the data input section (310). In another alternative embodiment, input is performed by the user (120) of the GUI within an input mask. The two options of loading the input as files or entering it directly by a user (120) allow the first step (S1) to be executed fully or semi-automatically.In one aspect, the second step (S2) of the method according to the invention comprises generating at least one regular expression for each text-based requirement (350) in a requirement verification section (312). In one embodiment, the generation of regular expressions includes analyzing and correcting the input text-based requirements. In this second step (S2), the text-based inputs, in any form of natural language, are analyzed and rearranged according to defined rules. According to the invention, the analysis and rearrangement of the text-based inputs is performed using machine learning methods, in particular natural language processing methods, and preferably by a first language processing model or LLM (large language model).The first LLM can take any known form, as long as it is configured to analyze natural language and convert it according to defined rules. These defined rules can be introduced during the initial LLM training through the corresponding training data. Alternatively, the defined rules can be subsequently reviewed or modified and confirmed or rejected via a user acceptance query. As an alternative to a specially trained language processing model, the language processing model can also consist of a base language model, with the output specifically optimized for generating a regular expression being created through prompt engineering. As previously described, contextual information is passed to the base model, which improves the generation of regular expressions.Examples of the desired input / output can be passed to the language processing model (LLM) as part of the prompt, or different prompts with subtasks can be used to convert requirements into regular expressions. These subtasks might, for example, first evaluate criteria for the regular expressions using the language processing model LLM, then generate improvement suggestions using the LLM, and then re-evaluate the criteria based on the improvements.

[0040] Figures 4A-4D show visualized embodiments of the second step (S2), the generation of regular expressions, within the requirements validation section (312). This is represented, for example, by a GUI for requirements validation (400) or a requirements validation user interface (410). In one embodiment, the requirements validation user interface (410) comprises a requirements input section (420) and a requirements output section (430). The requirements input section (420) is essentially configured to display, preprocess, add, or remove received, text-based requirements to be analyzed.In one embodiment, the request input section (420) comprises at least one index column for one or more request input indices (421), a description column for one or more request input descriptions (422), and a selection column for one or more request input selection elements (423). In another embodiment, the request input section (420) comprises at least two user interaction elements, such as a request analysis initialization element (424) and a request input removal element (425). The request output section (430) is essentially configured to display the output values ​​of the speech analysis and correction performed by the request verification section (312) or the first LLM and optionally to initiate user interaction.In one embodiment, the request output section (430) comprises at least two user interaction elements, such as a request output confirmation element (431) and a request output rejection element (432). In one embodiment, the request output section (430) comprises a status column for one or more request output states (433). In one embodiment, the...

[0041] The request output section (430) comprises at least two display elements, including at least one request output confirmation display (434) and one request output correction display (435). User interaction can be effected by means of one or more control elements (440) in conjunction with the computer system (200) and the GUI. The control elements (440) can be parts of the control device (230) of the computer system (200). In one embodiment, the control elements (440) comprise a mouse and a keyboard. Alternatively, any configuration for interaction between a user (120) and the GUI is possible. In one embodiment, the control element (440) comprises a representation within the GUI as a mouse pointer or triangle.

[0042] Referring to Figures 4A-4D, the second step (S2) in one aspect comprises one or more of the following user interactions: manually modifying text-based requirements within the requirements input descriptions (422); adding or removing text-based requirements within the requirements input selection elements (423); removing all entries from the requirements input section (420) using the requirements input removal element (425); and initiating text analysis and correction using the requirements analysis initialization element (424). Referring to Figure 4C, and upon initiating text analysis and correction through the user acceptance query of the requirements analysis initialization element (424), the requirements texts arranged in the requirements input section (420) are transferred to the first LLM and processed by it.The result of the analysis of the text-based requirements is initially displayed in the requirements output section (430), with each table entry being assigned a requirements output status (433). If no correction is required after reviewing the defined rules, a requirements output confirmation message (434) is displayed in the corresponding table entry. If deficiencies were discovered during the analysis of the text-based requirements, a requirements output correction message (435) is displayed in the corresponding table entry.

[0043] Referring to Figures 5A-5B, the requirement correction (500) of the text-based requirements, in one embodiment, takes place within a requirement correction user interface (511). In one embodiment, the requirement correction user interface (511) includes a section for requirement correction criteria (512) and one or more requirement correction suggestion lines (513). The requirement correction criteria (512) are essentially designed to specify the defined rules for correcting the text-based requirements and to perform their evaluation. In one embodiment, the requirement correction criteria (512) include one or more of the following criteria, such as atomicity, conciseness, unambiguity, consistent language, schema adherence, design independence, and purpose independence.The requirement correction suggestion lines (513) are essentially configured to generate a regular expression for each text-based requirement that conforms to the defined rules. In one embodiment, to complete the correction, a user acceptance query is performed, which can be done via the control element (440) using the requirement correction confirmation element (514) or the requirement correction rejection element (515).

[0044] Referring to Figures 4A-4D and 5A-5B, five different received, text-based requirements are checked in an exemplary embodiment. After starting the check via a user acceptance query using the requirements analysis initialization element (424), the preliminary result of the text analysis is initially displayed in the requirements output section (430). For example, corrective actions are displayed for the requirement texts with indices 2, 3, and 5 using the requirements output correction display (435). Referring to Figure 5A, the criterion of consistent language was flagged for the text-based requirement with index 2. In this exemplary embodiment, the text-based requirement with index 2 includes the phrase "The vehicle must be able to reverse manually.", which differs from the rest of the requirement texts or the predefined rules in that it is written in English.Accordingly, a German translation is suggested in the requirement correction suggestion line (513). A further user acceptance query is now performed via the requirement correction confirmation element (514) to accept the correction suggestion by the first LLM. Referring to Fig. 5B, in another embodiment, four different criteria were criticized for the text-based requirement with index 3. In this embodiment, the text-based requirement with index 3 includes the expression "A ventilation system and an air conditioning system must be provided for the climate control of the passenger compartment." Within the display for the requirement correction criteria (512), the first LLM criticized the atomicity, conciseness, unambiguity, and design independence.Criticisms regarding atomicity include, for example, "The requirement is not atomic because it contains the conjunction 'and,' which suggests multiple requirements in one statement." Criticisms regarding conciseness include, for example, "The requirement is not precise because it contains two statements: one about providing a ventilation system and another about providing an air conditioner." Criticisms regarding unambiguity include, for example, "The requirement is not unambiguous because it is unclear whether both a ventilation system and an air conditioner are required, or whether one or the other is sufficient for climate control in the passenger compartment." Criticisms regarding design independence include, for example, "The requirement states that both a ventilation system and an air conditioner must be used for climate control in the passenger compartment."This constitutes an unjustified restriction of the design, as the use of certain technologies is mandated.” Based on the requirement correction proposal lines (513A, 513B), the first LLM generated two new requirement texts as correction proposals. For example, for the first requirement correction proposal line (513A), the correction proposal reads, “The passenger compartment must be equipped with a ventilation system.” For example, for the second requirement correction proposal line (513B), the correction proposal reads, “The passenger compartment must be equipped with an air conditioning system.” A further user acceptance check is now performed via the requirement correction confirmation element (514) to accept the correction proposal by the first LLM. Referring to Fig.In step 4D, five regular expressions were generated from the five incoming, text-based requirements and transferred to the test case database (350) as requirement texts. The option of user acceptance queries allows a potential user (120) to interactively observe and semi-automatically execute the second process step (S2) of generating regular expressions. In an alternative embodiment, the first LLM simply receives the incoming, text-based requirements in any natural language and outputs the corresponding regular expressions as output variables without user interaction, allowing the second process step (S2) to be performed fully automatically.

[0045] Thus, the second process step (S2) is advantageously characterized according to the invention in that the generation of at least one regular expression for each text-based requirement (350) is performed by a first machine learning language processing model (LLM) which is produced using training data based on regular, text-based requirements that comply with predefined rules, comprising one or more of the syntax, grammar, atomicity, conciseness, uniqueness, consistent language, schema following, design independence, and purpose independence.

[0046] In one aspect, the third step (S3) of the method according to the invention comprises generating a test case database (350) from the regular expressions of the respective requirements in a data structuring section (340). Referring to Fig. 3A, the test case database (350) can be implemented in a GUI. In one embodiment, the test case database (350) comprises at least one identification column (351) and one description column (352). In an alternative embodiment, the test case database (350) additionally comprises one or more type columns (353), status columns (354), and test columns (355). According to the invention, the individual row or table entries of the test case database (350) are received from the data input section (311) and include, in particular, the regular expressions generated by the requirements verification section (312).In a further embodiment, generating the test case database (350) includes modifying the table entries through data processing, preferably within the data processing section (360). In an advantageous embodiment, the data processing section (360) comprises a second LLM, which is generated using training data based on regular, text-based requirements that conform to predefined rules. Alternatively, a base language model can be used, which, through the "prompt engineering" described above, can structure the regular expressions in a knowledge graph using the passed parameters and context information. In one embodiment, the second LLM is configured to receive user input in any natural language and, in response, process this into data for the test case database (350). Alternatively, the already improved regular expressions can also be used.The generation of data for the test case database (350) from the knowledge graph created by the second LLM can include one or more activities such as adding or removing data, reading and displaying data, and merging data. In one embodiment, the second LLM includes a chat interface, which is preferably located within the GUI, and more preferably within the data processing section (360). In an exemplary user interaction, the user (120) can compose commands in natural language within an input mask and thus transmit them to the second LLM. The second LLM can then execute the command or request further or missing information. In one embodiment, the second LLM is trained on the training data to relate the data processing exclusively to the test case database (350).In an exemplary embodiment, a user (120) could compose the command "How many wheels does the vehicle have?", assuming the technical system (130) is a vehicle. The LLM would then process this command and search the test case database (350) for a request text containing the requested query. The second LLM could then output, for example, a response such as "According to the request ID, the vehicle must have one wheel," where ID represents the corresponding identification number from the identification column (351). However, if a sample query from user (120) is "What will the weather be like today?", the second LLM will not be able to provide user (120) with an adequate answer.

[0047] Thus, the third process step (S3) is advantageously characterized according to the invention in that the generation of a test case database (350) from the regular expressions of the respective requirements is performed by a second language processing model (LLM) which is arranged within a data processing section (360), wherein the second LLM is produced using training data based on regular, text-based requirements that comply with predefined rules, wherein the second LLM is configured to receive text-based commands in natural language and, in response, to process data from the test case database (350), including adding or removing, reading and displaying, and merging data.

[0048] In one aspect, the fourth step (S4) of the method according to the invention comprises receiving one or more test-case-specific parameter information items in a parameter information section (313). Parameter information can include individual numerical values ​​that are tested or serve as comparison values ​​for the test results. The parameter information can also include complete or partial model or structural information of the technical system (130), which defines the structure and / or functionality of the technical system (130) or parts thereof. This parameter information can be arranged and visualized, for example, in the form of a structure tree (341) within the data structuring section (340), within the database section (320), or within the GUI. The parameter information is generated for the respective section based on the knowledge graph (332).

[0049] Thus, the fourth process step (S4) according to the invention is advantageously characterized in that the reception of one or more test case-specific parameter information includes individual numerical values ​​or at least partial structural information of the technical system (130), which are arranged within the data structuring section (340). In one aspect, the fifth step (S5) of the method according to the invention comprises combining at least one regular expression of the test case database (350) with at least one test case-specific parameter information to form one or more test case schemas in a selection section (331). A test case schema comprises all data and parameter information necessary to derive one or more test cases from it. Test case-specific information is thus to be extracted from the test case schema.Furthermore, the test case schema serves to represent the information, in particular existing data and elements and their relationships to one another. A test case schema can, for example, be an ER model (Entity-Relationship model). In one embodiment, information can be visualized and / or extracted within the test case schema by filtering or searching for entities or by linking back to requirement texts in the test case database. Referring to Fig. 6, the test case schema can be arranged and visualized within the computer program (600) in a knowledge graph (332). The knowledge graph comprises the graphical representation of the currently selected test case schema. If, for example, a test case or requirement is identified in the database section (620) that has not yet been tested, the test can be initiated and switched to the test section (630). Fig.Figure 6 shows a possible embodiment of the test section (630) within the GUI, which comprises a selection section (331), a knowledge graph (332), and a test generation section (333). In one embodiment, a desired requirement can be validated or selected via a menu selection in the selection section (331). In another embodiment, after selecting a requirement, associated test cases are linked from the parameter information and displayed for selection. By initializing the requirements analysis of the selected requirement via a user acceptance query, the loaded parameter information is then linked to the selected requirement and visualized in the knowledge graph (332). The knowledge graph essentially comprises the representation of functions, signals, messages, and their connections.In an advantageous manner, structural and functional information and connections are thus stored in the system and simultaneously presented clearly to the user (120) and used for test case generation. The fifth step (S5), the merging of requirements in the form of regular expressions and parameter information into test case schemas, can be performed fully automatically in one embodiment, whereby the necessary information for all test cases and requirements listed in the test case database (350) is extracted from the parameter information and merged into the respective, corresponding test case schemas. Based on the now generated test case schema, tests or algorithms for the tests can be generated by means of a user acceptance query or fully automatically.Thus, the fifth step (S5) is advantageously characterized according to the invention in that the merging of at least one regular expression of the test case database (350) with at least one test case-specific parameter information to form one or more test case schemas comprises the extraction of at least one structure-relevant piece of information, such as at least one function, one signal and one message, from the at least one parameter information and their connections from the at least one regular expression.

[0050] In one aspect, the sixth step (S6) of the method according to the invention comprises generating a test case-specific algorithm from one or more test case schemes in a test generation section (333). Referring to Fig. 6, a test template can be extracted by selecting a test case from the selection section (331). In one embodiment, the test template can be visualized in the test section (333) of the GUI. For example, by selecting a requirement according to Fig. 6 and a "minimum signal value test" from the selection section (331), a test template according to Table 1 can be extracted by means of user input, such as via a control element (440).

[0051] Table 1 - Example Test Case Template

[0052] 1. Initialize module

[0053] { FUNC['implements']}()

[0054] 2. Check if { TRIG. name} is triggered.

[0055] { FUNC.name}_check_trigger( { TRIG. name} )

[0056] 3. Set all signal values ​​to the lowest value.

[0057] { set MSG = FUNC['SENDS']}

[0058] { for SIG in MSG['CONTAINS']}

[0059] { SIG. name}_setValue( {SIG. min} )

[0060] { endfor}

[0061] 4. Message package

[0062] { MSG.name}_pack()

[0063] Table 1

[0064] This test template serves as a template for the test algorithm and is now populated with information extracted from the test case schema of the knowledge graph (332). For example, as shown in Fig. 6, user acceptance input via an algorithm generation element (631) extracts one or more pieces of information from the test case schema, such as functions, signals, and messages, and inserts them into the test case template. In this way, a test case-specific algorithm is generated from the test case template, for example, as shown in Table 2.

[0065] In one embodiment, the test case template and the test case-specific algorithm comprise machine-readable language. In another embodiment, the test case template and the test case-specific algorithm comprise the description of a process flow by executing the test. In yet another embodiment, the sixth step (S6), generating a test case-specific algorithm from one or more test case schemas, is fully automated in a test generation section (333). In an alternative embodiment, the test section (630) includes further elements, such as the automatic linking of parameter information and the test case schema to one or more requirements or regular expressions and / or the automatic assignment of an identification number (ID).

[0066] Table 2 - Example of a test case-specific algorithm

[0067] 1. Initialize module

[0068] AK_lnit()

[0069] 2. Check if AK_dyn is triggered.

[0070] AK_Func_check_trigger('AK_dyn')

[0071] 3. Set all signal values ​​to the lowest value.

[0072] AK_AccTarget_set_Value(0.0)

[0073] AK_AccDes_setValue(0.0)

[0074] 4. Message package

[0075] ABC_MSG_pack()

[0076] Table 2

[0077] Thus, the sixth step (S6) is advantageously characterized according to the invention in that the generation of a test case-specific algorithm from the one or more test case schemes, in a test generation section (333) comprises extracting information about at least one function, one signal and one message from the one or more test case schemes and inserting that information into a test case template, wherein the test case-specific algorithm comprises a process flow in machine-readable language.In one aspect, the seventh step (S7) of the method according to the invention comprises transmitting the test case-specific algorithm to a test environment (150), testing the associated function of the technical system (130) by executing the test case-specific algorithm through a test case peripheral (140) within the test environment (150), and returning the test results to the computer system (200) and evaluating them by comparing at least one test result with a test case-specific parameter.

[0078] Reference symbol list

[0079] 100 System

[0080] 110, 270 database

[0081] 120 users

[0082] 130 Technical System

[0083] 140 test case peripherals

[0084] 150 test environment

[0085] 200 computer systems

[0086] 210 Communication device

[0087] 220 processor

[0088] 230 Operating unit

[0089] 240 storage device

[0090] 250 Information transmission equipment

[0091] 260 network interface

[0092] 300, 600 computer program

[0093] 310, 610 Data entry section

[0094] 311 Request Input Section

[0095] 312 Requirements Verification Section

[0096] 313 Parameter Information Section

[0097] 320, 620 database section

[0098] 330, 630 Test section

[0099] 331 Selection section

[0100] 332 Knowledge graph

[0101] 333 T est generation section

[0102] 340 Data structuring section

[0103] 341 Structure tree

[0104] 350 test case database

[0105] 351 Identification column

[0106] 352 Description column

[0107] 353 Type column

[0108] 354 Status column

[0109] 355 Test column

[0110] 360 Data Processing Section

[0111] 400 Requirements Review

[0112] 410 Requirements Verification User Interface

[0113] 420 Request entry section Request entry index , 510 Request entry description

[0114] Request input selection elements, request analysis initialization element, request input removal element, request output section, request output confirmation element, request output rejection element, 520 request output status, request output confirmation indicator, 521 request output correction indicator, request correction control element, request correction user interface, request correction criteria, 513A, 513B request correction suggestion line

[0115] Requirement correction confirmation element, requirement correction negation element, algorithm generation element, test case automation element

Claims

Patent claims 1. Computer-implemented method for testing a function of a technical system (130) in a test case, the method comprising subsequent steps, which are at least partially executed by a processor (220) and a memory device (240) of a computer system (200), wherein a computer program (300) is arranged within the memory device (240), comprising a data input section (310), a database section (320) and a test section (330): - Receiving one or more text-based requirements, at least one of which defines a test case, comprising at least some portion in any form of natural language, automated by inviting documents through a requirements input section (311) or the data input section (310); - Generating at least one regular expression for each text-based request (350) that follows certain rules of syntax, grammar, language and vocabulary, by a first machine learning language processing model (LLM) created using training data based on regular, text-based requests that comply with the certain rules, in a request validation section (312) of the data input section (310); - Generating a knowledge graph (332) at least from the text-based requirements or the regular expressions and structural information of the technical system (130) generated therefrom in the test section (330); - Generating a test case database (350) from the regular expressions of the respective requirements and the information of the technical system (130) structured in the knowledge graph (332), wherein a second language processing model (LLM) is executed such that the knowledge graph (332) is generated from the regular expressions and the further structural information of the technical system by structuring the information and requirements for the respective test case, in a data structuring section (340), of the database section (320); - Receiving one or more test case-specific parameter information in a parameter information section (313) of the data input section (310), comprising receiving individual numerical values ​​or at least partial structural information of the technical system (130), which are arranged and tested within the data structuring section (340) or serve as comparison values ​​for the test results; - Merging at least one regular expression of the test case database (350) with at least one test case-specific parameter information to form one or more test case schemas, in a selection section (331) of the test section (330), comprising extracting at least one structural information, such as at least one function, one signal and one message, from the at least one parameter information and their connections from the at least one regular expression; - Generating a test case-specific algorithm from one or more test case schemas, in a test generation section (333) of the test section (330), comprising extracting information about at least one function, one signal and one message from one or more test case schemas and inserting that information into a test case template, wherein the test case-specific algorithm comprises a process flow in machine-readable language; - Transmitting the test case-specific algorithm to a test environment (150); - Testing the function of the technical system (130) by executing the test case-specific algorithm, through a test case peripheral (140), within the test environment (150); - Returning the test results to the computer system (200) and evaluating them by comparing at least one test result with a test case-specific parameter, characterized in that the generation of at least one regular expression, the analysis and correction of the entered text-based requirements by means of requirement correction criteria, comprising one or more of the following criteria, such as atomicity, conciseness, uniqueness, uniform language, schema following, design independence and purpose independence.

2. Computer-implemented method according to claim 1, characterized in that the invited documents comprise text- or table-based formats, such as Excel files, CSV files, editor files, CAN-DBC files, A2l files or hex files.

3. Computer-implemented method according to claim 2, characterized in that the invited documents comprise a table format or can be converted into such a format, wherein the table format comprises at least one column for identifying the respective entry and at least one column for the associated data content.

4. Computer-implemented method according to one of the preceding claims, characterized in that the analysis and correction of the input text-based requirements includes the evaluation of the requirement criteria and the generation of improvement suggestions by the first language processing model (LLM).

5. Computer-implemented method according to one of the preceding claims, characterized in that the test case database (350) comprises an identification column (351), a description column (352), a type column (353), a status column (354) and a test column (355).

6. Computer-implemented method according to one of the preceding claims, characterized in that receiving test case-specific parameter information comprises loading the rules of the text-based requirements as parameters into the parameter information section (313), wherein the parameters interact with the test case database (350) during the execution of the method.

7. Computer-implemented method according to one of the preceding claims, characterized in that the parameter information comprises model or structural information of the technical system (130) which defines the structure and / or the functioning of the technical system (130) or parts thereof.

8. Computer-implemented method according to one of the preceding claims, characterized in that the second language processing model (LLM) comprises a chat interface arranged within the data processing section (360), wherein the second language processing model (LLM) is configured to receive user input in any natural language and process it in response to data in the test case database (350).

9. Computer-implemented method according to claim 8, characterized in that a user interaction can be composed within an input mask and transmitted to the second language processing model, which can then execute a command or request further information.

10. Computer-implemented method according to one of the preceding claims, characterized in that a test case scheme includes all data and parameter information to derive one or more test cases from it, wherein information is extracted by backlinks to requirement texts of the test case database (350).

11. Computer-implemented method according to one of the preceding claims, characterized in that the request texts are loaded by a user intervention, wherein parts of the computer program (300) are displayed via a graphical user interface (GUI) and an interaction between a user (120) and the computer system (200) is provided, wherein the computer system (200) comprises an operating device (230) and an information transmission device (250), wherein elements of the operating device (230) and the information transmission device (250) are formed within the GUI.

12. Computer-implemented method according to claim 11, characterized in that the generation of at least one regular expression for each text-based request in the request validation section (312) is carried out via a request validation user interface (410) in the GUI, comprising a request input section (420) configured to preprocess, add or remove received text-based requests to be analyzed, and a request output section (430) configured to display the output values ​​of the analysis and correction of the first LLM and to bring about user interaction.

13. Computer-implemented method according to claim 12, characterized in that it comprises generating at least one regular expression, manually modifying, adding or removing the text-based requirements within request input selection elements (423), removing all entries of the request input section (420) by means of a request input removal element (425), and starting the analysis and correction by means of a request analysis initialization element (424) within the GUI.

14. Computer-implemented method according to one of claims 11 to 13, characterized in that the test case database (350) and the chat interface of the second LLM are implemented in the GUI, wherein the user (120) can compose commands in natural language within the input mask and thus transmit them to the second LLM, wherein the second LLM can then execute the commands or request further or missing information.

15. Computer system (200), for testing a technical system, by a method according to one of the preceding claims.

Images