Computer-implemented method and computer program product for unit test generation

Abstract

A computer-implemented method for generating unit tests of a function includes obtaining initial values ​​for parameters of the function that activate the function's code elements, such that if the code elements are not activated, the method iterates to obtain corrected values ​​for those parameters that activate the code elements, wherein obtaining the initial and corrected values ​​for the parameters is accomplished through interaction with an LLM.

Description

Technical Field

[0001] This invention relates to a computer-based method and computer for unit testing of generated functions. Background Technology

[0002] Today, software is becoming increasingly large and complex, and defects within it can cause financial losses and significant damage to the reputation of companies or organizations using it. Unit tests, which verify the correctness of individual functions or individual units of source code, play a crucial role in modern software development. However, generating unit tests is a tedious and labor-intensive task, leading some developers to avoid writing them. This fact has spurred research into different technologies that allow for the automation of unit test generation.

[0003] US20090094486A1 describes a matrix (denoted as S) for calculating the possible values ​​of each parameter p in a piece of software. p A system and method for randomly generating test cases for the software segment. Each matrix has m parameters. p The possible values ​​of q are represented by a base. This method uses the prime number q. p It is greater than or equal to m p The next prime number. To determine the matrix S associated with each parameter. p Which value in the matrix will be used in the unit tests to be generated? For each test number t, calculate the value n indicating the position of the selected value within the corresponding matrix. p , where n p =t mod q p +1, such that if n p > m p Then n p >n p mod m p However, in the system and method described in US20090094486A1, it is necessary for the developer to generate a matrix S of possible values ​​for each parameter p of the software segment. p This requires work that must be done by the developers of the software, and there is no guarantee that the generated test cases will test all lines of code.

[0004] US20180210823A1 describes a system and method for performing unit tests. The method involves generating source code based on compiled code received from a user. The source code is categorized into testable and untestable components, the testable components including source code units. Furthermore, rules are identified from a set of organized rules based on the source code units. Expected parameters corresponding to a source code unit are determined based on the rules and the content of the source code unit, wherein the content includes one or more input fields. To determine the expected parameters, a user interface can be provided to receive values ​​corresponding to the expected parameters from the user. Test cases are generated for each field of the source code unit using the expected parameters. Scriptless unit tests of the compiled code are performed by executing the test cases.

[0005] US11102081B1 describes a system and method for quantitative testing of network services and their supported APIs. To facilitate testing, test scripts can be generated. These test scripts can be parameterized using input data derived from a historical database. The test scripts can be executed, and responses received from the network during testing can be analyzed to determine metrics associated with the execution of the test scripts. These metrics can indicate conditions for successful testing, conditions for unsuccessful testing, other test metrics, or combinations thereof. After the initial test cycle, it is determined whether a stopping condition has been met; if not, another test cycle can be executed. The next test cycle can utilize a different set of input data than the input data used in the previous test cycle, allowing for the testing of additional conditions and functionalities.

[0006] The paper "An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation," published by Max Shäfer et al. on September 6, 2023, discloses a large-scale empirical evaluation of the effectiveness of LLMs in automated unit test generation. To this end, LLMs provide hints including the signature and implementation of the function to be tested, as well as usage examples extracted from documentation. Furthermore, if a generated test fails, a new test is generated to fix the problem by re-hinting at the failed test and error message to the model. Summary of the Invention

[0007] The purpose of this invention is to provide a computer-implemented method for unit testing of a generated function, a non-transitory computer-readable storage medium, and a computer, as defined in the claims.

[0008] The computer-implemented method of the present invention for generating unit tests for a function includes a receiving step, wherein the source code of the function is received, the function including at least one parameter and multiple lines of code. Next, the method includes an iterative unit test generation step, wherein multiple unit tests are generated for the function.

[0009] The iterative unit test generation step includes a selection step, in which a code element to be tested is selected from the source code; followed by an input generation step, in which initial values ​​are generated for the parameters of the function. The input generation step includes: an initial prompt generation step, in which an initial prompt is generated based on the source code of the function and the code element to be tested; a step of interacting with an LLM (Large Language Model), in which the LLM provides the initial prompt and is requested to return the initial values ​​of the parameters of the function that activate the code element to be tested; and an initial text output receiving step, in which initial text output containing the initial values ​​of the parameters of the function is received from the LLM.

[0010] Following the input generation step, the iterative unit test generation step includes an iterative verification step sequence that terminates when it is determined that the code element under test has been activated. The iterative verification step sequence includes: a simulation step, wherein the execution of the function is simulated by assigning simulated values ​​to each parameter of the function, and a trace of the function's execution is generated, the simulated values ​​being initially set to the initial values ​​of each parameter; an activation check step following the simulation step, wherein it is determined whether the code element under test has been activated in the simulation step; and an input correction step, which is performed after the activation check step if it is determined in the activation check step that the code element under test was not activated in the simulation step.

[0011] The input correction step includes: a correction prompt generation step, wherein a correction prompt is generated based on the source code of the function, the code element under test, the simulated values, and the trajectory obtained in the simulation step; a step of interacting with the LLM, wherein the LLM provides the correction prompt and is requested to return the correction values ​​of each parameter of the function that activates the code element under test; and a correction text output receiving step, wherein a correction text output containing the correction values ​​of each parameter of the function is received from the LLM, such that after receiving the correction text output, the simulated values ​​of each parameter are updated with the corresponding correction values, and the method returns to the simulation step.

[0012] The iterative unit test generation step includes: a code generation step executed after the iterative verification step sequence, wherein source code for unit tests of the code element to be tested is generated; a code coverage calculation step following the code generation step, wherein the coverage of the function is calculated; and a code coverage checking step, wherein it is determined whether all code elements of the function have been tested, such that if any code elements are still to be tested, the selection step is then executed to select another code element to be tested.

[0013] The computer-implemented method of this invention allows for the automatic generation of unit tests using commercially available LLMs without requiring additional training of the LLMs, and developers are not required to provide values ​​for the parameters of functions that will be used later in the unit test generation. Because of the use of an LLM, unit tests generated using the method of this invention are more similar to unit tests written by human developers compared to unit tests generated using other methods known in the prior art, and also achieve high coverage of the functions.

[0014] In a second aspect, the present invention also relates to a non-transitory computer-readable storage medium having stored thereon a plurality of instructions which, when executed by a computer, cause the computer to perform the steps of the method of the present invention.

[0015] In a third aspect, the present invention also relates to a computer including a processor and a non-transitory computer-readable storage medium operatively coupled to the processor, the non-transitory computer-readable storage medium storing thereon a plurality of instructions which, when executed by the processor, cause the computer to perform the steps of the method of the present invention.

[0016] These and other advantages and features of the present invention will become apparent from the accompanying drawings and the detailed description of the invention. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a first embodiment of the computer-implemented method of the present invention is shown.

[0018] Figure 2 A flowchart showing the details of input generation step 22 with the method of the present invention is provided.

[0019] Figure 3 A flowchart showing details of input correction step 25 with the method of the present invention is provided.

[0020] Figure 4a An example of the function's source code is shown, and Figure 4b It shows Figure 4a An example of the trajectory of function execution.

[0021] Figure 5 A flowchart illustrating a second embodiment of the computer-implemented method of the present invention is shown.

[0022] Figure 6a An example of the trajectory generated in simulation step 23 is shown, and Figure 6b This shows the cyclic compression step 31 according to Figure 6a An example of a compressed trajectory obtained from the trajectory.

[0023] Figure 7 A flowchart illustrating a third embodiment of the computer-implemented method of the present invention is shown.

[0024] Figure 8 A flowchart showing details of trajectory compression step 32 is provided.

[0025] Figure 9 A flowchart illustrating a fourth embodiment of the computer-implemented method of the present invention is shown.

[0026] Figure 10 A flowchart illustrating a fifth embodiment of the computer-implemented method of the present invention is shown.

[0027] Figure 11 A flowchart showing details of equation management step 33 is provided.

[0028] Figure 12a An example of the trajectory generated in simulation step 23 is shown, and Figure 12b This shows that in equation generation step 331, according to Figure 12a An example of a script for trajectory generation.

[0029] Figure 13a An example of the trajectory generated in simulation step 23 is shown, and Figure 13b This shows that in equation generation step 331, according to Figure 13a An example of a script for trajectory generation.

[0030] Figure 14 A schematic diagram of an embodiment of a computer 500 configured to implement the method of the present invention is shown.

[0031] Figure 15 A flowchart of a sixth embodiment of the computer-implemented method of the present invention is shown.

[0032] Figure 16a An example of the trajectory is shown, and Figure 16b and Figure 16c This shows the condition chain extraction step 34 based on Figure 16a The trajectory yields a condition chain and an example of the first unexecuted condition.

[0033] Figure 17 An implementation of an automated software testing system is shown. Detailed Implementation

[0034] Figure 1 A flowchart illustrating a first embodiment of the computer-implemented method of the present invention is shown.

[0035] The computer-implemented method 10 for generating unit tests for a function according to the present invention includes a receiving step 1, wherein the source code of the function is received, the function comprising at least one parameter and multiple lines of code. In the context of the present invention, the term "function" is used to refer to a software unit, a piece of software, or a software component whose source code is provided to the method of the present invention. The parameters of the function may be assigned one of one or more possible values. The computer-implemented method 10 includes an iterative unit test generation step 2 following the receiving step 1, wherein multiple unit tests are generated for the function.

[0036] The iterative unit test generation step 2 includes a selection step 21, in which a code element to be tested is selected from the source code. In each iteration of the iterative unit test generation step 2, the code element to be tested in that iteration is first selected, and then a unit test configured to test that code element is generated, such that the code element to be tested in each iteration is a different code element to be tested than the code element to be tested selected in the previous iteration.

[0037] In the context of this invention, the code element to be tested is a line of code, a boolean atomic subexpression, or any combination of both. The code element to be selected in step 21 depends on the coverage metric used in the method, as each coverage metric defines its own code element to be tested. To indicate to the LLM which code element is to be tested, the method of this invention converts the code element to be tested into a string that describes the code element in textual form; this string is the one provided to the LLM. Examples of different types of coverage metrics known in the prior art are shown below: - In instruction or statement coverage metrics, the element to be tested is the line of code of the function. Below is an example of a text string indicating the line of code to be tested to the LLM: “Pending activation of line 16”; In conditional coverage metrics, the code element under test is a boolean subexpression that is to be activated and associated with a conditional statement ("if", "while", "for", "switch", etc.). A boolean subexpression is a boolean expression that can be evaluated as true or false and cannot be further subdivided into simpler boolean expressions. The following is an example of a text string indicating the code element under test to the LLM for conditional coverage metrics: "At line 102 in the condition 'if (a> 10) or (x < a): 'the boolean subexpression (a > 10) remains to be tested in case it evaluates to true'."

[0038] However, the coverage metric used is not intended to limit the invention, and any coverage metric known in the prior art can be used.

[0039] In one embodiment of the invention, the code elements to be tested are obtained from an external program configured to measure the coverage of the function and provide a list of code elements to be tested, the external program using a given coverage metric.

[0040] In the context of this invention, the expression "activate code element" means that the code element is executed or evaluated when the function is executed or simulated. Therefore, when the code element to be tested is a line of code, it means executing the code statements within that line; and when the code element to be tested is a Boolean subexpression, it means evaluating that Boolean subexpression as true or false.

[0041] Each unit test of a function includes a call to the function, in which a value is assigned to each of the function's parameters so that the function can be executed. Therefore, in order to generate each unit test, it is necessary to identify and assign a value to each of the function's parameters so that the unit test can be executed. Thus, once the code element to be tested is selected, the iterative unit test generation step 2 includes an input generation step 22 following the selection step 21, in which initial values ​​for the function's parameters are generated using a large language model or LLM. Figure 2 A flowchart showing details of input generation step 22 is provided.

[0042] LLM, or Large Language Model, is a generative model that can be used to process natural language (NL) content and / or other inputs (such as code) to generate outputs in natural language (NL) and / or other formats (such as code), which are responsive to the inputs provided to it. In other known uses, LLMs can be used to perform tasks, instructions, or operations such that, once performed, they provide the output or result of the task. For this purpose, the task to be performed needs to be provided to the LLM via a natural language text message or prompt. Furthermore, the prompt may provide additional information to perform the task. LLMs are also known to be used to generate accurate answers to questions posed based on large amounts of input data. Several LLMs are currently known, such as OpenAI's GPT-2, GPT-3, or GPT-4, OPT, Meta's LLAMA or Galactica, Google's LaMDA or Gemini, etc. However, the LLM used in this invention is not intended to be limiting; any LLM known in the art that has been trained to operate in an instruction-based mode and is capable of interpreting the source code used to program the function can be used.

[0043] The input generation step 22 includes: an initial prompt generation step 221, wherein an initial prompt is generated based on the source code of the function and the code element to be tested; an interaction step 222 with the LLM, wherein the LLM provides the initial prompt and is requested to return the initial values ​​of the parameters of the function that activates the code element to be tested; and an initial text output receiving step 223, wherein initial text output containing the initial values ​​of the parameters of the function is received from the LLM.

[0044] In the context of this invention, a text message or prompt is an instruction or text input provided to an LLM to guide the behavior of the LLM and its output. Essentially, a text message or prompt provides the LLM with information about the task to be performed and / or the expected response type. Simultaneously, the LLM also generates text output in response to a given text message or prompt.

[0045] By providing the initial prompt to the LLM in step 222 of the interaction with the LLM, the LLM will have information about the source code of the function and which code element is to be tested, and can use it to generate initial text output.

[0046] In step 222 of the interaction with the LLM, the task requested by the LLM to be performed is included as text in the initial prompt, such that the initial prompt includes the task and information provided to the LLM to perform it, namely the source code of the function and the code element to be tested.

[0047] In one embodiment of the method, in step 222 of interacting with the LLM, the LLM is requested to perform the following sequential tasks: - First, the request is to locate the code element to be tested in the source code of the function; - Then, a request is made to analyze how the code element under test can be activated; and Finally, the request is to find a correct value for each parameter of the function that activates the code element under test, which is the initial value of the parameter.

[0048] When the function is simple, LLM can locate a correct initial value for each parameter, which activates the code element under test when unit tests are executed by assigning appropriate initial values ​​to each parameter. However, in complex functions, the initial value of the parameter may not be able to activate the code element under test.

[0049] The iterative unit test generation step 2, following the input generation step 22, includes an iterative verification step sequence 20, which terminates when it is determined that the code element under test has been activated. The iterative verification step sequence 20 includes a simulation step 23, wherein the execution of the function is simulated by assigning a simulated value to each parameter of the function, and a trajectory of the function's execution is generated, the simulated value being initially the initial value for each parameter. The iterative verification step sequence 20 includes an activation check step 24, following the simulation step 23, wherein it is determined whether the code element under test has been activated in the simulation step 23; and an input correction step 25, which is executed after the activation check step 24 if it is determined in the activation check step 24 that the code element under test has not been activated in the simulation step 23.

[0050] The input correction step 25 includes: a correction prompt generation step 251, wherein a correction prompt is generated based on the source code of the function, the code element under test, the simulated values, and the trajectory obtained in the simulation step 23; a step 252 of interacting with the LLM, wherein the LLM provides the correction prompt and is requested to return the correction values ​​of each parameter of the function that activates the code element under test; and a correction text output receiving step 253, wherein a correction text output containing the correction values ​​of each parameter of the function is received from the LLM, such that after receiving the correction text output, the simulated values ​​of each parameter are updated with the corresponding correction values, and the method returns to the simulation step 23. Figure 3 A flowchart showing details of input correction step 25 is provided.

[0051] In one embodiment of this method, in step 222 of interacting with the LLM, the LLM is requested to perform the following task: First, the request is to locate the code element to be tested in the source code of the function. Then, a request is made to analyze how the code element under test can be activated. Then, it requests analysis of the trajectory provided to it in the correction prompt and asks to find out why the code element under test was not activated with simulated values. Next, a solution must be found for the problem that the code element under test was not activated in simulation step 23, and... Finally, it is required to find the value of the parameter that activates the code element under test, which is the correction value of the parameter.

[0052] In the input correction step 25, the task requested from the LLM is included as text in the correction prompt, such that the correction prompt includes the task and information provided to the LLM to execute it, namely the source code of the function, the code element to be tested, the simulated value, and the trajectory obtained in the simulation step 23.

[0053] Each time the method of the present invention is executed for a new code element to be tested, the method attempts in the input generation step 22 to obtain initial values ​​for each parameter of the function such that the initial values ​​are correct values. In the context of the present invention, "correct values" for parameters refer to those values ​​for which the code element to be tested is activating and obtaining those values. If the code element to be tested is activated when the execution of the function is simulated by assigning simulated values ​​equal to the initial values ​​to each parameter, then the input correction step 25 is not executed, and the method proceeds directly to the code generation step 26. However, if the code element to be tested is not activated when the execution of the function is simulated with simulated values ​​equal to the initial values, the method of the present invention executes the input correction step 25 to attempt to obtain corrected values ​​for each parameter that are indeed capable of activating the code element to be tested. For this purpose, and given that the simulation step 23 has already been executed with simulated values ​​equal to the initial values, and an execution trajectory has been obtained, the method provides the trajectory and the simulated values ​​to the input correction step 25, so that the LLM has more information to use to obtain corrected values ​​for the parameters. Once input correction step 25 is completed, the simulated value of each parameter is updated with the corresponding correction value obtained in input correction step 25, and simulation step 23 is executed again to simulate the execution of the function by assigning a corresponding simulated value to each parameter, which has been updated with the correction value obtained in input correction step 25, and an execution trajectory is obtained. If it is determined that the code element under test was not activated in the last execution of simulation step 23, input correction step 25 is executed again, providing it with the simulated value assigned to the parameter and the trajectory obtained in the last execution of simulation step 23.

[0054] In other words, in the first iteration of the iterative verification step sequence 20 for the code element under test, the simulated value assigned to the parameter in the simulation step 23 is the initial value obtained in the input generation step 22, while in the remaining iterations, the simulated value is the corrected value obtained in the input correction step 25 of the previous iteration. However, the simulated value and trajectory provided to the input correction step 25 in a given iteration are the simulated value and trajectory assigned to the parameter in the simulation step 23 of the same iteration of the iterative verification step sequence 20.

[0055] In one implementation, the method has a maximum possible number of iterations for each code element to be tested, such that if the maximum possible number of iterations is reached, the method marks the code element to be tested as an unreachable code element and continues to perform the selection step 21 by selecting another code element to be tested, and thus continues to generate unit tests for the untested code elements.

[0056] The iterative unit test generation step 2 includes a code generation step 26, which is executed after the iterative verification step sequence 20, wherein source code for unit tests of the code element to be tested is generated. Function calls are implemented in the generated source code, wherein values ​​have been assigned to the parameters in the activation check step 24, indicating that the code element to be tested has been activated, so that the function can be executed and can reach the selected code element to be tested in the unit test.

[0057] In one embodiment of this method, when the function is executed or simulated, the function may return one or more output values, such that the output values ​​may be provided as output or return values ​​provided by the function, as values ​​of one or more output parameters of the function, as values ​​of one or more external variables whose values ​​are changed during the execution of the function, and / or as input parameters of one or more additional functions called from the function.

[0058] In one embodiment of this method, the source code of the unit test generated in the code generation step 26 includes a verification block after the function call, in which a verification block checks whether one or more output values ​​returned by the function when the function is executed correspond to one or more output values ​​expected when the function is executed with the values ​​assigned to the parameters in the unit test.

[0059] In one embodiment of the method of the present invention, the source code for the generated unit tests is programmed in Python. However, in other embodiments of the present invention, the source code may be generated using another programming language.

[0060] The iterative unit test generation step 2 includes a code coverage calculation step 27, which follows the code generation step 26, wherein the coverage of the function is calculated. Function coverage is a quality metric that assesses whether the generated unit tests test the code of the function, and the amount of code tested when the generated unit tests are executed. Ultimately, test coverage is a useful tool for identifying untested portions of software source code. It allows monitoring the quality of unit tests and instructs the creation of additional unit tests to test uncovered or untested areas. Different metrics are known for calculating the coverage of a function or a piece of software source code, such as statement coverage, condition coverage, branch coverage, MCDC, etc. However, the coverage metric used in the method of this invention is not intended to constitute a limitation thereto, and any coverage metric known in the prior art can be used.

[0061] In one embodiment of this method, in the code coverage calculation step 27, an external program configured to measure the coverage of a function is invoked, and the program is requested to measure the coverage of the function and provide the coverage measurement results and a list of code elements to be tested.

[0062] Finally, the method of the present invention includes a code coverage check step 28, wherein it is determined whether all code elements of the function have been tested, such that if there are still code elements to be tested in addition to those code elements that have been identified as unreachable in previous iterations of the method, then the selection step 21 of the code elements to be tested is subsequently performed to select another code element to be tested.

[0063] Identifying the correct values ​​for a function's parameters is a mathematically complex task. The method of this invention is characterized by its method of identifying the correct values ​​for these parameters. To this end, the method first interacts with an LLM to obtain initial values, and if these initial values ​​are incorrect, it iterates by requesting a correction value from the LLM until the correct correction value is found. During these iterations, the method provides the LLM with the simulated values ​​used and the trajectory obtained in previous simulations, and requests the LLM to consider the values ​​selected in previous simulations to obtain the values ​​of the parameters that activate the code element under test. The values ​​obtained by utilizing the LLM using the method according to this invention are more similar to the values ​​selected by a human developer than those obtained by other methods known in the art for generating unit tests. This makes unit tests generated by the method of this invention more readable and maintainable than those generated using the other methods.

[0064] In one embodiment of the method of the present invention, the parameters of the function are input parameters of the function, external variables of the program containing the function, or output values ​​of additional functions called from the function.

[0065] Input parameters provide data to a function; that is, they are the function's input variables, through which input data is provided to the function.

[0066] In one embodiment of the invention, the function forms part of a program or software, which may also include additional functions in addition to the function. In this embodiment, the program includes at least one external variable of the function. In the context of the invention, an external variable is a variable declared in the program outside the source code of the function, whose value is accessible from the function. That is, an external variable is a variable defined outside the function body, whose scope is all the functions constituting the program, such that any of these functions can access the external variable to read or write to it. To generate unit tests for the function, it is necessary to identify the correct value for each external variable whose value is accessed by the function, in cases where the function accesses the external variable.

[0067] In one embodiment of the invention, the function calls an additional function, and when the additional function is executed, output data is received from the additional function. In another embodiment of the invention, the additional function forms part of the same program that includes the function. In other embodiments, the additional function may be included in a library accessible from the function. In order to generate unit tests for the function, it is necessary to identify the correct values ​​for the output data provided by the additional function during execution when the function calls the additional function.

[0068] In one embodiment of the method of the present invention, the trajectory generated in simulation step 23 includes: - The lines of code executed in simulation step 23 are arranged in the order of execution; - Lines of code that were not executed in simulation step 23; and - For each executed line of code, in the execution simulation of the function executed in simulation step 23, the value that the parameter referenced in that line of source code has after the execution of that line of code.

[0069] In one embodiment of the invention, the trajectory generated in the simulation step 23 includes: for each executed line of code, if the value returned by the function is calculated in the line of code, the value returned by the function.

[0070] In one embodiment of this method, the trajectory generated in simulation step 23 includes the value of each executed condition in that particular execution. For example, in the case of an "if" statement, the trajectory includes true or false values ​​depending on whether the condition is true or false during the execution of the function.

[0071] Figure 4aAn example of the source code for a function implemented in the Python programming language is shown. The function takes two parameters, namely the function's input parameters "a" and "b". Figure 4b The simulation is shown when the value of parameter "a" is "4" and the value of parameter "b" is "1". Figure 4a An example of the trajectory generated in simulation step 23 during the execution of the function. In this example, lines not executed in the simulation are preceded by the symbol "#", while executed lines are not. The "#" symbol is used in Python to mark comments in code. Furthermore, the code elements to be tested are indicated by the symbol "<--". Figure 4b In this implementation, the code element to be tested is the fifth line of the trajectory. Furthermore, those lines of code that reference parameters are followed by the symbol "#" and the value of each of those parameters at the time of execution, and / or the value returned by the function if that function is evaluated in that line of code, and / or the value of the condition if a condition exists in that line of code.

[0072] In one embodiment of the method of the present invention, the function includes at least one internal variable, and the trajectory includes, for each executed line of code, the value of the internal variable referenced in that line of source code after execution of the function executed in simulation step 23. In the context of the present invention, an internal variable is a variable whose scope is restricted to the function in which it is declared. Internal variables are also called local variables. This means that the internal variable is only accessible within the function, and therefore, the internal variable can only be read from or written to from the function in which it is declared.

[0073] In one embodiment of the method of the present invention, the iterative verification step sequence 20 includes a compression step 30 following the activation check step 24 and preceding the input correction step 25, wherein a compressed trajectory is generated based on the trajectory generated in the simulation step 23, and this compressed trajectory is provided to the input correction step 25. The compressed trajectory is used in place of the trajectory generated in the simulation step 23 in the correction prompt generation step 251.

[0074] Through the compression step 30, the method of the present invention reduces the size of the trajectory to a suitable size that the LLM can manage in the subsequent input correction step 25, making it easier for the LLM to obtain corrected parameter values.

[0075] In one embodiment of the method of the present invention, the compression step 30 includes a loop compression step 31, wherein the trajectory generated in the simulation step 23 is compressed by removing information about loop execution, and a compressed trajectory is obtained; or a trajectory compression step 32. The trajectory compression step 32 includes: a compression hint generation step 321, wherein a compression hint is generated based on the source code of the function, the code element to be tested, the simulated value, and the trajectory generated in the simulation step 23; a step 322 of interacting with an LLM, wherein the LLM provides the compression hint and is requested to identify and remove irrelevant information from the trace; and a compressed text output receiving step 323, wherein compressed text output containing the compressed trajectory is received from the LLM.

[0076] Figure 5 A second embodiment of the method is shown, wherein the compression step 30 includes a cyclic compression step 31, wherein the trajectory generated in the simulation step 23 is compressed by removing information about the cyclic execution, and a compressed trajectory is obtained. That is, in this embodiment of the method, in each iteration of the iterative verification step sequence 20, the trajectory generated in the simulation step 23 of the same iteration is compressed in the cyclic compression step 31.

[0077] Figure 6a This example shows the trajectory generated in simulation step 23 when the simulation executes a function named "sum_values". This function takes one argument, the input parameter "values", which is an array of five numbers. Additionally, the function has two internal variables named "sum" and "i". In this trajectory, ... Figure 4b Similar to the trajectory shown, lines not executed in the simulation are preceded by the symbol "#", while executed lines are not. Furthermore, code elements under test are indicated by the symbol "<--". Additionally, lines of code that reference parameters or internal variables of the function, or those that compute the value returned by the function, are followed by the symbol "#" and the value of each parameter and / or internal variable, and / or the value returned by the function once that line of code is executed. The "sum_values" function has an iterative structure, namely a "for" loop that executes the same number of times as the values ​​in the array of "values" parameters. Figure 6a The implementation of the trajectory shown illustrates each iteration of the iterative structure. Figure 6b This shows the cyclic compression step 31 according to Figure 6aAn example of a compressed trajectory generated from the trajectory, wherein lines of code corresponding to intermediate iterations of the "for" loop of the function have been removed, lines of code corresponding to the first and last iterations of the "for" loop have been retained, and those lines corresponding to intermediate iterations have been replaced with ellipses.

[0078] In the cyclic compression step 31, repeating patterns are identified in the rows of the trajectory. Through cyclic compression, the method of the present invention reduces the size of the trajectory to a suitable size that the LLM can manage in the subsequent input correction step 25, making it easier for the LLM to obtain corrected parameter values.

[0079] To avoid the simulation step 23 simulating the function indefinitely in the event of an infinite loop, in one embodiment of the method of the invention, a predetermined maximum trajectory size is set in the simulation step 23, such that if the trajectory generated in the simulation step 23 exceeds the predetermined maximum trajectory size, the simulation step 23 ends and proceeds to the activation check step 24, wherein it is determined whether the code element under test has been activated based on the incomplete trajectory generated in the simulation step 23. The method of the invention is particularly advantageous in the event of an infinite loop in simulation step 23 because it allows the trajectory generated by the infinite loop to be compressed to a suitable size that the LLM can manage in the subsequent input correction step 25. Furthermore, it allows the presence of the infinite loop to be identified in the generated compressed trajectory, and when this compressed trajectory is provided to the input correction step 25, the LLM can analyze the reason for entering the infinite loop in the previous simulation step 23 and find correction values ​​for parameters to avoid infinite loops in subsequent simulations.

[0080] Figure 7 A third embodiment of the method of the present invention is shown, wherein the compression step 30 includes a trajectory compression step 32, wherein the trajectory generated in the simulation step 23 is compressed. Figure 8 A flowchart detailing the trajectory compression step 32 is shown. In each iteration of the iterative verification step sequence 20, the compression prompt includes simulating the trajectory generated in step 23 within the same iteration. This trajectory is provided to the LLM in step 322, where the interaction with the LLM occurs, enabling the LLM to identify and remove irrelevant information from the trajectory. The compression prompt further includes a task requested from the LLM in the trajectory compression step 32.

[0081] In one embodiment of this method, in step 322 of interacting with the LLM, the LLM is requested to perform the following task: First, the request is to identify non-relevant information in the trajectory, and Then, a request is made to remove the non-relevant information from the trajectory.

[0082] Through the trajectory compression step 32, the method of the present invention uses LLM to compress the trajectory by removing irrelevant information. Therefore, it reduces the size of the trajectory to a suitable size that the LLM can manage in the subsequent input correction step 25, making it easier for the LLM to obtain corrected parameter values.

[0083] exist Figure 9 In a fourth embodiment of the method of the present invention shown, the compression step 30 includes a loop compression step 31, wherein the trajectory generated in the simulation step 23 is compressed by removing information about loop execution in the source code to obtain an intermediate compressed trajectory; and a trajectory compression step 32. The trajectory compression step 32 includes: a compression hint generation step 321, wherein a compression hint is generated based on the source code of the function, the code element to be tested, the simulated value, and the intermediate compressed trajectory; a step 322 of interacting with the LLM, wherein the LLM provides the compression hint and is requested to identify and remove irrelevant information from the intermediate compressed trajectory; and a compressed text output receiving step 323, wherein compressed text output containing the compressed trajectory is received from the LLM, and the compressed trajectory is provided to the input correction step 25. In an implementation of the method, in each iteration of the iterative verification step sequence 20, the trajectory generated in the simulation step 23 of the same iteration is compressed in the cyclic compression step 31 to obtain an intermediate trajectory, and the intermediate trajectory generated in the cyclic compression step 31 of the same iteration is compressed in the trajectory compression step 32 to obtain a compressed trajectory provided to the input correction step 25.

[0084] In another embodiment of the computer-implemented method of the present invention, the iterative verification step sequence 20 includes an equation management step 33 between the activation check step 24 and the input correction step 25.

[0085] The equation management step 33 includes an equation generation step 331, which includes an equation generation prompt generation step 3311, wherein the equation generation prompt is generated based on the source code of the function, the code element to be tested, the simulated value, and the received trajectory.

[0086] The equation generation step 331 includes a step 3312 of interaction with the LLM following the equation generation prompt generation step 3311, wherein the LLM provides the equation generation prompt and is requested to select one of the parameters of the function, identify multiple equations defining the values ​​of the parameters based on the values ​​of the other parameters, and generate source code for a script. This source code is provided to an external tool configured to solve mathematical equations, allowing the calculation of the values ​​of the selected parameters. The equation generation step 331 includes an equation generation text output receiving step 3313 following the LLM interaction step 3312, wherein equation generation text output containing the source code of the script is received from the LLM. The script includes multiple instructions for solving the multiple equations. The instructions are adapted to be executed by the external tool.

[0087] The equation management step 33 includes an equation solving step 332 following the equation generation step 331, wherein the external tool provides the source code of the script generated in the equation generation step 331 and is requested to execute the script.

[0088] The equation management step 33 includes a result receiving step 333 located after the equation solving step 332, wherein the result obtained when executing the script is received from the external tool.

[0089] The equation management step 33 includes a report generation step 334 following the result receiving step 333. The report generation step 334 includes a report generation prompt generation step 3341, wherein a report generation prompt is generated based on the source code of the function, the code element under test, the simulated value, the received trajectory, the source code of the script, and the results received from the external tool. The report generation step 334 includes a step 3342 following the report generation prompt generation step 3341, involving interaction with the LLM, where the LLM provides the report generation prompt and is requested to generate a report on the results obtained from the external tool, and a report generation text output receiving step 3343, where a report generation text output containing the report is received from the LLM, such that in the input correction step 25, the correction prompt is also generated based on the report.

[0090] Figure 10A fifth embodiment of the computer-implemented method of the present invention is shown, wherein the equation management step 33 is performed between the activation check step 24 and the input correction step 25, that is, after the compression step 30 and before the input correction step 25. In this fifth embodiment, the compression step 30 includes the cyclic compression step 31 and the trajectory compression step 32, the trajectory compression step 32 being performed after the cyclic compression step 31, and the equation management step 33 being performed after the trajectory compression step 32. In this embodiment, the trajectory received in the equation management step 33 is the compressed trajectory generated in the compression step 30, specifically the compressed trajectory generated in the trajectory compression step 32.

[0091] In another embodiment of this method, the equation management step 33 is executed after the compression step 30, which includes only the cyclic compression step 31. In this embodiment, the trajectory received in the equation management step 33 is the compressed trajectory generated in the cyclic compression step 31.

[0092] In another embodiment of this method, the equation management step 33 is performed after the compression step 30, which includes only the trajectory compression step 32. In this embodiment, the trajectory received in the equation management step 33 is the compressed trajectory generated in the trajectory compression step 32.

[0093] In another embodiment of this method, the equation management step 33 is performed after the activation check step 24, and the trajectory received in the equation management step 33 is the trajectory generated in the simulation step 23. The equation management step 33 may or may not be followed by a compression step 30.

[0094] In one embodiment of this method, in step 3342 of interacting with the LLM, the LLM is requested to perform the following task: First, request analysis of the received trajectory. Next, we request an analysis of the results provided by the external tool, and - Then, request that a report be written on the results.

[0095] In the report generation step 334, the tasks requested from the LLM are included as text in the report generation prompt, such that the report generation prompt includes those tasks, as well as information provided to the LLM to execute them, namely the source code of the function, the code element to be tested, the simulated values, the received trajectory, the source code of the script, and the results received from the external tool. Figure 11A flowchart showing details of equation management step 33 is provided.

[0096] In one embodiment of this method, in the equation solving step 332, the external tool provided to it with the source code of the script generated in the equation management text output receiving step 3313 is Mathematica, Matlab, SageMath, or another tool known in the art that is configured to solve mathematical equations.

[0097] The method of this invention is particularly advantageous for creating unit tests for complex functions, where the value of one parameter of the function is mathematically dependent on other parameters or internal variables of the function, and where complex calculations are required to find the correct value for the parameter. The method of this invention uses an LLM to generate a script with equations to be solved, such that the script can be solved in the external tool. Furthermore, once the results of the script being executed in the external tool are received, the method of this invention again uses an LLM to generate a report with the obtained results, which is then provided to the input correction step 25, allowing the LLM to also use the information in the report to obtain corrected values ​​for the function's parameters. For this purpose, the LLM used in the method of this invention must support the programming language used to generate the requested script; that is, it must be able to generate scripts suitable for execution in the external mathematical problem-solving tool used by the method.

[0098] Figure 12a An example of the trajectory provided to the equation generation prompt for step 3311 is shown. The trajectory indicates that the code element to be tested is line 5 of the trajectory, which will be executed if the value of parameter "x" is less than or equal to the value of parameter "y".

[0099] Figure 12b This is illustrated in step 3313 of the equation management text output receiving process, when the LLM is provided in the equation generation prompt. Figure 12a The example given is a Python script generated by LLM using the SageMath library, used to calculate the trajectory. In this example, LLM selects the parameter "x" and identifies multiple equations from which the value of "x" is obtained, represented as a function of the other parameters (in this example, parameter "y"). Next, LLM generates a trajectory like... Figure 12b The source code of the script shown allows the calculation of the value of parameter "x" when provided to external tools. In other embodiments of the invention, the script can be generated using other programming languages.

[0100] When Figure 12bWhen a script is provided to a Python interpreter with SageMath and the interpreter is asked to run the script, the Python interpreter returns the result of the parameter "x", such as "x <= 1".

[0101] Figure 13a An example of the trajectory provided to the equation generation prompt for step 3311 is shown. The trajectory indicates that the code element to be tested is line 3 in the function's source code, which will be executed if the value of the parameter "s" equals the number of seconds from January 1, 1970, to a certain Monday.

[0102] Figure 13b This is illustrated in step 3313 of the equation management text output receiving process, when the LLM is provided in the equation generation prompt. Figure 13a The example of a Python script generated by LLM is shown here. In other embodiments of the invention, the script can be generated using other programming languages.

[0103] By Figure 13b The script is provided to the Python interpreter and the Python interpreter is requested to execute the script. The Python interpreter returns the result obtained when the script is executed, which will be the number of seconds corresponding to a certain Monday.

[0104] In one embodiment of the method of the present invention, in the equation generation hint generation step 3311, the equation generation hint is also generated based on the parameter selected in the previous iteration of the iterative verification step sequence 20 for the same function and the same code element to be tested, in the equation generation step 3311, and wherein, in the step 3312 of interacting with the LLM, the LLM is requested to select a parameter different from the parameter selected in the previous iteration.

[0105] In complex functions, where more than one parameter of the function can be mathematically represented as the value of the other parameters, the method of the present invention is particularly advantageous because it is not limited to solving equations that express one parameter as a function of the other parameters, but rather allows for the experimentation with a different parameter in subsequent iterations when the results obtained from external tools cannot generate the correct value for that parameter.

[0106] In another embodiment of the computer-implemented method of the present invention, the iterative verification step sequence 20 includes a condition chain extraction step 34 between the activation check step and the input correction step 25.

[0107] In the condition chain extraction step 34, a condition chain that must be executed until the code element under test is reached is obtained from the received trajectory, such that the correction prompt in the input correction step 25 is also generated based on the condition chain. The condition chain includes the source code of the function and the conditions that must be evaluated as true during the execution of the function in order to execute the code element under test. Figure 16a An example of the trajectory is shown, and Figure 16b It shows from Figure 16a The trajectory originates from the example of the conditional chain obtained in step 34 of the conditional chain extraction.

[0108] In one embodiment of this method, in the condition chain extraction step 34, the first unexecuted condition in the chain is also obtained, so that in the input correction step 25, the correction prompt is also generated based on the first unexecuted condition. Figure 16c It shows from Figure 16a Starting from the trajectory, the first instance of the unexecuted condition is obtained in step 34 of the condition chain extraction.

[0109] Figure 15 A sixth embodiment of the computer-implemented method of the present invention is shown, wherein the condition chain extraction step 34 is executed after the equation management step 33, and the equation management step 33 is further executed after the compression step 30, the compression step 30 including the loop compression step 31 and the trajectory compression step 32. In this embodiment, the trajectory received at the condition chain extraction step 34 is the compressed trajectory generated in the compression step 30, i.e., the compressed trajectory generated in the trajectory compression step 32. In another embodiment of the method, the condition chain extraction step 34 is executed after the equation management step 33, and the equation management step 33 is further executed after the compression step 30, the compression step 30 including only the loop compression step 31. In this embodiment, the trajectory received in the condition chain extraction step 34 is the compressed trajectory generated in the loop compression step 31. In another embodiment of the method, the condition chain extraction step 34 is executed after the equation management step 33, and the equation management step 33 is further executed after the compression step 30, the compression step 30 including only the trajectory compression step 32. In the embodiment described above, the trajectory received in the conditional chain extraction step 34 is the compressed trajectory generated in the trajectory compression step 32.

[0110] In another embodiment of the invention, the conditional chain extraction step 34 is executed after the compression step 30, and the compression step 30 includes the cyclic compression step 31 and the trajectory compression step 32. In this embodiment, the trajectory received in the conditional chain extraction step 34 is the compressed trajectory generated in the compression step 30, i.e., the compressed trajectory generated in the trajectory compression step 32. In another embodiment of the method, the conditional chain extraction step 34 is executed after the compression step 30, and the compression step 30 includes only the cyclic compression step 31. In this embodiment, the trajectory received in the conditional chain extraction step 34 is the compressed trajectory generated in the cyclic compression step 31. In another embodiment of the method, the conditional chain extraction step 34 is executed after the compression step 30, and the compression step 30 includes only the trajectory compression step 32. In this embodiment, the trajectory received in the conditional chain extraction step 34 is the compressed trajectory generated in the trajectory compression step 32.

[0111] In another embodiment of the method, the condition chain extraction step 34 is located after the activation check step 24, and the trajectory received in the condition chain extraction step 34 is the trajectory generated in the simulation step 23. There may or may not be an equation management step 33 before or after the condition chain extraction step 34.

[0112] In at least one embodiment, a computer 500 or computing device 500 for implementing the method of the present invention may include one or more processors 502, one or more memory elements 504, storage device 506, bus 508, one or more network processor units 510 interconnected with one or more network input / output (I / O) interfaces 512, one or more I / O interfaces 514, and a computer program. Figure 14 A schematic diagram of an embodiment of a computer 500 configured to implement the method of the present invention is shown.

[0113] The computer program includes a plurality of instructions that, when executed by the processor 502, cause the processor 502 to perform the steps of the method of the present invention. In at least one embodiment, one or more of the processors 502 are at least hardware processors configured to perform various tasks, operations, and / or functions for the computer 500 according to software and / or instructions (e.g., instructions in a computer program) configured for the computer 500.

[0114] In at least one embodiment, the memory element 504 and / or storage device 506 are configured to store data, information, software, and / or instructions associated with the computer 500, and / or are configured for logic of the memory element 504 and / or storage device 506. In one embodiment of the computer 500, the computer program and the data, information, software, and / or instructions associated with the computer program are stored in any combination of the memory element 504 and / or storage device 506.

[0115] In one embodiment of the method of the present invention, the function is stored in memory element 504 and / or storage device 506.

[0116] In one embodiment of the method of the present invention, the source code of the unit test generated for the code element to be tested of the function is stored in memory element 504 and / or storage device 506.

[0117] In at least one embodiment, the bus 508 can be configured as an interface that allows one or more components of the computer 500 to communicate with each other to exchange information and / or data. The bus 508 can be implemented using any architecture designed to exchange control, data, and / or information between processors, memory / storage elements, peripheral devices, and / or any other hardware and / or software components that can be configured for the computer 500. In at least one embodiment, the bus 508 can be implemented as a fast interconnect housed in a core, possibly using shared memory between processes (e.g., logic), which can allow for efficient communication paths between processes.

[0118] In several embodiments, the network processor unit 510 enables communication between the computer 500 and other systems, entities, etc., via the network I / O interface 512 (wired and / or wireless). In several embodiments, the network processor unit 510 may be configured as a combination of hardware and / or software, such as one or more Ethernet controllers and / or interface controllers or cards, drivers and / or Fibre Channel (e.g., optical) controllers, wireless receivers / transmitters / transceivers, baseband processors / modems, and / or other similar network interface drivers and / or controllers now known or potentially developed in the future, to allow the computer 500 to communicate with other systems, entities, etc., thereby facilitating the operation of various embodiments of the methods described herein. In several embodiments, the network I / O interface 512 may be configured as one or more Ethernet ports, Fibre Channel ports, any other I / O ports now known or potentially developed in the future, and / or antennas / antenna arrays. Therefore, the network processor unit 510 and / or the network I / O interface 512 may include interfaces suitable for receiving, transmitting, and / or otherwise communicating data and / or information in a network environment.

[0119] I / O interface 514 allows for the input and output of data and / or information with other entities that may be connected to computer 500. For example, I / O interface 514 can provide connectivity to external devices such as keyboards, keypads, touchscreens, and / or any other suitable input and / or output devices now known or potentially developed in the future. External devices may also include non-transitory computer-readable storage media such as database systems, USB drives, portable optical discs or disks, and memory cards. In some cases, external devices may also be mechanisms for displaying data to a user, such as computer monitors, displays, etc.

[0120] In several embodiments, the computer program may include instructions that, when executed, cause one or more processors 502 to perform operations, which may in particular include: providing general control operations for the computer 500, interacting with other entities, systems, etc. described herein, maintaining and / or interacting with stored data, information, parameters, etc. (e.g., memory elements, storage devices, data structures, databases, tables, etc.), and combinations thereof, to allow the execution of operations necessary to implement the methods of the present invention.

[0121] In some cases, the computer program of this embodiment may be obtained through a non-transitory computer-readable storage medium (e.g., magnetic or optical media, magneto-optical media, CD-ROM, DVD, storage device, etc.). In some cases, the non-transitory computer-readable storage medium may also be removable. Other examples may include optical discs and magnetic disks, USB storage devices, and smart cards, which can be inserted into and / or otherwise connected to a computer for transfer to another computer-readable storage medium.

[0122] In one implementation, the computer program is executed in a distributed environment, wherein a portion of the computer program runs on a first computer 500 and another portion of the computer program runs on a second computer 500, and the two computers 500 communicate with each other.

[0123] In one embodiment, the LLM is executed in computer 500. However, in other embodiments of this method, the LLM may be executed in another computer communicating with computer 500.

[0124] In one embodiment, the external program configured to measure the coverage of a function is executed in computer 500. However, in other embodiments of this method, the external program may be executed in another computer communicating with computer 500.

[0125] In one embodiment, the external tool configured to solve mathematical equations is executed in computer 500. However, in other embodiments of this method, the external tool may be executed in another computer communicating with computer 500.

[0126] In a second aspect, the present invention also relates to a non-transitory computer-readable storage medium storing a plurality of computer-readable instructions which, when executed by a computer 500, cause the computer 500 to perform the steps of the method of the present invention.

[0127] In a third aspect, the present invention also relates to a computer 500, the computer 500 including a processor 502 and a computer-readable storage medium operatively coupled to the processor 502, the computer-readable storage medium having a plurality of computer-readable instructions stored therein, which, when executed by the processor 502, cause the computer 500 to perform the steps of the method of the present invention.

[0128] In a fourth aspect, the present invention also relates to an automated software testing system 600 based on a hardware-in-the-loop (HIL) architecture. The automated software testing system 600 verifies the correct operation of the program and / or function of an electronic control unit 400 (e.g., an ECU of a vehicle). The automated software testing system 600 includes multiple sensors 200 or input devices, and multiple actuators 300 or devices configured to perform actions. It also includes an electronic control unit 400 communicating with the sensors 200 and actuators 300, wherein the program or function to be tested is executed. The automated software testing system 600 further includes multiple actuators 300', multiple sensors 200', and a computer 500 of the present invention, which communicates with the sensors 200', actuators 300', and electronic control unit 400. Each actuator 300' is configured to generate the necessary environment for the associated sensor 200, such that the sensor 200 provides a certain value to the electronic control unit 400. Similarly, the sensor 200' is configured to detect actions performed by the associated actuator 300, convert the detected actions into values, and provide them to the computer 500.

[0129] The automated software testing system 600 further includes an electrical and network system configured to connect an electronic control unit 400 to a sensor 200, an actuator 300, and a computer 500, and to connect the computer 500 to the actuator 300' and the sensor 200'. Figure 17 An embodiment of the automated software testing system according to the present invention is shown.

[0130] The computer 500 of the present invention is configured to generate unit tests of the function, wherein each parameter of the function has an associated value in each unit test. For each unit test, the computer 500 is configured to communicate with an actuator 300' such that a sensor 200 associated with each parameter of the function provides the value of that parameter in the unit test to an electronic control unit 400. The computer 500 of the present invention is also configured to request the electronic control unit 400 to execute the function. The electronic control unit 400 is configured to read the value of each sensor 200 and use the value of the sensor 200 as the value of the function's parameters to execute the function. Once the function has been executed, the electronic control unit 400 is configured to send a signal to each corresponding actuator 300, the signal representing the output value of the function associated with that actuator. The computer 500 is also configured to communicate with the sensors 200' and receive a value from each sensor 200' corresponding to the output value of the function.

[0131] In one embodiment of the automated software testing system 600, the computer 500 is configured to verify whether the value received from the sensor 200' (which represents the output of the function) corresponds to the expected output value of the function.

[0132] In one embodiment of the automated software testing system 600 of the present invention, the sensor 200 may also be a digital input, which may be provided to an electronic control unit or via a CAN bus, etc. In another embodiment of the automated software testing system 600 of the present invention, the actuator may also be a digital output, which may be provided via a CAN bus, etc.

Claims

1. A computer-implemented method for unit testing of generated functions, the computer-implemented method comprising the following steps: - Receiving step (1), wherein the source code of the function is received, the function including at least one parameter and multiple lines of code, and - Iterative unit test generation step (2), wherein multiple unit tests are generated for the function, and the iterative unit test generation step (2) includes: - Selection step (21), wherein, the code element to be tested is selected from the source code, and - Input generation step (22), wherein initial values ​​are generated for each parameter of the function. The characteristic feature is that the input generation step (22) includes: - Initial hint generation step (221), wherein an initial hint is generated based on the source code of the function and the code element to be tested. - Step (222) of interacting with the LLM, wherein the LLM provides the initial prompt and is requested to return the initial values ​​of the parameters of the function that activates the code element under test, and - Initial text output receiving step (223), wherein initial text output containing the initial values ​​of the parameters of the function is received from the LLM. Furthermore, the iterative unit test generation step (2) further includes: - An iterative verification step sequence (20) following the input generation step (22), which terminates when it is determined that the code element to be tested has been activated, the iterative verification step sequence (20) includes: - Simulation step (23), wherein the execution of the function is simulated by assigning simulation values ​​to each parameter, and the trajectory of the function execution is generated, wherein the simulation values ​​are initially the initial values. - An activation check step (24) following the simulation step (23) determines whether the code element to be tested was activated in the simulation step (23), and - Input correction step (25): If it is determined in the activation check step (24) that the code element to be tested was not activated in the simulation step (23), then the input correction step (25) is performed after the activation check step (24), the input correction step (25) including: - Correction hint generation step (251), wherein correction hints are generated based on the source code of the function, the code element to be tested, the simulated value, and the trajectory obtained in the simulation step (23). - Step (252) of interacting with the LLM, wherein the LLM provides the correction prompt and is requested to return the correction values ​​of the parameters of the function that activates the code element under test, and - Correction text output receiving step (253), wherein, correction text output containing correction values ​​of each parameter of the function is received from the LLM. This ensures that after receiving the correction text output, the simulated values ​​of each parameter are updated with the corresponding correction values, and the method returns to the simulation step (23). - A code generation step (26) is performed after the iterative verification step sequence, wherein source code for the unit tests of the code element to be tested is generated. - The code coverage calculation step (27) following the code generation step (26) includes calculating the coverage of the function, and - Code coverage check step (28), wherein it is determined whether all code elements of the function have been tested, such that if there are still any code elements to be tested, then the selection step (21) is performed to select another code element to be tested.

2. The computer-implemented method according to claim 1, wherein, The parameters of the function are: - The input parameters of the function, - External variables of the program including the function, or - The output value of the additional function called from the function mentioned.

3. The computer-implemented method according to claim 1 or 2, wherein, The trajectory includes: - The lines of code executed in the simulation step (23) are arranged in the order of execution. - Lines of code not executed in the simulation step (23), and - For each executed line of code, in the execution simulation of the function executed in the simulation step (23), the parameter referenced in the line of code has a value after the line of code is executed.

4. The computer-implemented method according to claim 3, wherein, The function includes at least one internal variable, and the trajectory includes: for each executed line of code, in the execution simulation of the function executed in the simulation step (23), the value of the internal variable referenced in the line of code after the execution of the line of code.

5. The computer-implemented method according to any one of the preceding claims, wherein, The iterative verification step sequence (20) includes a compression step (30) after the activation check step (24) and before the input correction step (25), wherein a compressed trajectory is generated based on the trajectory generated in the simulation step (23), and the compressed trajectory is provided to the input correction step (25).

6. The computer-implemented method according to claim 5, wherein, The compression step (30) includes: - Cyclic compression step (31), wherein the trajectory generated in the simulation step (23) is compressed by removing information about the cyclic execution, and the compressed trajectory is obtained, or - Trajectory compression step (32), the trajectory compression step (32) includes: - Compression hint generation step (321), wherein compression hints are generated based on the source code of the function, the code element to be tested, the simulated value, and the trajectory generated in the simulation step (23). - Step (322) of interacting with the LLM, wherein the LLM provides the compression prompt and is requested to identify and remove irrelevant information from the trajectory, and - Compressed text output receiving step (323), wherein compressed text output containing the compression trajectory is received from the LLM.

7. The computer-implemented method according to claim 5, wherein, The compression step (30) includes: - Cyclic compression step (31), wherein the trajectory generated in the simulation step (23) is compressed by removing information about the cyclic execution in the source code, and an intermediate compressed trajectory is obtained, and - Trajectory compression step (32), the trajectory compression step (32) includes: - Compression hint generation step (321), wherein compression hints are generated based on the source code of the function, the code element to be tested, the simulated value, and the intermediate compression trajectory. - Step (322) of interacting with the LLM, wherein the LLM provides the compression hints and is requested to identify and remove irrelevant information from the intermediate compression tracks, and - Compressed text output receiving step (323), wherein compressed text output containing the compression trajectory is received from the LLM.

8. The computer-implemented method according to any one of the preceding claims, wherein, The iterative verification step sequence (20) includes an equation management step (33) between the activation check step (24) and the input correction step (25), wherein the equation management step (33) includes: - Equation generation step (331), which includes: - Equation generation hint generation step (3311), wherein equation generation hints are generated based on the source code of the function, the code element to be tested, the simulated value, and the received trajectory. - Step (3312) of interacting with the LLM, wherein the LLM provides the equation generation prompts and is asked to select one of the parameters of the function, and identifies multiple equations that define the values ​​of the parameters based on the values ​​of the other parameters, and generates source code for a script, the source code of which is provided to an external tool configured to solve mathematical equations, the source code allowing the calculation of the values ​​of the selected parameters, and - Equation generation text output receiving step (3313), wherein the equation generation text output containing the source code of the script is received from the LLM. - Equation solving step (332), wherein the external tool provides the source code of the script generated in the equation generation step (331) and is requested to execute the script. - Result receiving step (333), wherein the result obtained when executing the script is received from the external tool, and - Report generation step (334), which includes: - Report generation prompt generation step (3341), wherein a report generation prompt is generated based on the source code of the function, the code element to be tested, the simulated value, the received trajectory, the source code of the script, and the result received from the external tool. - Step (3342) of interacting with the LLM, wherein the LLM provides the report generation prompt and is requested to generate a report on the results obtained from the external tool, and - Report generation text output receiving step (3343), wherein a report generation text output containing the report is received from the LLM, such that in the input correction step (25), the correction prompt is also generated based on the report.

9. The computer-implemented method according to claim 8, wherein, In the equation generation hint generation step (3311), the equation generation hint is also generated based on the parameter selected in the equation generation step (331) for the same function and the same code element to be tested in the previous iteration of the iterative verification step sequence (20), and wherein, in the step (3312) of interacting with the LLM, the LLM is requested to select a parameter different from the parameter selected in the previous iteration.

10. The computer-implemented method according to any one of the preceding claims, wherein, The iterative verification step sequence (20) includes a condition chain extraction step (34) between the activation check step (24) and the input correction step (25), wherein, starting from the received trajectory, a condition chain that must be executed until the code element to be tested is obtained, such that in the input correction step (25), the correction prompt is also generated based on the condition chain.

11. The computer-implemented method according to claim 10, wherein, In the condition chain extraction step (34), the first unexecuted condition in the condition chain is also obtained, so that in the input correction step (25), the correction prompt is generated based on the first unexecuted condition.

12. A non-transitory computer-readable storage medium having stored thereon a plurality of instructions which, when executed by a computer (500), cause the computer (500) to perform the method steps according to any one of claims 1 to 11.

13. A computer (500) comprising a processor (502) and a non-transitory computer-readable storage medium operatively coupled to the processor (502), the non-transitory computer-readable storage medium storing thereon a plurality of instructions which, when executed by the processor (502), cause the computer (500) to perform the method steps according to any one of claims 1 to 11.

Images