An industrial software test case generation method and device based on a large language model

Abstract

The application discloses an industrial software test case generation method and device based on a large language model, which comprises the following steps: acquiring a prompting strategy and a knowledge base, wherein the knowledge base is used for storing background description information of industrial software test case generation; according to the prompting strategy, guiding the large language model to generate test cases according to the knowledge base, and outputting corresponding first test cases; acquiring a system function description model; using the large language model to optimize the first test cases according to the system function description model, and generating second test cases; acquiring test case deployment information, using the large language model to perform code conversion processing on the second test cases according to the test case deployment information and the system function description model, and generating corresponding test case execution files. The application can realize automatic generation of test cases, improve the generation efficiency of test cases, improve the quality of test cases, meet the test requirements of industrial software systems, and can be widely applied to the field of artificial intelligence technology.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and apparatus for generating industrial software test cases based on a large language model. Background Technology

[0002] With the increasing complexity of software systems and the shortening of development cycles, traditional manual test case construction methods can no longer meet the testing needs of modern software systems. Automated testing, as an effective solution, can significantly improve testing efficiency and ensure software quality. However, existing test cases are almost entirely constructed manually, consuming a significant amount of time and resulting in low efficiency in test case generation. Furthermore, manually constructed test cases tend to be of low quality and cannot meet the requirements of automated software system testing. Summary of the Invention

[0003] The main objective of this application is to propose a method and apparatus for generating industrial software test cases based on a large language model, which can achieve automated test case generation and improve the efficiency of test case generation.

[0004] On the one hand, embodiments of this application propose a method for generating industrial software test cases based on a large language model, the method comprising the following steps:

[0005] Acquire prompting strategies and a knowledge base, wherein the knowledge base is used to store background description information for the generation of industrial software test cases;

[0006] According to the prompting strategy, the large language model is guided to generate test cases based on the knowledge base and output the corresponding first test case;

[0007] Obtain the system functional description model;

[0008] Using the large language model and the system function description model, the first test case is optimized to generate the second test case;

[0009] Obtain test case deployment information, and use the large language model to perform code conversion processing on the second test case based on the test case deployment information and the system function description model to generate the corresponding test case execution file.

[0010] In some embodiments, the acquisition of the prompting strategy and knowledge base specifically includes:

[0011] Obtain the background description information, which includes at least system function information, system application programming interface, system parameter description information, system interaction principle information, and role definition information;

[0012] Construct the knowledge base and store the background description information in the knowledge base;

[0013] Obtain test case generation prompts, which include at least the role information of the current generation environment, test case format constraints, and a sample template for the test case.

[0014] Based on the test cases, generate prompt information and the knowledge base, and generate the prompt strategy.

[0015] In some embodiments, obtaining the system functional description model specifically includes:

[0016] Obtain system function description information; the system function description information includes description information of multiple system functions and description information of the interaction flow of each system component when executing each system function;

[0017] Based on the system function description information, mathematical modeling is performed to establish the corresponding system function description model.

[0018] In some embodiments, optimizing the first test case using the large language model based on the system function description model to generate a second test case specifically includes:

[0019] The system function description model and the first test case are input into the large language model, and the large language model is used to determine the test sequence data that matches the first test case from the system function description model;

[0020] Get the test path correction prompt;

[0021] The test sequence data, the first test case, and the test path correction prompt are input into the large language model. The first test case is corrected using the large language model, and the corrected first test case is output. The corrected first test case is then used as the second test case.

[0022] In some embodiments, the step of inputting the test sequence data, the first test case, and the test path correction prompt into the large language model, using the large language model to correct the first test case, outputting the corrected first test case, and obtaining the corrected first test case as the second test case specifically includes:

[0023] Using the large language model, the prompt words are corrected according to the test path. The test process of the first test case and the test sequence data is compared to determine the process comparison result.

[0024] When the process comparison result indicates that the test process is inconsistent, the first test case is corrected based on the test sequence data using the large language model, and the second test case is output.

[0025] In some embodiments, obtaining test case deployment information and using the large language model to perform code conversion processing on the second test case based on the test case deployment information and the system function description model to generate a corresponding test case execution file specifically includes:

[0026] Retrieve the code generation template knowledge base and code generation prompts corresponding to the test cases;

[0027] The test sequence data, the code generation template knowledge base, the code generation prompts, and the second test case are input into the large language model. The large language model is then used to output the test case execution file corresponding to the second test case. The test case execution file includes the test case name, test case description, test interface address, interface input parameters, the values ​​of the interface input parameters, test case code, and test case script execution code.

[0028] In some embodiments, the method further includes:

[0029] Construct the large language model;

[0030] The large language model is pre-trained using the knowledge base, which also includes a few-shot learning dataset.

[0031] On the other hand, embodiments of this application propose an industrial software test case generation device based on a large language model, the device comprising:

[0032] The first module is used to acquire prompting strategies and a knowledge base, wherein the knowledge base is used to store background description information for the generation of industrial software test cases;

[0033] The second module is used to guide the large language model to generate test cases based on the knowledge base according to the prompting strategy, and output the corresponding first test cases.

[0034] The third module is used to obtain the system functional description model;

[0035] The fourth module is used to optimize the first test case based on the system function description model using the large language model, and generate the second test case.

[0036] The fifth module is used to obtain test case deployment information, and use the large language model to perform code conversion processing on the second test case based on the test case deployment information and the system function description model to generate the corresponding test case execution file.

[0037] The embodiments of this application include at least the following beneficial effects: This application provides a method and apparatus for generating industrial software test cases based on a large language model. It acquires a prompting strategy, a knowledge base, a system functional description model, and test case deployment information. Based on the prompting strategy, it guides the large language model to output a corresponding first test case according to the knowledge base. The large language model then optimizes the first test case based on the system functional description model to generate a second test case. Finally, based on the test case deployment information and the system functional description model, the large language model performs code conversion processing on the second test case to generate a corresponding test case executable file. This application can achieve automated test case generation through a large language model, improving the efficiency and quality of test case generation and meeting the testing needs of industrial software systems. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of a method for generating industrial software test cases based on a large language model, provided in an embodiment of this application.

[0041] Figure 2 This is a schematic diagram of the structure of an industrial software test case generation device based on a large language model provided in an embodiment of this application;

[0042] Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0044] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0045] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0047] In existing technologies, implementation schemes for test case generation can be roughly divided into three types: The first is the traditional manual writing of test cases, which relies on fixed templates and the experience of test engineers. Although it can cover basic testing requirements to a certain extent, it suffers from low efficiency, easy omissions, and repetitions. The second is to use large language models to generate test cases and automatically generate test scenarios, which improves efficiency and consistency. However, due to the randomness of the model and insufficient understanding of complex industry standards, the generated test cases may have problems with accuracy and incomplete coverage. The third scheme is to use test management tools and automated testing frameworks to transform test cases into executable test tasks, automatically execute them, and generate reports. However, in complex industrial software systems, due to complex business processes and frequent changes in requirements, the workload of writing and maintaining test scripts is huge, and the flexibility and coverage of test scripts often cannot meet the needs of high-frequency, large-scale testing tasks.

[0048] Based on this, this application proposes a method and apparatus for generating industrial software test cases based on a large language model. This method can automatically generate test cases, avoiding the inefficient manual writing process that relies on human experience. It combines a large language model and a Petri net model, and corrects deviations in functional path generation by optimizing the test case generation strategy. The Petri net model helps to more accurately describe the state and behavior of industrial software systems, ensuring that the generated test cases not only conform to the expected functional paths but also cover complex interactions and business logic within the system, improving the accuracy and rigor of the test cases. Generating test case execution files based on the large language model allows the generated test cases to be directly converted into executable test scripts, avoiding tedious manual script writing.

[0049] Reference Figure 1 , Figure 1 This is an optional flowchart of a method for generating industrial software test cases based on a large language model, provided in an embodiment of this application. The method may include, but is not limited to, steps S101 to S105:

[0050] Step S101: Obtain the prompting strategy and knowledge base. The knowledge base is used to store background description information for the generation of industrial software test cases.

[0051] Step S102: According to the prompting strategy, guide the large language model to generate test cases based on the knowledge base and output the corresponding first test case;

[0052] Step S103: Obtain the system functional description model;

[0053] Step S104: Optimize the first test case based on the system function description model using the large language model to generate the second test case;

[0054] Step S105: Obtain test case deployment information, and use the large language model to perform code conversion processing on the second test case based on the test case deployment information and the system function description model to generate the corresponding test case execution file.

[0055] In some embodiments, step S101 may include, but is not limited to, steps S201 to S204:

[0056] Step S201: Obtain background description information, which includes at least system function information, system application programming interface, system parameter description information, system interaction principle information, and role definition information;

[0057] Step S202: Construct a knowledge base and store the background description information in the knowledge base;

[0058] Step S203: Obtain test case generation prompt information. The test case generation prompt information includes at least the role information of the current generation environment, test case format constraint information, and a sample template of the test case.

[0059] Step S204: Generate prompt information and knowledge base based on test cases, and generate prompt strategy.

[0060] In some embodiments, the knowledge base provides detailed key system information for the large language model, such as component functions, interaction principles, parameter requirements, and API interfaces, by introducing standardized domain knowledge. For example, the knowledge base may include, but is not limited to, the following detailed key system information:

[0061] Role definition information: Defining roles helps large language models understand context and objectives. For example, when the role is defined as "test engineer," the model will tend to generate content related to software testing, avoiding generating irrelevant answers. This explicit role definition ensures that the model's output is more focused and accurate;

[0062] Component Basic Information: To guide the generation of the large language to better align with the testing content of industrial software component systems, real-world data from currently used industrial software component systems is acquired as background description information and added to the knowledge base. This background description information may include, but is not limited to, component function introductions, initialization setup requirements, and corresponding API interfaces. Furthermore, the knowledge base is divided into filling units based on components, which enhances its scalability.

[0063] System interaction principles: Interaction principles include clearly defining the preconditions for component use, interaction constraints, and the execution flow of system functions. Preconditions for component use refer to the basic conditions and states that each component must meet before interaction. These preconditions ensure that components interact in the correct environment and state, thereby avoiding errors and anomalies caused by unmet conditions. Interaction constraints refer to the rules and restrictions that each component must follow during the interaction process, aiming to ensure smooth and error-free interaction. The execution flow of industrial software component system functions refers to a series of defined actions, clarifying the role and task of each component in the execution of a specific function. The entire execution flow must strictly follow the predefined steps to ensure the accuracy and reliability of system functions.

[0064] System parameter description information: This section uses natural language to annotate components involving parameter requirements, such as coordinate information, image data, and control commands. These include parameter types, ranges, default values, and constraints. This information helps the large language model focus on these parameter requirements during production, ensuring that tests cover various boundary conditions and anomalies.

[0065] System Application Programming Interface: This section describes the component's API interface in detail, including the method name, parameter requirements, response format, and sample code for each API interface. The API interface documentation provides a mapping between the large language model and the component's API interface, facilitating code generation for functional testing.

[0066] Generate content format constraints: By defining clear JSON format constraints, the data structure, field types, required fields, and optional fields of the generated test cases can be standardized, so that the data can be accurately identified and used during transmission and processing. This also helps to locate and solve problems more quickly during development, debugging, and maintenance, ultimately improving the overall performance and stability of the system.

[0067] Few-shot learning datasets: By providing multiple pairs of test case samples with expected outputs, large language models are trained to solve tasks, which can effectively increase the efficiency of large language models in outputting expected content.

[0068] In some embodiments, taking the construction of a knowledge base for a componentized industrial software system as an example, the knowledge base is used to store detailed description information of the componentized industrial software system (such as the background description information mentioned above), including basic information of the industrial software system components, interaction principles, parameter requirements, API interfaces corresponding to the components, and prompt templates, etc. This information provides a clear knowledge foundation for the large language model through standardized description, enabling the large language model to generate test cases that meet actual needs more accurately. By expressing the technical details of complex componentized industrial software systems in natural language, this knowledge base simplifies the knowledge input process and realizes efficient automation of test case generation. It not only effectively solves the problem of insufficient coverage of complex scenarios by traditional methods, but also improves the scalability and accuracy of test cases in various practical application scenarios.

[0069] In some embodiments, a prompting strategy is generated based on prompt information and a knowledge base generated from test cases. This strategy guides the large language model in generating test cases, enabling it to accurately understand user input and, combined with knowledge base information, generate comprehensive test content that conforms to API parameter formats. Through this prompting strategy, the large language model can generate test cases based on the interaction logic and data dependencies of different APIs. The generated test cases not only meet system functional requirements but also possess sufficient coverage and accuracy, improving the effectiveness of test case generation and providing technical support for automated testing of complex industrial software systems.

[0070] This prompting strategy utilizes a large language model and prompting engineering to automatically generate test cases. By effectively using the large language model to model the generation process, the prompting strategy guides the large language model to accurately understand the provided knowledge background description and target generation process model. This prompting strategy enables the large language model to quickly locate and extract the knowledge base information required to perform the test case generation task.

[0071] For example, the prompting strategy can be: input the role definition of the current generation environment → input the system knowledge background → input the constraints of the generated content format → input a small number of sample templates of expected output → output the generated result.

[0072] In some embodiments, step S103 may include, but is not limited to, steps S301 to S302:

[0073] Step S301: Obtain system function description information; the system function description information includes description information of multiple system functions and description information of the interaction flow of each system component when executing each system function;

[0074] Step S302: Based on the system function description information, perform mathematical modeling and establish the corresponding system function description model.

[0075] In some embodiments, the system function description model described above is a Petri net model. A Petri net model is a logical model used to describe and analyze asynchronous concurrent systems, and can express concurrent events and state changes in the system.

[0076] Based on the component workflows involved in each system function (such as the system function description information mentioned above), Petri net modeling is performed to obtain the corresponding Petri net model. The Petri net model describes the interaction process between components in a mathematical model and stores it in JSON format. Combining the Petri net model with test case generation can effectively improve the test case coverage, make the generated test paths meet the requirements of engineering testing, reduce the risk of missing key boundary conditions or abnormal scenarios, and thus ensure the reliability and accuracy of test results. The test path can be used to generate multiple test cases.

[0077] Optionally, the Petri net model is defined using JSON format. The corresponding JSON Schema specifies the structure of the Petri net model, comprising three main parts: nodes, changes, and connections. Nodes represent the set of all nodes in the Petri net, which can be system states, resources, or other important elements. Changes represent the set of all transitions, i.e., operations or events that change the state of the system. Connections are an array of objects, each defining a connection relationship, including a source node and a target node. These connections describe the transitive relationship from a node to a transition, or from a transition to the next node. This structured approach clearly expresses the composition of the Petri net model and the interactions between its components, thus ensuring the correctness and consistency of the Petri net model. In practical applications, this JSON Schema structure facilitates the verification and parsing of the Petri net model, enabling the generated test cases to comprehensively cover the system's functionality, while improving the accuracy and efficiency of test path generation.

[0078] In some embodiments, step S104 may include, but is not limited to, steps S401 to S403:

[0079] Step S401: Input the system function description model and the first test case into the large language model, and use the large language model to determine the test sequence data that matches the first test case from the system function description model;

[0080] Step S402: Obtain test path correction prompts;

[0081] Step S403: Input the test sequence data, the first test case, and the test path correction prompt into the large language model, use the large language model to correct the first test case, output the corrected first test case, and obtain the corrected first test case as the second test case.

[0082] In some embodiments, the system function description model not only records the functional execution flow of each component in the system, but also systematically defines the interaction relationship between each component. The system function description model, test path correction prompts and the first test case are input into the large language model. The large language model corrects the first test case and outputs the aforementioned second test case. This allows the large language model to adjust the generated test cases according to the actual application scenario, cover a more comprehensive test path, and improve the accuracy and effectiveness of the testing work.

[0083] Optionally, the large language model outputs test case JSON data that meets the input requirements. To improve visualization, the test case JSON data will be converted into an Excel spreadsheet using Python's pandas library. This data conversion method not only allows the test cases to be presented in a more intuitive way, but also makes it easier for test engineers to view, edit, and analyze the test cases. Through this Excel spreadsheet, users can more easily manage and optimize the test process, further improving the quality and coverage of test cases, and enhancing the comprehensiveness and accuracy of the testing work.

[0084] In some embodiments, step S403 may include, but is not limited to, steps S501 to S502:

[0085] Step S501: Use the large language model to correct the prompt words according to the test path, compare the test process of the first test case and the test sequence data, and determine the process comparison result;

[0086] Step S502: When the process comparison result shows that the test process is inconsistent, the first test case is corrected based on the test sequence data using the large language model, and the second test case is output.

[0087] In some embodiments, step S105 may include, but is not limited to, steps S601 to S602:

[0088] Step S601: Obtain the code generation template knowledge base and code generation prompts corresponding to the test cases;

[0089] Step S602: Input the test sequence data, code generation template knowledge base, code generation prompt words, and second test case into the large language model. Use the large language model to output the test case execution file corresponding to the second test case. The test case execution file includes the test case name, test case description, test interface address, interface input parameters, the value of the interface input parameters, test case code, and test case script execution code.

[0090] In some embodiments, specifically, test cases are quickly converted into executable script code using a large language model, reducing the tedium of manual coding and improving the efficiency and accuracy of testing.

[0091] Optionally, the aforementioned large language model encapsulates the service request methods and service result verification methods of components for generating test case workflow script code. This encapsulation method abstracts the underlying API call and result verification process, reduces the probability of errors when the large language model generates business code, enhances the accuracy and maintainability of code generation during test automation, improves testing efficiency, and enables rapid code deployment. The test code writing only needs to focus on key information such as the component's API interface and input parameters, greatly simplifying the writing and maintenance of test cases.

[0092] Based on the generated second test case and the test sequence data in its corresponding Petri net model, it can be confirmed that it can match the designed code interface processing. The nodes in the test sequence data can be regarded as the triggers of API requests, while the transitions represent the expected results of the previous node's actions, i.e., the verification of service results. During the code generation process, based on the capabilities of the large language model, complex business logic can be automatically generated, including the calling of component services and the verification processing of return status.

[0093] In some embodiments, a code generation template knowledge base and code generation prompts are introduced to implement behavioral programming (BP). Behavioral programming emphasizes the system's response in different states. By separating and combining these states and behaviors, the system's behavior and state transitions are defined more explicitly. This approach not only improves code readability but also makes operational logic more intuitive and clear, facilitating test case generation, code maintenance, and subsequent optimization. Through this structured representation, system code generation becomes more efficient, reducing maintenance costs and improving the overall controllability of the testing workflow.

[0094] In some embodiments, the test case generation method described above may include, but is not limited to, steps S701 to S702:

[0095] Step S701: Construct a large language model;

[0096] Step S702: Pre-train the large language model using a knowledge base, which also includes a few-shot learning dataset.

[0097] In some embodiments, the few-shot learning dataset includes multiple pairs of test case samples of expected outputs. The large language model is pre-trained using this few-shot learning dataset, which effectively increases the efficiency of the large language model in outputting expected content.

[0098] In some embodiments, taking a component-based industrial software system as an example, the above-mentioned industrial software test case generation method based on a large language model is applied to generate test cases for the component-based industrial software system. Specifically, the automatic dispensing function scenario in the component-based industrial software system is selected. This scenario describes the entire process of a robotic arm cooperating with a vision camera to control the glue gun to complete the dispensing action by identifying and moving the target position. After the dispensing action is completed, the robotic arm resets, which signifies the completion of the entire function. The specific test case generation process for this scenario is as follows:

[0099] The first step is to learn how to operate the automatic dispensing function, as follows:

[0100] 1) The system uses the robotic arm base moving component to reset the robotic arm, initialize the glue gun component and the vision camera component;

[0101] 2) The system calls the vision camera component and transmits the coordinates of the dispensing position;

[0102] 3) The system calls the robotic arm base moving component to move to the dispensing position according to the position sequence recognized by the vision camera;

[0103] 4) The system calls the robotic arm's moving component to move to the dispensing distance;

[0104] 5) The system calls the dispensing component to complete the dispensing;

[0105] 6) The system calls to reset the robotic arm's movement components;

[0106] 7) Repeat steps 3) to 6) until all required dispensing coordinates have been dispensed.

[0107] 8) The system calls to reset the robotic arm base moving component.

[0108] The second step involves combining the operation steps of the automatic dispensing function with the testing background of the modular industrial software system to generate a knowledge base for editing test guidance knowledge, naming it `test_case_generation_data.md`. Then, by calling the multi-turn dialogue API interface of the large language model, the user's expected content, generation requirements, and `test_case_generation_data.md` file are packaged and transmitted to the large language model. After receiving the transmitted data, the large language model uses test case generation prompts to guide the analysis of the packaged file, generating and returning the JSON format data of the first test case. The JSON format data of the first test case is stored in the `test_outcome.md` file.

[0109] The third step involves correcting the prompts via the test path. The large language model compares the test case generation data in the `test_outcome.md` file with the real-world Petri net model `llm_petri.md` file of the industrial software component system using test sequence alignment. Based on the test case name, the large language model finds the corresponding Petri net model test sequence data in the `llm_petri.md` file and performs a test flow comparison. After the flow comparison is complete, assuming the test flow matches the corresponding test flow in the Petri net model (i.e., the test flow is consistent), the large language model returns the final JSON data and a description of the matching result. If the test flow is inconsistent, the first test case is corrected based on the Petri net model test sequence data, generating a second test case. The second test case's JSON data and a description of the correction are returned, and the JSON data of the second test case is stored in the `test_final_outcome.md` file.

[0110] The fourth step involves using Python's pandas library to convert the `test_final_outcome.md` file into JSON data and generate an Excel spreadsheet.

[0111] The fifth step involves taking the `test_final_outcome.md` and `llm_petri.md` files as input, which are then processed by the large language model to filter out Petri net model data (i.e., the Petri net model test sequence data mentioned above) that corresponds to the functions of the test cases. This data is then integrated and output as the `petri_code_data.md` file. Subsequently, the `petri_code_data.md` file and the code generation template knowledge base `llm_code_gen.md` file are taken as input together. The code generation prompts instruct the large language model to generate the corresponding test case execution files.

[0112] Reference Figure 2 , Figure 2 This is an optional structural diagram of an industrial software test case generation device based on a large language model provided in this application embodiment. The device is used to implement the aforementioned industrial software test case generation method and may include:

[0113] The first module is used to obtain the prompting strategy and knowledge base. The knowledge base is used to store the background description information of the industrial software test cases generated.

[0114] The second module is used to guide the large language model to generate test cases based on the knowledge base according to the prompting strategy, and output the corresponding first test case.

[0115] The third module is used to obtain the system functional description model;

[0116] The fourth module is used to optimize the first test case based on the system function description model using a large language model, and generate the second test case.

[0117] The fifth module is used to obtain test case deployment information. Based on the test case deployment information and the system function description model, the large language model is used to perform code conversion processing on the second test case and generate the corresponding test case execution file.

[0118] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0119] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described industrial software test case generation method. This electronic device can be any smart terminal, including tablet computers.

[0120] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0121] Please see Figure 3 , Figure 3 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0122] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0123] The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the industrial software test case generation method of the embodiments of this application.

[0124] The input / output interface 903 is used to implement information input and output;

[0125] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0126] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0127] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0128] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described industrial software test case generation method.

[0129] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0130] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0131] This application provides a method and apparatus for generating industrial software test cases based on a large language model. It can achieve automated test case generation through a large language model, improve the efficiency of test case generation, enhance the quality of test cases, and meet the needs of automated software system testing.

[0132] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0133] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0134] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0135] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0136] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0137] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0138] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for generating industrial software test cases based on a large language model, characterized in that, The method includes the following steps: Acquire prompting strategies and a knowledge base, wherein the knowledge base is used to store background description information for the generation of industrial software test cases; The aforementioned strategy for obtaining prompts and the knowledge base specifically include: Obtain the background description information, which includes at least system function information, system application programming interface, system parameter description information, system interaction principle information, and role definition information; Construct the knowledge base and store the background description information in the knowledge base; Obtain test case generation prompts, which include at least the role information of the current generation environment, test case format constraints, and a sample template for the test case. Based on the test cases, generate prompt information and the knowledge base, and generate the prompt strategy; According to the prompting strategy, the large language model is guided to generate test cases based on the knowledge base and output the corresponding first test case. Obtain the system function description model, which is a Petri net model; Using the large language model and the system function description model, the first test case is optimized to generate the second test case; The step of optimizing the first test case based on the system function description model using the large language model to generate the second test case specifically includes: The system function description model and the first test case are input into the large language model, and the large language model is used to determine the test sequence data that matches the first test case from the system function description model; Get the test path correction prompt; The test sequence data, the first test case, and the test path correction prompt are input into the large language model. The first test case is corrected using the large language model, and the corrected first test case is output. The corrected first test case is then obtained as the second test case. Obtain test case deployment information, and use the large language model to perform code conversion processing on the second test case based on the test case deployment information and the system function description model to generate the corresponding test case execution file.

2. The industrial software test case generation method according to claim 1, characterized in that, The acquisition of the system functional description model specifically includes: Obtain system function description information; the system function description information includes description information of multiple system functions and description information of the interaction flow of each system component when executing each system function; Based on the system function description information, mathematical modeling is performed to establish the corresponding system function description model.

3. The industrial software test case generation method according to claim 1, characterized in that, The step of inputting the test sequence data, the first test case, and the test path correction prompt into the large language model, using the large language model to correct the first test case, outputting the corrected first test case, and obtaining the corrected first test case as the second test case specifically includes: Using the large language model, the prompt words are corrected according to the test path. The test process of the first test case and the test sequence data is compared to determine the process comparison result. When the process comparison result indicates that the test process is inconsistent, the first test case is corrected based on the test sequence data using the large language model, and the second test case is output.

4. The industrial software test case generation method according to claim 1, characterized in that, The step of obtaining test case deployment information, and using the large language model to perform code conversion processing on the second test case based on the test case deployment information and the system function description model, to generate the corresponding test case execution file, specifically includes: Retrieve the code generation template knowledge base and code generation prompts corresponding to the test cases; The test sequence data, the code generation template knowledge base, the code generation prompts, and the second test case are input into the large language model. The large language model is then used to output the test case execution file corresponding to the second test case. The test case execution file includes the test case name, test case description, test interface address, interface input parameters, the values ​​of the interface input parameters, test case code, and test case script execution code.

5. The industrial software test case generation method according to claim 1, characterized in that, The method further includes: Construct the large language model; The large language model is pre-trained using the knowledge base, which also includes a few-shot learning dataset.

6. An industrial software test case generation device based on a large language model, characterized in that, The device includes: The first module is used to acquire prompting strategies and a knowledge base, wherein the knowledge base is used to store background description information for the generation of industrial software test cases; The aforementioned strategy for obtaining prompts and the knowledge base specifically include: Obtain the background description information, which includes at least system function information, system application programming interface, system parameter description information, system interaction principle information, and role definition information; Construct the knowledge base and store the background description information in the knowledge base; Obtain test case generation prompts, which include at least the role information of the current generation environment, test case format constraints, and a sample template for the test case. Based on the test cases, generate prompt information and the knowledge base, and generate the prompt strategy; The second module is used to guide the large language model to generate test cases based on the knowledge base according to the prompting strategy, and output the corresponding first test case. The third module is used to obtain the system functional description model; The fourth module is used to optimize the first test case based on the system function description model using the large language model, and generate the second test case. The step of optimizing the first test case based on the system function description model using the large language model to generate the second test case specifically includes: The system function description model and the first test case are input into the large language model, and the large language model is used to determine the test sequence data that matches the first test case from the system function description model; Get the test path correction prompt; The test sequence data, the first test case, and the test path correction prompt are input into the large language model. The first test case is corrected using the large language model, and the corrected first test case is output. The corrected first test case is then obtained as the second test case. The fifth module is used to obtain test case deployment information, and use the large language model to perform code conversion processing on the second test case based on the test case deployment information and the system function description model to generate the corresponding test case execution file.

Images