Test case generation method and device

Abstract

The invention provides a test case generation method and device. The test case generation method comprises the steps of obtaining model information of a to-be-tested model, wherein the model information comprises test logic information of the to-be-tested model and / or data format information of the to-be-tested model; the model information of the to-be-tested model is input into an AI model to generate a plurality of test cases of the to-be-tested model, and the test cases are used for testing the test logic of the to-be-tested model. Compared with a traditional scheme in which the test cases are generated through manual handwriting, the test case generation method and device are helpful for quickly generating a large number of test cases, so that sufficient tests are realized, and the consumption of human resources is reduced.

Description

Technical Field

[0001] This application relates to the field of testing, and particularly to a method and device for generating test cases. Background Art

[0002] Model in the loop (MIL) testing is a testing method based on model-based design (MBD). Since the results of MIL testing will affect subsequent various tests, MIL testing is crucial. Currently, the test cases for MIL testing mainly rely on MIL test engineers to generate according to the model to be tested. However, it is unrealistic to generate test cases in this way that relies on manual writing. For the rapid generation of a large number of test cases, manual writing is not only difficult to achieve sufficient testing, but also requires a large amount of manpower. Summary of the Invention

[0003] Embodiments of this application provide a method and device for generating test cases, which will be introduced from the following aspects.

[0004] In a first aspect, a method for generating test cases is provided, including: obtaining model information of a model to be tested, where the model information includes test logic information and / or data format information of the model to be tested; inputting the model information of the model to be tested into an artificial intelligence (AI) model to generate multiple test cases for the model to be tested, and the multiple test cases are used to test the test logic of the model to be tested.

[0005] As a possible implementation, the method further includes: inputting the multiple test cases into the model to be tested to obtain an output result corresponding to each test case in the multiple test cases; comparing the data format of the output result with the data format of the output result of the model to be tested to obtain a comparison result; screening out valid test cases from the multiple test cases according to the comparison result, and the data format of the output result corresponding to the valid test cases matches the data format of the output result of the model to be tested.

[0006] As a possible implementation, the method further includes: generating a vector corresponding to each test case in the multiple test cases; performing similarity detection on the vectors corresponding to the multiple test cases to obtain the similarity between the vectors corresponding to the multiple test cases; screening out valid test cases of the model to be tested from the multiple test cases according to the similarity between the vectors corresponding to the multiple test cases, and the similarity between the vectors corresponding to the valid test cases of the model to be tested is greater than a similarity threshold.

[0007] As a possible implementation, the method further includes: determining the coverage range of each test case among the multiple test cases; and screening, according to the coverage range of each test case among the multiple test cases, multiple valid test cases for the to-be-tested model from the multiple test cases, where the coverage ranges corresponding to the multiple valid test cases for the to-be-tested model do not overlap with each other.

[0008] As a possible implementation, the to-be-tested model is a Simulink model.

[0009] In a second aspect, a test case generation device is provided, including: an acquisition unit configured to acquire model information of a to-be-tested model, where the model information includes test logic information and / or data format information of the to-be-tested model; and a processing unit configured to input the model information of the to-be-tested model into an AI model to generate multiple test cases for the to-be-tested model, where the multiple test cases are used to test the test logic of the to-be-tested model.

[0010] As a possible implementation, the processing unit is configured to: input the multiple test cases into the to-be-tested model to obtain an output result corresponding to each test case among the multiple test cases; compare the data format of the output result with the data format of the output result of the to-be-tested model to obtain a comparison result; and screen out valid test cases from the multiple test cases according to the comparison result, where the data format of the output result corresponding to the valid test cases matches the data format of the output result of the to-be-tested model.

[0011] As a possible implementation, the processing unit is configured to: generate a vector corresponding to each test case among the multiple test cases; perform similarity detection on the vectors corresponding to the multiple test cases to obtain the similarity between the vectors corresponding to the multiple test cases; and screen the valid test cases for the to-be-tested model from the multiple test cases according to the similarity between the vectors corresponding to the multiple test cases, where the similarity between the vectors corresponding to the valid test cases for the to-be-tested model is greater than a similarity threshold.

[0012] As a possible implementation, the processing unit is configured to: determine the coverage range of each test case among the multiple test cases; and screen, according to the coverage range of each test case among the multiple test cases, multiple valid test cases for the to-be-tested model from the multiple test cases, where the coverage ranges corresponding to the multiple valid test cases for the to-be-tested model do not overlap with each other.

[0013] As a possible implementation, the to-be-tested model is a Simulink model.

[0014] In a third aspect, a computing device is provided. The computing device includes a memory, a processor, and an input / output interface. The memory is used to store a computer program. The processor is used to call and run the computer program from the memory. The input / output interface is used to receive input data and output data, so that the computing device executes the method in the first aspect above.

[0015] In a fourth aspect, a computer program product is provided. The computer program product includes: computer program code, which, when running on a computer, causes the computer to execute the method in the first aspect above.

[0016] In a fifth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores program code, which, when running on a computer, causes the computer to execute the method in the first aspect above.

[0017] In this application, by obtaining the model information of the model to be tested and inputting it into the AI model, a large number of test cases are generated. Compared with the traditional solution of generating test cases by manual writing, this solution helps to quickly generate a large number of test cases, thereby achieving sufficient testing and reducing the consumption of human resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The figure shows a schematic flowchart of the method for generating test cases provided by an embodiment of this application.

[0019] Figure 2 The figure shows a schematic flowchart of using a large model to generate test cases provided by an embodiment of this application.

[0020] Figure 3 The figure shows a schematic flowchart of generating a test experience library provided by an embodiment of this application.

[0021] Figure 4 The figure shows a schematic structural diagram of the test case generation device provided by an embodiment of this application.

[0022] Figure 5 The figure shows a schematic block diagram of the computing device provided by an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments.

[0024] MBD is a system design method that focuses on using mathematical models to capture, analyze, design, and verify the behavior of complex systems. The core idea of this design method is to express the functional and performance requirements of the system through graphical or mathematical models, and then automatically generate code to implement these models.

[0025] MBD is widely used in fields such as control engineering, signal processing, communication systems, automotive electronics, and aerospace. In the field of automotive electronics, MBD-based design specifically refers to the way of software development using MATLAB and / or Simulink models. Simulink is a model-based design tool, which is a graphical tool for modeling and simulation. Users can use Simulink for algorithm development, system simulation, testing, and code generation (such as C, C++). For users, without writing a large amount of programs, they can construct complex systems only through simple and intuitive mouse operations.

[0026] In the software R & D process of automotive electronic control units (ECUs), many types of tests are involved, such as: MIL test, software in the loop (SIL) test, processor in the loop (PIL) test, hardware in the loop (HIL) test, etc. These test methods are at different stages of the software development and verification process, and they respectively correspond to different levels from model verification to final hardware testing. Through these tests, it is possible to gradually transition from early proof of concept to final product verification, ensuring that the final product can meet all functional and performance requirements.

[0027] Among them, the MIL test refers to implementing a closed-loop test at the model level and is a test method based on MBD. The prerequisite for the MIL test is to have a controlled object model, which can be a self-built model or a purchased off-the-shelf model. This MIL test can usually be applied in the following two scenarios, such as system engineers using a control algorithm model to control the controlled object model to verify the algorithm, or software engineers conducting model-level integration tests.

[0028] At the same time, the MIL test is also an economical and effective test method because it can verify the model in the early stage of development, thus exposing defects in the early stage of development, which helps to identify and correct errors early and further reduce the development cycle and cost. In the software testing process of automotive ECUs, the MIL test is the most critical test. This is because the acceptance criteria for the MIL test must originate from the functional requirements and there is no other reference basis. Moreover, the results of the MIL test will directly affect the subsequent SIL test and PIL test because the subsequent test cases often depend on the test cases of the MIL test.

[0029] Therefore, the test cases for the MIL test are crucial. Currently, the test cases for the MIL test mainly rely on MIL test engineers to generate based on the model to be tested. That is to say, currently, the test cases for the MIL test are generated manually by handwriting.

[0030] However, it is unrealistic to rely on manual handwriting to generate test cases. On the one hand, it is difficult to provide a large number of test cases by manual handwriting. For the model to be tested, sufficient test cases are required to ensure the correctness of the model. Especially for some complex models, a large number of test cases are needed to ensure full coverage of the model design. On the other hand, the generation of test cases is time-consuming and laborious. Generally, the time spent on designing and generating test cases is more than the time spent on executing test cases. Correspondingly, manual handwriting cannot generate test cases quickly, resulting in a large amount of manpower required in the early stage of test case generation. Therefore, for the rapid generation of a large number of test cases, manual handwriting is not only difficult to achieve sufficient testing but also requires a large amount of manpower.

[0031] To address the problems mentioned above, the embodiments of the present application generate a large number of test cases by obtaining the model information of the model to be tested and inputting it into the AI model. Compared with the traditional solution of relying on manual handwriting to generate test cases, this solution helps to quickly generate a large number of test cases, thus achieving sufficient testing and reducing the consumption of human resources.

[0032] The following combines Figure 1 to introduce the method for generating test cases provided by the embodiments of the present application. Figure 1 The method shown can be executed by a computing device. For example, the computing device can be a calculator, a server, etc. Of course, in the embodiments of the present application, Figure 1 the method shown can also be executed by other devices with relevant functions.

[0033] As shown in Figure 1 shown, Figure 1 the method shown includes steps S110 to S120.

[0034] In step S110, obtain the model information of the model to be tested.

[0035] In some implementation manners, the model to be tested can be understood as the model that needs to be tested. The model to be tested can be an MBD model. For example, the model to be tested can be a Simulink model. When the model to be tested is a Simulink model, its model structure is relatively complex. For testers, it is more difficult to generate test cases only by manual writing in the face of the model design with a complex architecture. Therefore, the embodiments of the present application can more specifically solve the problem of generating test cases in the Simulink model.

[0036] Generally, the model information of the model to be tested includes the test logic information of the model to be tested and / or the data format information of the model to be tested. Among them, the test logic information of the model to be tested can be understood as the logic and principles followed when designing and generating test cases, and it involves how to systematically design and generate test cases to verify the attributes such as the function and performance of the model to be tested. For example, when the model to be tested obtains an input, the model to be tested will perform certain processing operations on the input internally and finally output. The processing operation logic information inside the model to be tested here can be understood as the test logic information.

[0037] The data format information of the model to be tested can be understood as data types, data dimensions, etc. Among them, the data types can include, for example, one or more of the following: boolean type, integer type, floating-point type, enumeration type, structure type, alias type. The data dimensions can include, for example, one or more of the following: one-dimensional data, two-dimensional data, multi-dimensional data, high-dimensional data.

[0038] In the embodiments of the present application, the manner of obtaining the model information of the model to be tested is not limited. In some implementation manners, the model information of the model to be tested can be obtained by communicating with the engineer who designed the model to be tested. This implementation manner is more subjective and can quickly obtain the required model information. In other implementation manners, relevant documents can be consulted to obtain the model information of the model to be tested. For example: design documents (including architecture design, interface design, etc.) to understand the internal structure and working process of the model to be tested, which helps to understand the test logic. In addition, the model information of the model to be tested can also be obtained by reading the data in the database, so as to quickly master the data format information of the model to be tested.

[0039] In step S120, input the model information of the model to be tested into the AI model to generate multiple test cases for the model to be tested.

[0040] In some implementations, the AI model adopts a large language model, which is a deep learning model trained with a large amount of text data. By learning the rules and knowledge of language, it can generate fluent natural language text or understand the meaning of language text. For example, GPT-3 and GPT-4 of OpenAI, BERT and T5 of Google, Wenxin Yiyan of Baidu, Spark Model of iFlytek, etc.

[0041] In some implementations, multiple test cases are used to test the test logic of the model under test. For example, multiple test cases are input into the model under test, and the model under test outputs results according to its test logic for testing.

[0042] In the embodiments of the present application, the generation method of multiple test cases for the model under test is not limited. In some implementations, relevant AI models can be directly used to generate multiple test cases for the model under test. For example, GPT-4 can be directly used to generate test cases. This implementation is relatively simple and convenient for testers, but the quality of test cases generated by this general model is not very good.

[0043] In some other implementations, relevant AI models can be fine-tuned to generate multiple test cases for the model under test. For example, GPT-4 can be fine-tuned to generate multiple test cases for the model under test. This model fine-tuning technology refers to supervised training on specific task data based on relevant AI models. By adjusting the parameters in the training process, the AI model is guided in the functions of a specific field, further improving the performance of the AI model in specific tasks, such as further improving the performance of the AI model in the task of generating test cases.

[0044] In addition, the embodiments of the present application do not limit the fine-tuning method of the AI model either. For example: full fine-tuning, parameter-efficient fine-tuning (PEFT), adapter tuning, low-rank adaptation of large language model (LoRA) fine-tuning. The above fine-tuning methods have their own advantages and applicable scenarios. Selecting the appropriate fine-tuning method can significantly improve the performance of the model while reducing the training time and computing cost.

[0045] Taking LoRA fine-tuning as an example, this fine-tuning method is a technique for efficiently fine-tuning pre-trained models. The core idea of LoRA fine-tuning is to freeze the weights of the pre-trained model and then, by only training low-rank matrices and injecting the parameters of these low-rank matrices into each layer of the Transformer architecture, fine-tuning is achieved. LoRA learns an approximate low-rank decomposition representation of the incremental parameter matrix during the fine-tuning process, that is, the parameters are dimensionally reduced. In this way, the approximately parameterized low-rank decomposition matrix trained will not have a significant discount in effect compared with other methods.

[0046] By performing the above process, LoRA fine-tuning can significantly reduce the number of parameters in the fine-tuning stage, thereby reducing the computational cost and storage requirements of the model. During inference, LoRA can directly merge the weights of the original pre-trained model with the trained LoRA weights, so there is no additional overhead. At the same time, LoRA can be flexibly switched between different fine-tuning tasks without causing inference latency. Despite the reduction in the number of parameters, LoRA can still maintain the original capabilities of the pre-trained model while adapting to new tasks or datasets. In short, the LoRA architecture provides an efficient and resource-saving method for fine-tuning AI models by introducing low-rank matrices, especially suitable for deep learning models that need to handle a large number of parameters and for scenarios with limited resources.

[0047] For the generation of test cases in the embodiments of this application, using LoRA to fine-tune the AI model can not only ensure that the AI model generates specific test cases at a relatively fast speed, but also ensure the capabilities of the AI model itself such as generating reports and information extraction, and reduce the impact of model overfitting caused by external data input for training.

[0048] As introduced above, after obtaining the model information of the model to be tested, it is input into the AI model to generate multiple test cases for the model to be tested. However, among these generated multiple test cases, there may be some invalid test cases, and the output results corresponding to these invalid test cases do not meet the requirements for the data format of the output results in the model to be tested. Therefore, it is necessary to run multiple test cases and check the data format of their corresponding output results to further screen out valid test cases.

[0049] In some implementations, multiple test cases are input into the model to be tested to obtain the output result corresponding to each test case among the multiple test cases. Then, the data format of the output result is compared with the data format of the output result of the model to be tested to obtain a comparison result. Finally, valid test cases are selected from the multiple test cases according to the comparison result. Here, "valid" can be understood as that the data format of the output result corresponding to the test case matches the data format of the output result of the model to be tested. In addition, the requirements of the model to be tested for the data format of the output result can be queried in relevant documents, such as a data dictionary. Since the output results are directly compared, testers need to manually write the expected output results in advance, which is a large workload. And this implementation method that does not directly compare the output results but only compares the data formats can further reduce the workload of testers.

[0050] For example, when the data type of the output result defined by the model to be tested is floating-point, multiple test cases are run at this time to determine whether the data format of the output results of the multiple test cases is floating-point. By retaining the test cases with the data format of the output result being floating-point and removing the test cases with the data format of the output result not being floating-point, valid test cases are thus selected.

[0051] Since the model information of the model to be tested obtained is relatively broad, the AI model will generate a large number of test cases. However, there may be a situation where multiple test cases are highly similar among these test cases, or it can be said that there may be a situation where multiple test cases repeatedly cover the model to be tested. The subsequent process of inputting the test cases into the model to be tested is still manual operation. Therefore, the redundant test cases in the above possible situations should be deleted to further reduce the workload of testers performing the test work and save the resources for subsequent test operations.

[0052] In the embodiments of the present application, there is no limitation on the method for deleting redundant test cases. In some implementations, a vector similarity search technique can be used to delete redundant test cases, which can also be understood as selecting valid test cases. This method for deleting redundant test cases can delete multiple highly similar test cases.

[0053] For example, for multiple test cases generated by an AI model, vectors corresponding to each test case are generated. Then, similarity detection is performed on the vectors corresponding to the multiple test cases to obtain the similarity between the vectors corresponding to the multiple test cases. Finally, based on the similarity between the vectors corresponding to the multiple test cases, valid test cases for the model to be tested are screened from the multiple test cases, that is, redundant test cases are deleted. In addition, the embodiments of the present application do not limit the method of screening valid test cases according to the similarity between vectors. For example, a similarity threshold can be set in advance. After calculating the similarity between the vectors corresponding to the multiple test cases, the similarity is compared with the similarity threshold. When the similarity between the vectors is greater than the similarity threshold, the test case corresponding to the vector is a valid test case.

[0054] In some other implementation manners, the coverage of the test cases for the model to be tested can be detected to delete redundant test cases, which can also be understood as screening out valid test cases. The coverage here refers to the degree to which the model part covered when the test case is executed, and it is an index to measure the test integrity. In addition, detecting the coverage of the test cases can be implemented by using relevant tools. For example, the relevant tool can be the Coverage tool. Of course, in the embodiments of the present application, detecting the coverage of the test cases can also be implemented by using other tools with relevant detection functions.

[0055] The above-mentioned detection of the coverage of the test cases for the model to be tested includes: determining the coverage of each test case in the multiple test cases, and screening multiple valid test cases for the model to be tested from the multiple test cases according to the coverage of each test case in the multiple test cases. For example, the Coverage tool can be used to detect the coverage of the test cases. When the coverage of the test cases does not overlap with each other, they are valid test cases. This implementation manner further deletes redundant test cases with repeated coverage, thereby reducing the test work pressure of subsequent testers and saving computing resources.

[0056] For the sake of easy understanding, in the following, taking the model to be tested as a Simulink model and the AI model as GPT-4 as an example, combined with Figure 2 the method for generating test cases in the embodiments of the present application is introduced. It should be noted that the examples below are only for helping those skilled in the art to understand the embodiments of the present application, rather than limiting the embodiments of the present application to the specific numerical values or specific scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes according to the examples given below, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0057] Figure 2 The figure shows a schematic flowchart of generating test cases using a large model. Figure 2The method shown includes steps S210 to S240.

[0058] In step S210, the large model is fine-tuned.

[0059] To better adapt to the need for generating test cases, according to the characteristics of the Simulink model to be tested, the existing large-scale pre-trained language model GPT-4 is fine-tuned.

[0060] In step S212, the LoRA architecture is designed.

[0061] First, a low-rank matrix is defined. A pair of low-rank matrices A and B are added to each layer that needs to be adjusted, where A is a small matrix and B is a large matrix. The result of multiplying these two matrices is approximately equal to the change in the original weights. To enable the model to learn task-specific knowledge, LoRA modules need to be inserted at specific positions, and here the specific position can be the self-attention part of the Transformer model.

[0062] In step S214, the large model fine-tuning training is carried out.

[0063] Before starting the fine-tuning, the parameters in the LoRA module are randomly initialized. Then, an optimizer is set for fine-tuning training, and the Adam optimizer is configured to update the LoRA parameters. The prepared dataset is used to fine-tune the model. During this process, only the parameters in the LoRA module are updated, while the other parameters of the pre-trained model GPT-4 remain unchanged.

[0064] In step S220, test cases are initially generated.

[0065] In step S222, the input and output interfaces in the model to be tested are obtained according to the data dictionary.

[0066] The input and output interfaces in the model to be tested, as well as the data type, data dimension, maximum and minimum value data information in the data dictionary, are obtained. At the same time, boundary conditions and outlier situations are considered to increase the coverage of the test cases. Combining the above information, the large model will generate diverse test cases for the model to be tested. Taking a simple model to be tested as an example, the input interfaces are Input_sw, Input_1, Input_2, the output interface is Output, and the other component is the switch module. The logic of this test model is that when Input_sw is equal to 1, the value of Output is Input_1, and when Input_sw is not equal to 1, the value of Output is Input_2.

[0067] The information input to the large model is as follows: There are 3 input interfaces, namely Input_sw, Input_1, and Input_2, and 1 output interface, which is Output. According to the data dictionary, the requirements for the input interfaces are that the data type of the first input, Input_sw, is boolean and the dimension is 1; the data type of the second input, Input_1, is uint8 and the dimension is 1; the data type of the third input, Input_2, is uint8 and the dimension is 1. Generate as many test cases as possible for all input interfaces based on the above information, and only output the results of the combined test cases.

[0068] The following are some test case combinations output by the large model based on the above considerations:

[0069] 1. Input_sw = False, Input_1 = 0, Input_2 = 0

[0070] 2. Input_sw = False, Input_1 = 128, Input_2 = 0

[0071] 3. Input_sw = False, Input_1 = 255, Input_2 = 0

[0072] 4. Input_sw = False, Input_1 = 0, Input_2 = 128

[0073] 5. Input_sw = False, Input_1 = 0, Input_2 = 255

[0074] 6. Input_sw = False, Input_1 = 128, Input_2 = 128

[0075] 7. Input_sw = False, Input_1 = 128, Input_2 = 255

[0076] 8. Input_sw = False, Input_1 = 255, Input_2 = 128

[0077] 9. Input_sw = False, Input_1 = 255, Input_2 = 255

[0078] 10. Input_sw = True, Input_1 = 0, Input_2 = 0

[0079] 11. Input_sw = True, Input_1 = 128, Input_2 = 0

[0080] 12.Input_sw = True, Input_1 = 255, Input_2 = 0

[0081] 13.Input_sw = True, Input_1 = 0, Input_2 = 128

[0082] 14.Input_sw = True, Input_1 = 0, Input_2 = 255

[0083] 15.Input_sw = True, Input_1 = 128, Input_2 = 128

[0084] 16.Input_sw = True, Input_1 = 128, Input_2 = 255

[0085] 17.Input_sw = True, Input_1 = 255, Input_2 = 128

[0086] 18.Input_sw = True, Input_1 = 255, Input_2 = 255

[0087] In step S224, data cleaning and normalization are performed.

[0088] Data cleaning and normalization are performed to avoid obvious errors for subsequent processing. For each generated test case, check whether it conforms to the definition in the data dictionary, clear the data points that do not meet the requirements, and format the remaining data to ensure that they can be accepted by the model to be tested.

[0089] In step S226, valid test cases are filtered based on the output interface.

[0090] Valid test cases are further filtered by running the test cases and checking whether the results are within the expected range. Each cleaned and normalized test case is executed in a simulated environment, then the corresponding output results are collected and compared with the output format specified in the data dictionary. Finally, the test cases that result in output results outside the defined range are removed.

[0091] In step S230, redundant test cases are removed.

[0092] Since the information of the initially given input interface is relatively broad, a large number of test cases will be generated. To effectively remove redundant test cases, the following two steps are used for removal.

[0093] In step S232, preliminary removal is performed using vector similarity search technology.

[0094] In step S233, the test cases are converted into vectors of a certain form.

[0095] Each test case is converted into a vector representation of a fixed length. Since the test cases are structured data, the feature values can be used as vectors. Then, for each test case, a corresponding vector representation is generated. Suppose there are N test cases, and finally N vectors {v 1 , v 2 ,..., v N} will be obtained, where each vector v i is a d-dimensional vector. For example, the first and second test cases output by the large model in the above combined test cases are represented as vectors v 1 = [0, 0, 0]; v 2 = [0, 128, 0].

[0096] In step S234, the efficient nearest neighbor search library Faiss is used for similarity detection.

[0097] Due to the large amount of test case data, the IndexIvFFlat index type is selected, and the vectors of all test cases are added to the constructed index for subsequent fast search.

[0098] Determine the number k of nearest neighbors to search for and the similarity threshold τ. Here, k can be set to 2, and τ can be adjusted according to specific requirements. Perform a nearest neighbor search for each vector to find the k vectors that are most similar to it. The search results include a distance matrix and an index matrix. Among them, the distance matrix D represents the distance between the current vector and the nearest neighbors, and each row contains k distance values. The index matrix I represents the positions of the nearest neighbors in the original vector set, and each row contains k index values.

[0099] In the process of removing redundant test cases, first define a similarity threshold τ. If the distance between two vectors is less than τ, they are considered similar. Secondly, traverse each test case and its nearest neighbors to identify redundancy. If the distance between two vectors is found to be less than τ, mark one of them as redundant. The specific steps are as follows:

[0100] For each test case v i , check its nearest neighbors v i1 , v i2 , …… v ik . If D{i, j} < τ, then v i and v j are considered similar. Mark v j as redundant and keep the vector that appears first.

[0101]

[0102] where D is the dimension of the vector, and v i，d and v j，d are the values of v i and v j in the d-th dimension respectively.

[0103] In step S236, the Coverage function is used to analyze the coverage of test cases.

[0104] The Coverage function is used to analyze the coverage of test cases. When there is a situation of duplicate coverage of test cases, the redundant test cases will be deleted at this time.

[0105] In step S240, verification and analysis are performed.

[0106] In step S242, the expected results of the output interface are completed.

[0107] After generating the test cases for the input interface as described above, the expected results of the output interface are completed according to the logic of the model to be tested.

[0108] In step S244, import according to the requirements of the test software and generate a report.

[0109] The complete test cases including the input interface and the output interface are imported in the form required by the test software (such as BTC), the test cases are executed and their correctness is judged. Moreover, an execution report, a coverage report, etc. are generated.

[0110] In step S246, the report is analyzed.

[0111] The execution report and / or coverage report generated in step S244 above are analyzed.

[0112] Figure 3 The figure shows a schematic flow diagram of generating a test experience library. The above LoRA fine-tuning module 320 is used to fine-tune the pre-trained model GPT-4 310. The experience generated during this fine-tuning process can be input into the use case experience library 330. Of course, the fine-tuning process can also learn from the experience stored in the use case experience library 330. Among them, the process of initially generating test cases corresponds to Figure 2 step S220 in

[0113] After fine-tuning GPT-4, the large model can initially generate test cases, which can then be input into the model 340 to be tested to determine whether the output results corresponding to the test cases meet the definitions in the data dictionary. If the output results meet the definitions in the data dictionary, the experience generated during this process can be input into the use case experience library 330. If the output results do not meet the definitions in the data dictionary, the test cases corresponding to the output results can be fed back to the use case garbage library 360. Among them, the process of determining whether the output results meet the definitions in the data dictionary corresponds to Figure 2 step S226 in

[0114] In addition, the built-in coverage test toolbox 350 of the Simulink model, such as the Coverage function, can be used to analyze the coverage of test cases. When there is a situation of repeated coverage of test cases, the redundant test cases can be fed back to the use case garbage library 360. When there is no repeated coverage of test cases, the experience learned during the analysis of the coverage of test cases can also be input into the use case experience library 330 for subsequent learning and reference from the use case experience library. Among them, the process of analyzing the coverage range of test cases corresponds to Figure 2 step S236 in

[0115] As described above in conjunction with Figures 1 to 3 , the method embodiments of the present application have been described in detail. Next, the apparatus embodiments of the present application will be described in detail in conjunction with Figures 4 to 5 It should be understood that the descriptions of the method embodiments and the apparatus embodiments correspond to each other. Therefore, the parts not described in detail can be referred to the previous method embodiments.

[0116] Figure 4 FIG. 400 is a test case generation apparatus provided by an embodiment of the present application. The apparatus 400 includes an acquisition unit 410 and a processing unit 420.

[0117] The acquisition unit 410 is configured to acquire model information of the model to be tested, where the model information includes test logic information and / or data format information of the model to be tested;

[0118] The processing unit 420 is configured to input the model information of the model to be tested into the AI model to generate multiple test cases for the model to be tested, and the multiple test cases are used to test the test logic of the model to be tested.

[0119] In some implementations, the processing unit 420 is configured to: input the multiple test cases into the model to be tested to obtain the output result corresponding to each test case in the multiple test cases; compare the data format of the output result with the data format of the output result of the model to be tested to obtain a comparison result; and screen out valid test cases from the multiple test cases according to the comparison result, where the data format of the output result corresponding to the valid test cases matches the data format of the output result of the model to be tested.

[0120] In some implementations, the processing unit 420 is configured to: generate a vector corresponding to each test case in the multiple test cases; perform a similarity detection on the vectors corresponding to the multiple test cases to obtain the similarity between the vectors corresponding to the multiple test cases; and screen out the valid test cases of the model to be tested from the multiple test cases according to the similarity between the vectors corresponding to the multiple test cases, where the similarity between the vectors corresponding to the valid test cases of the model to be tested is greater than a similarity threshold.

[0121] In some implementations, the processing unit 420 is configured to: determine the coverage range of each test case in the multiple test cases; and screen out multiple valid test cases of the model to be tested from the multiple test cases according to the coverage range of each test case in the multiple test cases, where the coverage ranges corresponding to the multiple valid test cases of the model to be tested do not overlap.

[0122] In some implementations, the model to be tested is a Simulink model.

[0123] Figure 5 is a schematic block diagram of a computing device according to an embodiment of the present application. Figure 5 The illustrated computing device 500 may include: a memory 510, a processor 520, and an input / output interface 530. Among them, the memory 510, the processor 520, and the input / output interface 530 are connected through an internal connection path. The memory 510 is used to store instructions, and the processor 520 is used to execute the instructions stored in the memory 510 to control the input / output interface 530 to receive input data and information and output operation result data, etc.

[0124] In some implementations, the processor 520 is configured to: obtain the model information of the model to be tested, where the model information includes the test logic information and / or the data format information of the model to be tested; and input the model information of the model to be tested into an AI model to generate multiple test cases for the model to be tested, where the multiple test cases are used to test the test logic of the model to be tested.

[0125] In some implementations, the processor 520 is configured to: input the multiple test cases into the model to be tested to obtain the output result corresponding to each test case among the multiple test cases; compare the data format of the output result with the data format of the output result of the model to be tested to obtain a comparison result; and screen out valid test cases from the multiple test cases according to the comparison result, where the data format of the output result corresponding to the valid test cases matches the data format of the output result of the model to be tested.

[0126] In some implementations, the processor 520 is configured to: generate a vector corresponding to each test case among the multiple test cases; perform a similarity detection on the vectors corresponding to the multiple test cases to obtain the similarity between the vectors corresponding to the multiple test cases; and screen out the valid test cases of the model to be tested from the multiple test cases according to the similarity between the vectors corresponding to the multiple test cases, where the similarity between the vectors corresponding to the valid test cases of the model to be tested is greater than a similarity threshold.

[0127] In some implementations, the processor 520 is configured to: determine the coverage range of each test case among the multiple test cases; and screen out multiple valid test cases of the model to be tested from the multiple test cases according to the coverage range of each test case among the multiple test cases, where the coverage ranges corresponding to the multiple valid test cases of the model to be tested do not overlap.

[0128] In some implementations, the model to be tested is a Simulink model.

[0129] It should be understood that in the embodiments of the present application, the processor 520 may be a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0130] The memory 510 may include a read-only memory and a random access memory, and provide instructions and data to the processor 520. A part of the processor 520 may further include a non-volatile random access memory. For example, the processor 520 may also store information about the device type.

[0131] In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 520 or the instructions in the form of software. The method for requesting uplink transmission resources disclosed in the embodiments of the present application can be directly embodied as being executed and completed by a hardware processor, or executed and completed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 510, and the processor 520 reads the information in the memory 510 and combines its hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0132] It should be understood that in the embodiments of the present application, the processor 520 may be a central processing unit (CPU), and the processor 520 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0133] The embodiments of the present application also provide a computer program product, which includes: computer program code. When the computer program code runs on a computer, the computer is enabled to execute the method in any of the above embodiments.

[0134] The embodiments of the present application also provide a computer-readable storage medium, which stores program code. When the computer program code runs on a computer, the computer is enabled to execute the method in any of the above embodiments.

[0135] It should be understood that in the embodiments of the present application, "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0136] It should be understood that the term " / and" in this article is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after.

[0137] It should be understood that in various embodiments of the present application, the magnitude of the sequence numbers of the above processes does not imply the order of execution, and the order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0138] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in an electrical, mechanical, or other form.

[0139] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0141] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be read by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a digital video disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0142] As described above, the above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for generating a test case, characterized in that: include: Acquire model information of a model to be tested, wherein the model information includes test logic information of the model to be tested and / or data format information of the model to be tested; The model information of the model to be tested is input into an artificial intelligence AI model to generate multiple test cases for the model to be tested, and the multiple test cases are used to test the test logic of the model to be tested.

2. The method according to claim 1, characterized in that The method further comprises: Input the multiple test cases into the model to be tested to obtain an output result corresponding to each test case in the multiple test cases; Comparing the data format of the output result with the data format of the output result of the model to be tested to obtain a comparison result; Valid test cases are screened out from the multiple test cases according to the comparison result, and the data format of the output results corresponding to the valid test cases matches the data format of the output results of the model to be tested.

3. The method according to claim 1, characterized in that The method further comprises: Generate a vector corresponding to each test case in the multiple test cases; Performing similarity detection on the vectors corresponding to the multiple test cases to obtain similarities between the vectors corresponding to the multiple test cases; According to the similarity between the vectors corresponding to the multiple test cases, valid test cases for the model to be tested are screened from the multiple test cases, and the similarity between the vectors corresponding to the valid test cases for the model to be tested is greater than a similarity threshold.

4. The method according to claim 1, characterized in that The method further comprises: Determining coverage of each test case in the plurality of test cases; According to the coverage of each test case in the multiple test cases, multiple valid test cases of the model to be tested are screened from the multiple test cases, and the coverages corresponding to the multiple valid test cases of the model to be tested do not overlap with each other.

5. The method according to claim 1, characterized in that The model to be tested is a Simulink model.

6. A test case generation device, characterized in that: include: An acquiring unit, configured to acquire model information of a model to be tested, wherein the model information includes test logic information of the model to be tested and / or data format information of the model to be tested; A processing unit is used to input the model information of the model to be tested into the AI ​​model to generate multiple test cases for the model to be tested, and the multiple test cases are used to test the test logic of the model to be tested.

7. The device according to claim 6, characterized in that The processing unit is used for: Input the multiple test cases into the model to be tested to obtain an output result corresponding to each test case in the multiple test cases; Comparing the data format of the output result with the data format of the output result of the model to be tested to obtain a comparison result; Valid test cases are screened out from the multiple test cases according to the comparison result, and the data format of the output results corresponding to the valid test cases matches the data format of the output results of the model to be tested.

8. The device according to claim 6, characterized in that The processing unit is used for: Generate a vector corresponding to each test case in the multiple test cases; Performing similarity detection on the vectors corresponding to the multiple test cases to obtain similarities between the vectors corresponding to the multiple test cases; According to the similarity between the vectors corresponding to the multiple test cases, valid test cases for the model to be tested are screened from the multiple test cases, and the similarity between the vectors corresponding to the valid test cases for the model to be tested is greater than a similarity threshold.

9. The device according to claim 6, characterized in that The processing unit is used for: Determining coverage of each test case in the plurality of test cases; According to the coverage of each test case in the multiple test cases, multiple valid test cases of the model to be tested are screened from the multiple test cases, and the coverages corresponding to the multiple valid test cases of the model to be tested do not overlap with each other.

10. The device according to claim 6, characterized in that The model to be tested is a Simulink model.

Images