Application program testing method and device, equipment and medium

Abstract

The embodiment of the invention relates to an application program testing method and device, equipment and a medium. The method comprises the steps of obtaining target training data after data enhancement; adjusting the initial model based on model evaluation to obtain a target model; training the target model based on the target training data to obtain a test case generation model; and inputting the tested function of the application program into the test case generation model to obtain a corresponding unit test case, and performing unit testing on the tested function of the application program based on the unit test case. By adopting the technical scheme, data enhancement is performed on the training data, the comprehensiveness of the training data is improved, and the trained basic model is adjusted according to the model evaluation, so that the basic model has better test case generation capability. The test case generation model is generated based on the adjusted training data and the basic model, and the unit test case generated by the test case generation model has a high coverage rate and a low error rate.

Description

Technical Field

[0001] The present disclosure relates to the field of testing technology, and in particular to a method, device, equipment and medium for testing an application. Background Art

[0002] In order to ensure the correctness of the application code, corresponding test cases can be generated for each function and other units of the application. In the related art, test cases can be generated through models, but the effect is not good, the coverage of test cases is low, and the error rate is high. Summary of the invention

[0003] In order to solve the above technical problems, the present disclosure provides a method, device, equipment and medium for testing an application.

[0004] The present disclosure provides a method for testing an application program, the method comprising:

[0005] Obtain target training data after data enhancement;

[0006] Adjust the initial model based on model evaluation to obtain the target model;

[0007] Training the target model based on the target training data to obtain a test case generation model;

[0008] The tested function of the application is input into the test case generation model to obtain a corresponding unit test case, and a unit test is performed on the tested function of the application based on the unit test case.

[0009] The present disclosure also provides a device for testing an application program, the device comprising:

[0010] An acquisition module is used to obtain target training data after data enhancement;

[0011] An adjustment module is used to adjust the initial model based on model evaluation to obtain a target model;

[0012] A training module, used to train the target model based on the target training data to obtain a test case generation model;

[0013] The input module is used to input the tested function of the application program into the test case generation model to obtain the corresponding unit test case, and perform unit test on the tested function of the application program based on the unit test case.

[0014] An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement a testing method for an application as provided in an embodiment of the present disclosure.

[0015] The embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the test method of the application program provided by the embodiment of the present disclosure.

[0016] The technical solution provided by the embodiment of the present disclosure has the following advantages over the prior art: the test solution for the application provided by the embodiment of the present disclosure obtains the target training data after data enhancement; adjusts the initial model based on the model evaluation to obtain the target model; trains the target model based on the target training data to obtain the test case generation model; inputs the tested function of the application into the test case generation model to obtain the corresponding unit test case, and performs unit testing on the tested function of the application based on the unit test case. By adopting the above technical solution, the training data is enhanced to improve the comprehensiveness of the training data, and the basic model for training is adjusted according to the model evaluation, so that the basic model has a better ability to generate test cases. In addition, a test case generation model is generated based on the adjusted training data and the basic model, and the unit test cases generated by the test case generation model can have a higher coverage rate and a lower error rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0018] Figure 1 A flowchart of a method for testing an application program provided by an embodiment of the present disclosure;

[0019] Figure 2 A flowchart of another application testing method provided by an embodiment of the present disclosure;

[0020] Figure 3 A schematic diagram of a structural splitting provided in an embodiment of the present disclosure;

[0021] Figure 4 A schematic diagram of a first corresponding relationship provided in an embodiment of the present disclosure;

[0022] Figure 5 A schematic diagram of a second corresponding relationship provided in an embodiment of the present disclosure;

[0023] Figure 6 A flowchart of another application testing method provided by an embodiment of the present disclosure;

[0024] Figure 7 A schematic diagram of a unit test case generation process provided by an embodiment of the present disclosure;

[0025] Figure 8 A schematic diagram of the structure of a testing device for an application program provided in an embodiment of the present disclosure;

[0026] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0028] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0029] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0030] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0032] In order to ensure the correctness of the application code, corresponding test cases can be generated for each function and other units of the application for testing. In the related art, in order to generate high-quality and high-coverage unit test cases, program analysis, testing technology, and testing algorithms can be integrated in the process of generating unit test cases, such as static analysis, dynamic analysis, genetic algorithms, symbolic execution, fuzz testing, mutation testing, etc. The unit test generation capability can be deployed in the client. After the unit test code is generated based on program analysis, random mutations and genetic algorithm iterations are performed to obtain branches with larger coverage, which are then screened through mutation testing to finally generate high-quality unit test cases. In the above-mentioned related art, the generation of test cases through models has the problem of poor effect, low coverage and high error rate of the generated unit test cases.

[0033] In order to solve the above problems, the embodiments of the present disclosure provide a method for testing an application program, which is introduced below in conjunction with specific embodiments.

[0034] Figure 1 The present invention provides a flowchart of a method for testing an application program, which can be executed by a testing device for the application program, wherein the device can be implemented by software and / or hardware and can generally be integrated in an electronic device. Figure 1 As shown, the method includes:

[0035] Step 101: Obtain target training data after data enhancement.

[0036] The target training data may be training data after data enhancement processing. This embodiment does not limit the programming language of the target training data. Data enhancement may be an enhancement processing performed on the training data. There are many ways to enhance data, which are not limited in this embodiment. For example, data enhancement may be achieved through format adjustment, or data enhancement may be achieved through a unit testing tool.

[0037] In the disclosed embodiment, the application program testing device may perform data enhancement processing on the initial training data to obtain the target training data. Alternatively, the application program testing device may obtain the target training data that has been subjected to data enhancement processing in advance.

[0038] In some embodiments of the present disclosure, there are multiple methods for obtaining target training data after data enhancement, which are not limited in this embodiment. Examples are as follows:

[0039] In an optional implementation, obtaining target training data after data enhancement includes: obtaining initial training data, and adjusting the format of the initial training data to obtain target training data.

[0040] The initial training data may be training data that has not been subjected to data enhancement processing, and the initial training data may be training data provided by an existing test case data set, and this embodiment does not limit the existing test case data set. The format adjustment may be splitting the training data into function-based units, and adjusting the corresponding relationship between the split functions.

[0041] For example, Figure 2 A flowchart of another application testing method provided by an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the format of the initial training data is adjusted to obtain the target training data, including:

[0042] Step 201: Structurally split the unit test cases in the initial training data to obtain multiple test functions.

[0043] Unit testing can be to generate corresponding test code for each tested function included in each class that needs to be tested in the application so as to test each method through the test code, that is, each tested function is regarded as a unit to be tested, and the application is tested through unit testing. Unit test cases can be test codes generated for each tested function in the application for testing. The unit test cases can be stored in a pre-set code repository. The test function can be the function used to test the tested function in the unit test case.

[0044] In this embodiment, the test device of the application can scan the code warehouse to obtain the unit test cases stored in the code warehouse, and use the parsing tool to split the unit test cases to obtain multiple separate functions, which are multiple test functions. The parsing tool can be a parser generation tool and an incremental parsing library. The parsing tool can build a specific syntax tree for the source code file and support the parsing of multiple programming languages. When used specifically, the dynamic library corresponding to the programming language can be compiled, and then the dynamic library is loaded to write a parsing script for parsing.

[0045] Figure 3 A schematic diagram of a structural splitting provided in an embodiment of the present disclosure, such as Figure 3 As shown, the parsing tool can parse a class (Class) into attributes (Fields), constructors (Constructs), and functions (Methods), and parse functions into permissions (Access), signatures (Signature), and specific implementations (Body).

[0046] Step 202: Establish a first correspondence between the tested functions of the application and the test functions, wherein each tested function in the first correspondence corresponds to one test function.

[0047] The application may be an application for unit testing, and this embodiment does not limit the functions implemented by the application. The tested function may be a method in a target class (Target Class) of the application that needs to be tested. A class may encapsulate multiple methods, functions, and variables, and the number of tested functions may be one or more. The first correspondence may be a correspondence between a test function and a tested function recorded in a one-to-one manner.

[0048] In this embodiment, after the application testing device determines the test functions included in each unit test case, it can determine the tested function tested by each test function, and determine the relationship of one test function corresponding to one tested function as the first corresponding relationship.

[0049] Figure 4 A schematic diagram of a first corresponding relationship provided in an embodiment of the present disclosure, such as Figure 4 As shown, test file 1 includes test function 1 and test function 2. Test function 1 tests tested functions 1 and 2, and test function 2 tests tested functions 3 and 4. In the first correspondence, it is recorded that: test function 1 corresponds to tested function 1, test function 1 corresponds to tested function 2, test function 2 corresponds to tested function 3, and test function 2 corresponds to tested function 4. That is, in the first correspondence, the test function and the tested function are converted into a one-to-one style.

[0050] Step 203: convert the first corresponding relationship into a second corresponding relationship in which one tested function corresponds to multiple test functions, and combine the tested function, the test function and the second corresponding relationship to obtain target training data.

[0051] The second corresponding relationship may be a corresponding relationship between a test function and a test function based on a record of the tested function. In the second corresponding relationship, one or more test functions corresponding to a tested function may be determined based on the tested function as a statistical basis.

[0052] In this embodiment, the testing device of the application can parse the first correspondence relationship, determine the test function and the tested function recorded in a one-to-one manner, and use the tested function as a statistical basis to merge the first correspondence relationships with the same tested function to obtain a second correspondence relationship in which one tested function corresponds to multiple test functions, and determine the tested function, the test function, and the second correspondence between the tested function and the test function as the target training data.

[0053] Figure 5 A schematic diagram of a second corresponding relationship provided in an embodiment of the present disclosure, such as Figure 5As shown, in the first corresponding relationship, test function 1, test function 3 and other test functions all correspond to the tested function 1. Further, with the tested function 1 as the statistical basis, in the second corresponding relationship, the tested function 1 corresponds to the test function 1, test function 3 and other test functions at the same time.

[0054] In the above scheme, since there are many test branches for each tested function in the actual unit testing process, different test branches need to be tested to cover more abnormal situations. That is, in the actual unit testing process, multiple test functions can test the same tested function. Therefore, in the second corresponding relationship of the target training data, one tested function corresponds to multiple test functions, and the second corresponding relationship is similar to the relationship between the tested function and the test function in the actual unit testing process, so it can improve the coverage of the unit test cases generated subsequently.

[0055] In another optional implementation, obtaining the target training data after data enhancement includes: generating the target training data after data enhancement through a unit testing tool.

[0056] The unit testing tool may be a tool that can automatically generate unit testing cases, and this embodiment does not limit the unit testing tool.

[0057] In this embodiment, the tested function in the training data is processed by the unit test tool to generate corresponding unit test cases, and the test device of the application program can compress (minimize) the unit test cases so that one test function tests one tested function. Further, the relationship between the test function and the tested function is converted so that one tested function corresponds to multiple test functions, and high-quality target training data is obtained. The data volume of the target training data in this embodiment is not limited. For example, the data volume of the target training data can be about 70,000.

[0058] In the above scheme, the generation of target training data is realized based on the existing unit testing tool, and the target training data can be generated quickly. Based on the better performance of the unit testing tool, the coverage of the subsequent unit test cases generated based on the target training data can be improved, and the error rate of the unit test cases can be reduced.

[0059] Step 102: Adjust the initial model based on the model evaluation to obtain the target model.

[0060] Among them, model evaluation can be an evaluation of the performance of the model on the unit test generation task. The evaluation dimensions of model evaluation can be multiple, and this embodiment does not limit it. The initial model can be an initially determined basic model. The initial model can be a medium model, and the medium model can be a model with a smaller parameter amount than the large model and larger than the small model. This embodiment does not limit the parameter amount of the initial model. For example, the parameter amount of the initial model can be between 500M and 30B. The target model can be the final determined basic model. The basic model can be the original model on which the model training depends, and the basic model can be understood as the original model before the unit test case generation function training is performed.

[0061] In the disclosed embodiment, the testing device of the application program may perform model evaluation on the initial model, and adjust the initial model as a whole or an initial sub-model in the initial model according to the result of the model evaluation to obtain a target model.

[0062] In some embodiments of the present disclosure, adjusting the initial model based on model evaluation to obtain a target model includes:

[0063] A plurality of sub-models are obtained, and a plurality of index values ​​of each sub-model in a plurality of dimensions are determined based on model evaluation; a target sub-model whose multiple index values ​​in a plurality of dimensions are all the largest among the plurality of sub-models is determined, and the initial model is adjusted based on the target sub-model to obtain a target model.

[0064] Among them, the sub-model can be a candidate model. The index value can be the result value corresponding to a dimension when evaluating the model, that is, an index value can be determined for the model for each dimension, and then multiple index values ​​can be obtained. The target sub-model can be the sub-model with the best model evaluation result, that is, the target sub-model can be the sub-model with the best index value.

[0065] In this embodiment, the above-mentioned multiple dimensions may include at least one of static analysis results, complexity, dynamic analysis results, test correctness, and coverage. Among them, static analysis results and complexity may be dimensions related to availability, and availability may characterize the degree to which the test cases generated by the model can be compiled and used. Dynamic analysis results, test correctness, and coverage may be dimensions related to effectiveness, and effectiveness may characterize the degree to which the test results of the unit test cases generated by the model are effective.

[0066] Specifically, the static analysis results can record the code executableness of the test cases generated by the model, and the static analysis results can also be understood as parameters that record the syntactic correctness of the test cases. The static analysis results may include: at least one of the number of compilation errors and the percentage of syntax errors (Syntax Error), and the number of compilation errors may be the number of compilation errors determined by diagnostic processing (Diagnostic). The complexity can record the complexity of the test cases generated by the model, and the complexity can be determined based on the number of function calls contained in the test cases. The complexity may include: at least one of the complexity of the original (Predict) unit test cases output by the sub-model and the complexity of the unit test cases after compilation (Compile).

[0067] The dynamic analysis result may include at least one of a test pass rate (Passing) and a test failure rate (Failing). The test pass rate may be the percentage of unit test cases with test pass results. The test failure rate may be the percentage of unit test cases with test failure results.

[0068] Test correctness can be a parameter that records whether the unit test case correctly calls the test target. Specifically, since the test result of an invalid unit test case is also a test pass, but the invalid unit test case only includes meaningless statements and does not correctly call the test target (i.e., the tested function). Therefore, the unit test case must contain the code that correctly calls the tested function and run through to be considered as having test correctness. The test correctness may include: at least one of the number of correct unit test cases and the number of tested functions that are correctly tested.

[0069] The coverage rate may be a parameter for measuring the effectiveness of a unit test case. The coverage rate may represent the coverage of the target code during the test process. The coverage rate may be the proportion of the target code covered by the unit test case.

[0070] In the disclosed embodiment, the testing device of the application can obtain multiple sub-models, and perform model evaluation on each sub-model from multiple dimensions to obtain the index value of each sub-model in different dimensions. Further, the target sub-model is determined according to the index value.

[0071] There are many methods for determining the target sub-model based on the index value, which is not limited in this embodiment. In an optional implementation, the test device of the application can determine the sub-model with the largest index values ​​as the target sub-model. If there is no sub-model with the largest index values, the test device of the application can adjust the sub-model until a sub-model with the largest index values ​​is obtained, and determine the sub-model as the target sub-model.

[0072] In another optional implementation, the testing device of the application can obtain a scenario identifier, and query in the identifier-indicator correspondence relationship according to the scenario identifier, determine the target indicator value in the indicator value, and determine the submodel with the largest target indicator value as the target submodel. Among them, the identifier-indicator correspondence relationship can be a correspondence between the recorded scenario identifier and the indicator value. This embodiment does not limit the identifier-indicator correspondence relationship. For example, the target indicator value corresponding to the scenario identifier that represents the direct running of the unit test case can be availability. The target indicator value corresponding to the scenario identifier that represents the running of the unit test case after manual inspection can be effectiveness.

[0073] After determining the target sub-model, the test device of the application can adjust the initial model based on the target sub-model to obtain the target model. The specific adjustment method can include replacing part of the model in the initial model with the target sub-model, or directly determining the target sub-model as the target model.

[0074] In the above scheme, the sub-model with the highest values ​​of multiple indicators in multiple dimensions is determined as the target sub-model, and the initial model is adjusted based on the target sub-model, so that the target model as the basic model has better performance in generating test cases.

[0075] Step 103: Train the target model based on the target training data to obtain a test case generation model.

[0076] The test case generation model may be a model for generating unit test cases for an application.

[0077] In the disclosed embodiment, after determining the target training data and the target model, the application test device can use the target training data to train the target model to obtain a test case generation model. There are many methods for training the target model based on the target training data, which are not limited in this embodiment.

[0078] Step 104: input the tested function of the application into the test case generation model to obtain the corresponding unit test case, and perform unit test on the tested function of the application based on the unit test case.

[0079] Among them, this embodiment does not limit the function, number, etc. of the tested function. The unit test case can be a test case for unit testing an application. This embodiment does not limit the number of the unit test cases. For example, the number of the unit test cases can be multiple, and then the unit test cases can be used to unit test the tested function in the application from multiple dimensions.

[0080] In an embodiment of the present disclosure, after the generation of the test case generation model is completed, the test device of the application can obtain the tested function that needs to be unit tested in the application, and input the tested function into the test case generation model, and the test case generation model outputs at least one unit test case. After obtaining the at least one unit test case, the tested function is unit tested in at least one dimension using the at least one unit test case to obtain one or more unit test results of the tested function.

[0081] The test scheme for the application provided by the embodiment of the present disclosure obtains the target training data after data enhancement; adjusts the initial model based on the model evaluation to obtain the target model; trains the target model based on the target training data to obtain the test case generation model; inputs the tested function of the application into the test case generation model to obtain the corresponding unit test case, and performs unit test on the tested function of the application based on the unit test case. The above technical scheme is adopted to perform data enhancement on the training data, thereby improving the comprehensiveness of the training data, and adjusting the basic model for training according to the model evaluation, so that the basic model has a better ability to generate test cases. In addition, a test case generation model is generated based on the adjusted training data and the basic model, and the unit test cases generated by the test case generation model can have a higher coverage rate and a lower error rate.

[0082] For example, Figure 6 A flowchart of another application testing method provided by the embodiment of the present disclosure is shown as follows: Figure 6 As shown, in some embodiments, training a target model based on target training data to obtain a test case generation model may include:

[0083] Step 601: extract a first prompt word in an indication format for each sample data in the target training data, wherein the indication format includes a task description, input data, and target content of output data, and the target content of the input data is context.

[0084] The target training data may include a plurality of sample data, and the sample data may include a tested function and a test function corresponding to the tested function recorded in the first correspondence. The indication format may be a format including a field and a target content pre-set for the prompt word. The target content may be the specific content included in the field.

[0085] Prompt words can be a medium for users to interact with models. The prompt words can be understood as a piece of text or command used to guide and trigger the model to perform a specific task or a specific response. Through prompt words, interaction between users and models based on instruction formats can be achieved. Task description (Instruction) can be a text that describes the task that the model needs to perform. Input (Input) data can be data input into the model. Output (Output) data can be data output after the model executes the task, and the output data can be a test function. Context (Context) can be the background environment for executing the tested function, and the background environment can include the function configuration environment.

[0086] For example, the first prompt word can be: {"instruction":"Generate tests for the\"target method signature\"""input":"context","output":"test1;test2"}, where "Generate tests for the\"target method signature\"" is the task description; "context" is the input data; and "test1;test2" is the output data.

[0087] In this embodiment, the test device of the application program can parse the target training data to obtain a plurality of sample data included in the target training data. For each sample data, the task description, context, and test function in the sample data are extracted respectively. Furthermore, the context is used as the input data of the model, and the test function is used as the output data of the model. Furthermore, a first prompt word including the task description, input data, and output data is generated.

[0088] In some embodiments of the present disclosure, the target content of the output data is a unit test case of the function under test, and the unit test case includes a case start header.

[0089] The use case start header may be a common start code portion of a unit test case. For example, the use case start header may include: "public class TestArrayUtils{@Test public void testtoString(){".

[0090] In this embodiment, the use case start header of the unit test case can be supplemented in the output data so that the test case generation model can have the function of continuing to write the test case based on the use case start header. This method of continuing to write based on the use case start header can reduce the possibility of the test case generation model generating invalid code, and enable the test case generation model to directly generate the corresponding unit test case.

[0091] Step 602: truncate and clean the context of the first prompt word corresponding to each sample data to obtain a second prompt word.

[0092] The truncation may be the interception of a part of the context according to the code type. The cleaning may be the removal of the part of the prompt word or the context that exceeds the preset length. The first prompt word may be the prompt word that has not been truncated and cleaned. The second prompt word may be the prompt word obtained by intercepting and cleaning the first prompt word.

[0093] In some embodiments of the present disclosure, the context in the first prompt word corresponding to each sample data is truncated and cleaned to obtain the second prompt word, including:

[0094] The methods and properties of other classes except the tested function in the first prompt words corresponding to each sample data are filtered and combined according to a preset format to obtain intermediate prompt words; the intermediate prompt words are cleaned according to a preset length to obtain second prompt words.

[0095] Among them, the method of other classes can be a method with a permission (Access) of private (Private), and the attribute of other classes can be an attribute with a permission of private. The preset format can be an arrangement format of the content in a preset context. The middle prompt word can be a prompt word that completes the truncation process but does not complete the cleaning process. The preset length can be a maximum length of the preset code, and there are multiple preset lengths, which are not limited in this embodiment. For example, the preset length can be 2k, and the preset length can be determined according to the processing speed of the basic model for input data of different lengths.

[0096] In some embodiments of the present disclosure, the preset format includes the function under test, a set of public classes, a constructor, signatures of other methods other than the function under test, and properties. Among them, the constructor can be a function that implements object initialization. Other methods can be understood as other functions other than the function under test in the context.

[0097] In this embodiment, the preset format may include the function under test after removing methods and properties with private permissions, a public class set, a constructor, signatures of other methods except the function under test, and properties.

[0098] In the above implementation, the preset format includes the signatures and properties of the tested function, a set of public classes, a constructor, and other methods other than the tested function. The prompt word determined according to the preset format retains more comprehensive information in the context, increases the amount of information contained in the second prompt word, and thereby enables the test case generation model trained based on the second prompt word to have a lower error rate.

[0099] In an optional implementation, the preset format may include: the function under test, a set of public classes, a constructor, signatures and properties of other methods except the function under test, a set of public classes of the associated classes of the function under test, and constructors of the associated classes. Among them, the associated classes of the function under test may be classes associated with the function under test determined based on function call chain analysis. The preset format additionally includes information about the associated classes of the function under test, so that in the context of the prompt word, not only the information about the function under test but also the information about the associated classes of the function under test can be recorded, thereby improving the richness of the input data.

[0100] In another optional implementation, the preset format may include one of the following: a tested function, a set of public classes; or, a tested function, a set of public classes, a constructor; or, a signature of a tested function, a set of public classes, a constructor, or other methods other than the tested function.

[0101] In this embodiment, the test device of the application can obtain the first prompt word corresponding to each sample data, and extract the context in the first prompt word. For the context, the methods and classes with private permissions are eliminated to obtain the first intermediate context, and the content in the first intermediate context is eliminated and combined according to the code type recorded in the preset format to obtain the second intermediate context. The prompt word including the second intermediate context is determined as the intermediate prompt word. Furthermore, if the length of the intermediate prompt word exceeds the preset length, the part of the intermediate prompt word that exceeds the preset length is eliminated to obtain the second prompt word.

[0102] In the above scheme, by intercepting the context, the content included in the context of the prompt word is unified, so that the model can be trained more effectively. By cleaning the prompt word, the speed of the model processing the prompt word is improved.

[0103] Optionally, in some embodiments of the present disclosure, after the intermediate prompt word is cleaned according to a preset length to obtain the second prompt word, the testing method of the application further includes:

[0104] If the length of the middle prompt word is greater than the preset length more than the preset number of times, the preset format is adjusted to a new preset format, and the candidate second prompt word is determined based on the new preset format; wherein the new preset format includes the tested function, the public class set, and the constructor.

[0105] In the above scheme, when the middle prompt word is frequently longer than the preset length, the less important content in the upper and lower parts is reduced, and the length of the prompt word is reduced, thereby avoiding frequent cleaning operations on the prompt word according to the preset length, and avoiding the deletion of important content in the prompt word due to truncation according to a fixed length.

[0106] Step 603: input the second prompt word of each sample data of the target training data into the target model for training, and adjust the parameters in a full adjustment manner during the training process to obtain a test case generation model.

[0107] The full adjustment method may be a training method in which all parameters in the target model can be adjusted during the training process, that is, the parameters in the target model are not fixed during the training process.

[0108] In this embodiment, after determining to use the target model as the basic model, the application program's testing device may continue pre-training the target model to obtain a target model that has completed the continued pre-training. And after determining the second prompt word, the application program's testing device may input the second prompt word corresponding to each sample data into the target model that has completed the continued pre-training, adjust the model parameters in the target model that has completed the continued pre-training, until the target model meets the pre-set training end condition, and determine the target model as the test case generation model.

[0109] In some embodiments of the present disclosure, an adjustment network may be pre-set after the target model. Accordingly, after the second prompt word of each sample data of the target training data is input into the target model for training, the test device of the application program may fix the model parameters of the target model unchanged, adjust the parameters of the adjustment network, thereby realizing the training of the target model and the adjustment network as a whole, and using the trained target model and the adjustment network as a whole as the test case generation model.

[0110] In the above scheme, the task description, input data, and output data in the sample data are extracted as the first prompt word, so that the first prompt word has better information comprehensiveness. In addition, the first prompt word is truncated and cleaned to obtain the second prompt word. When the effective content of the information carried by the prompt word has little impact, the amount of prompt word data is appropriately reduced, and the efficiency of data processing is improved while ensuring the training effect of the model.

[0111] In some embodiments of the present disclosure, after inputting the tested function of the application into the test case generation model to obtain the corresponding unit test case, the testing method of the application further includes:

[0112] Repair unit test cases based on extraction rules and / or completion rules.

[0113] Among them, the extraction rule is also called the unit test case extraction rule, and the extraction rule can be a keyword-based method extraction and combination rule in the unit test case. The keyword can be a word contained in the pre-set method to be extracted. The completion rule also declares the completion rule. The completion rule can be a rule for extracting variable types using a parsing tool and completing the call path according to the variable type. The call path can be the path based on which the call is made, and the call path can be understood as a declaration (Import) path.

[0114] In the process of generating unit test cases from the test case generation model, truncation may occur, which may cause the unit test cases to fail to compile. In this embodiment, methods are extracted by keywords, and further, the extracted methods are recombined so that the combined new methods are available. In the process of repairing the unit test cases based on the completion rules, the test device of the application can collect the source files of the unit test cases, and use the parsing tool to extract the variable types of the unit test cases, and further, match and complete the call path according to the extraction results of the variable types.

[0115] In the above scheme, by repairing the unit test cases with extraction rules and / or completion rules, the compiling passability of the unit test cases is improved, the availability of the unit test cases is improved, and the human resources consumed for repair are reduced.

[0116] Figure 7 A schematic diagram of a unit test case generation process provided by an embodiment of the present disclosure, such as Figure 7As shown, the tested function and the unit test case are obtained, and multiple test functions are obtained by grammatical analysis (AstAnalyzer) of the unit test case, and then the second correspondence relationship between one tested function and multiple test functions is determined. In addition, the target training data is obtained according to the combination of the tested function, the test function and the second correspondence relationship, and each sample data in the target training data is extracted according to the indication format (Instrcut Format) to obtain the corresponding prompt word. The prompt word includes: task description (TaskInfo), input data, and output data. Among them, the target content of the input data is the relevant context (RelativityContext), and the target content of the input data is the test function. Further, the prompt word is input into the target model, the target model is trained, and the test case generation model is obtained. The target model is a basic model determined by pre-training and fine-tuning in advance. After obtaining the test case generation model, the tested function is input into the test case generation model to obtain the corresponding unit test case. The test device of the application can repair the unit test case, and use the unit test case to test the tested function to obtain the test result. And the test case generation model is evaluated based on the test result.

[0117] A model with the ability to generate unit tests needs to have one or more of the following capabilities: test code format adjustment capability, task understanding capability, and code semantics understanding capability. The above capabilities can correspond to each stage of determining the test case generation model. Then, by making targeted adjustments to the model, the effect of the model on the unit test generation task can be optimized. In the pre-training stage, the model's understanding of language semantics is affected. Accordingly, the results of the basic model and the pre-training data set need to be adjusted accordingly. In the model fine-tuning stage, the model's understanding of the language and the unit test generation task are affected. Accordingly, the collection of training data needs to be adjusted, and the training data needs to be enhanced, and the organization of training data and the model's fine-tuning training method need to be adjusted. In the model fine-tuning stage, the model's understanding of the format is affected. Accordingly, high-quality data needs to be collected and the data organization needs to be adjusted. In the prompt word stage, the model's understanding of the task from the input command and the format of the data returned by the model are affected. Accordingly, different task generation methods need to be tried.

[0118] In summary, the pre-training stage of the basic model affects the basic level of the model's understanding of language and semantics after the training, so it can be improved accordingly by selecting different basic models.

[0119] The disclosed embodiment designs a model generation scheme for the unit test generation task, which can improve the performance of the model in the unit test generation task, and is not limited by the number of basic model parameters. It can be applied to the basic model with the number of parameters ranging from 500M to 13B, which affects the cost of subsequent model training and model deployment. In addition, by enhancing the training data, generating prompt words based on the training data, and fine-tuning the target model, the effect of the medium model in the single-source test case generation task is improved. The coverage of the test case generation model in the disclosed embodiment is significantly compared with the existing model, the coverage can be increased by 164%, the compilation error rate can be reduced by 35%, and it can be implemented faster.

[0120] Figure 8 This is a schematic diagram of the structure of a testing device for an application provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated in an electronic device. Figure 8 As shown, the test fixture for this application includes:

[0121] An acquisition module 801 is used to acquire target training data after data enhancement;

[0122] An adjustment module 802 is used to adjust the initial model based on the model evaluation to obtain a target model;

[0123] A training module 803 is used to train the target model based on the target training data to obtain a test case generation model;

[0124] The input module 804 is used to input the tested function of the application program into the test case generation model to obtain corresponding unit test cases, and perform unit testing on the tested function of the application program based on the unit test cases.

[0125] Optionally, the acquisition module 801 includes:

[0126] A first acquisition submodule is used to acquire initial training data, and adjust the format of the initial training data to obtain the target training data;

[0127] Alternatively, the second acquisition submodule is used to generate target training data after data enhancement through a unit testing tool.

[0128] Optionally, the first acquisition submodule is used to:

[0129] Structurally splitting the unit test cases in the initial training data to obtain multiple test functions;

[0130] Establishing a first correspondence between the tested function of the application and the test function, wherein each of the tested functions in the first correspondence corresponds to one of the test functions;

[0131] The first corresponding relationship is converted into a second corresponding relationship in which one of the tested functions corresponds to a plurality of the test functions, and the tested function, the test function and the second corresponding relationship are combined to obtain the target training data.

[0132] Optionally, the adjustment module 802 is used to:

[0133] Acquire multiple sub-models, and determine multiple indicator values ​​of each of the sub-models in multiple dimensions based on model evaluation;

[0134] A target sub-model whose multiple index values ​​in the multiple dimensions are all maximum among the multiple sub-models is determined, and the initial model is adjusted based on the target sub-model to obtain the target model.

[0135] Optionally, the training module 803 includes:

[0136] An extraction submodule, configured to extract a first prompt word in an indication format from each sample data in the target training data, wherein the indication format includes a task description, input data, and target content of output data, and the target content of the input data is a context;

[0137] A processing submodule, used for truncating and cleaning the context in the first prompt word corresponding to each of the sample data to obtain a second prompt word;

[0138] The training submodule is used to input the second prompt word of each sample data of the target training data into the target model for training, and adjust the parameters in a full adjustment manner during the training process to obtain the test case generation model.

[0139] Optionally, a processing submodule is used to:

[0140] Filter the methods and properties of other categories except the tested function in the first prompt words corresponding to each of the sample data and combine them according to a preset format to obtain an intermediate prompt word;

[0141] The intermediate prompt word is cleaned according to a preset length to obtain a second prompt word.

[0142] Optionally, the preset format includes a function under test, a set of public classes, a constructor, signatures of other methods except the function under test, and properties.

[0143] Optionally, the target content of the output data is a unit test case of the function under test, and the unit test case includes a case start header.

[0144] Optionally, the application program testing device 800 further includes:

[0145] The repair module is used to input the tested function of the application into the test case generation model, obtain the corresponding unit test case, and then repair the unit test case based on the extraction rule and / or the completion rule.

[0146] Optionally, the extraction rules are used to extract and combine methods in the unit test case based on keywords; the completion rules are used to extract variable types using a parsing tool and complete the call path according to the variable type.

[0147] The application testing device provided in the embodiments of the present disclosure can execute the application testing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0148] A computer program product, comprising a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the test method of the above application program are implemented

[0149] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.

[0150] The following specific reference Fig. 9 , which shows a schematic diagram of the structure of an electronic device 900 suitable for implementing the embodiment of the present disclosure. The electronic device 900 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0151] like Fig. 9 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0152] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0153] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the test method of the application program of the embodiment of the present disclosure are executed.

[0154] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0155] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0156] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0157] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains the target training data after data enhancement; adjusts the initial model based on the model evaluation to obtain the target model; trains the target model based on the target training data to obtain a test case generation model; inputs the tested function of the application into the test case generation model to obtain the corresponding unit test case, and performs unit testing on the tested function of the application based on the unit test case.

[0158] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0159] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0160] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.

[0161] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0162] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0163] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute any of the application program testing methods provided by the present disclosure.

[0164] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0165] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0166] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0167] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for testing an application, characterized in that: include: Obtain target training data after data enhancement; Adjust the initial model based on model evaluation to obtain the target model; Training the target model based on the target training data to obtain a test case generation model; The tested function of the application is input into the test case generation model to obtain a corresponding unit test case, and a unit test is performed on the tested function of the application based on the unit test case.

2. The method according to claim 1, characterized in that Obtain the target training data after data enhancement, including: Acquire initial training data, and adjust the format of the initial training data to obtain the target training data; Alternatively, generate target training data after data augmentation through the unit testing tool.

3. The method according to claim 2, characterized in that The format of the initial training data is adjusted to obtain the target training data, including: Structurally splitting the unit test cases in the initial training data to obtain multiple test functions; Establishing a first correspondence between the tested function of the application and the test function, wherein each of the tested functions in the first correspondence corresponds to one of the test functions; The first corresponding relationship is converted into a second corresponding relationship in which one of the tested functions corresponds to a plurality of the test functions, and the tested function, the test function and the second corresponding relationship are combined to obtain the target training data.

4. The method according to claim 1, characterized in that: Based on the model evaluation, the initial model is adjusted to obtain the target model, including: Acquire multiple sub-models, and determine multiple indicator values ​​of each of the sub-models in multiple dimensions based on model evaluation; A target sub-model whose multiple index values ​​in the multiple dimensions are all maximum among the multiple sub-models is determined, and the initial model is adjusted based on the target sub-model to obtain the target model.

5. The method according to claim 1, characterized in that The target model is trained based on the target training data to obtain a test case generation model, including: Extracting a first prompt word in an indication format from each sample data in the target training data, wherein the indication format includes a task description, input data, and target content of output data, and the target content of the input data is a context; Truncating and cleaning the context in the first prompt word corresponding to each of the sample data to obtain a second prompt word; The second prompt word of each sample data of the target training data is input into the target model for training, and parameters are adjusted in a full adjustment manner during the training process to obtain the test case generation model.

6. The method according to claim 5, characterized in that The context of the first prompt word corresponding to each of the sample data is truncated and cleaned to obtain a second prompt word, including: Filter the methods and properties of other categories except the tested function in the first prompt words corresponding to each of the sample data and combine them according to a preset format to obtain an intermediate prompt word; The intermediate prompt word is cleaned according to a preset length to obtain a second prompt word.

7. The method according to claim 6, characterized in that The preset format includes a function under test, a public class set, a constructor, signatures of other methods except the function under test, and properties.

8. The method according to claim 5, characterized in that The target content of the output data is a unit test case of the function under test, and the unit test case includes a case start header.

9. The method according to claim 1, characterized in that: After inputting the tested function of the application into the test case generation model to obtain the corresponding unit test case, the method further includes: The unit test case is repaired based on the extraction rule and / or the completion rule.

10. The method according to claim 9, characterized in that The extraction rules are used to extract and combine methods in the unit test cases based on keywords; The completion rule is used to extract the variable type using a parsing tool and complete the call path according to the variable type.

11. A testing device for an application, characterized in that: include: An acquisition module is used to obtain target training data after data enhancement; An adjustment module is used to adjust the initial model based on model evaluation to obtain a target model; A training module, used to train the target model based on the target training data to obtain a test case generation model; The input module is used to input the tested function of the application program into the test case generation model to obtain the corresponding unit test case, and perform unit test on the tested function of the application program based on the unit test case.

12. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the test method of the application program described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the test method of the application program described in any one of claims 1 to 10.

Images