POSIX interface test method and test system

Abstract

The invention discloses a POSIX interface testing method and system, and belongs to the technical field of interface testing, and the method comprises the steps: building a testing environment which comprises a first model, and inputting a testing case into the first model; performing fine tuning training on the first model, and setting an input condition and annotation data; performing multi-round test verification on the first model after fine tuning, performing performance evaluation, recording the performance of the first model on different test sets, and performing parameter adjustment on the first model based on a performance evaluation result; and creating an interface test environment, deploying the fine-tuned model in the interface test environment, performing interface test, recording test parameters of the interface, and generating a test report. In the implementation process of the technical scheme, the first model is subjected to fine tuning training, so that the first model can accurately understand and process the test case, codes and annotations meeting requirements are generated, the interface test process is simplified, and the test efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of interface testing, and in particular to a POSIX interface testing method and testing system. Background Art

[0002] The POSIX interface standard is an interface standard that provides a unified interface specification for various operating systems to ensure the portability and compatibility of software between different systems. It is of great significance to promote the standardization of software development and improve the interoperability between systems.

[0003] Before using the above interface standards, interface testing is usually required to verify its compliance and stability, ensure efficient operation in actual applications, and avoid system failures and data loss caused by interface problems. With the development of big models, in recent years, big model code generation technology has been used to utilize pre-trained large language models to automatically generate source code or scripts to complete specific tasks or implement specific functions, thereby improving the readability and maintainability of the code, and combined with interface testing to further improve test efficiency and accuracy.

[0004] However, in the actual application of the above technical solutions, there are still some problems. For example, the design of test cases needs to comprehensively cover various interface functions to ensure the integrity of the test. In addition, some obscure interfaces are used less frequently, resulting in a lack of sufficient reference code examples in the design of test cases, which increases the difficulty of testing. In addition, the existing large model technology gives both design and code through code, but the entire running result is chaotic and difficult to meet actual needs. At the same time, the code style and test case design results are difficult to meet the requirements.

[0005] Therefore, it is necessary to provide a POSIX interface testing method and testing system to solve the above problems.

[0006] It should be noted that the above information disclosed in this background technology section is only for understanding the background technology of the present application concept, and therefore, it may contain information that does not constitute the prior art. Summary of the invention

[0007] Based on the above problems existing in the prior art, the problem to be solved by this application is: to provide a POSIX interface testing method and testing system, to fine-tune the existing large model so that it generates code and annotations that meet the requirements, and to improve the efficiency of interface testing.

[0008] The technical solution adopted by the present application to solve the technical problem is: a POSIX interface testing method, including:

[0009] Building a test environment, the test environment including a first model, and inputting test cases into the first model, wherein the number of test cases for a single test case is at least five, and parameters of the same type are prohibited from being added as case supplements;

[0010] Fine-tune the first model and set input conditions and annotated data to ensure that the model can accurately understand and process test cases, and continuously optimize model parameters during the fine-tuning process;

[0011] The fine-tuned first model is verified through multiple rounds of testing and performance evaluation is performed, its performance on different test sets is recorded, and parameters of the first model are adjusted based on the performance evaluation results;

[0012] Create an interface test environment and deploy the fine-tuned model in it to perform interface testing, record the test parameters of the interface, and generate a test report.

[0013] During the implementation of the technical solution of the present application, the first model is fine-tuned and trained to enable it to accurately understand and process test cases, thereby generating code and annotations that meet the requirements, simplifying the interface testing process, and improving testing efficiency.

[0014] Furthermore, the total number of test cases is not less than five thousand, and the number of construction cases of a single test case is at least five and no more than eight.

[0015] Furthermore, fine-tuning the first model further includes the following steps:

[0016] Select a training set, which includes input conditions and labeled data, and preprocess the selected training set;

[0017] Select a base model and set fine-tuning parameters, including learning rate, training rounds, batch size, weight decay, and gradient clipping;

[0018] Input the preprocessed training set into the basic model, train the basic model with task-specific data, and monitor the basic model during the training process;

[0019] Efficiently fine-tune the parameters of the basic model and only update some parameters in the model to reduce the amount of calculation and improve training efficiency.

[0020] Furthermore, the input conditions include code style, input parameter type, value range, and code function, and the annotation data includes code data, comments, and test case design text that meet the programming style requirements.

[0021] Furthermore, the basic model is BERT or GPT-3.

[0022] Furthermore, during the fine-tuning process, a dynamic adjustment strategy is adopted to optimize parameters in real time according to the model performance to ensure the best fine-tuning effect. After the fine-tuning is completed, multiple rounds of testing and verification are carried out to evaluate the performance of the model in different scenarios.

[0023] Furthermore, the process of interface testing includes the following steps:

[0024] Create a test file environment and build a directory through scripts, generate a file directory based on the tested interface, configure interface test parameters, and automatically modify the configuration development environment according to the interface name;

[0025] Design test cases, use prompts to design test cases for each Posix interface, and clarify the input and output requirements of the test cases;

[0026] Call the fine-tuned basic model, verify the interface functions according to the designed test cases, record the results of each call, and analyze data consistency and response time;

[0027] Generate test code and comments, and combine the requirements of code standards and code templates to generate code and comments that meet the requirements.

[0028] Furthermore, the input and output requirements of the test cases include whether the test process is legal, whether the test results meet expectations, and whether exceptions are handled properly.

[0029] A POSIX interface testing system, comprising:

[0030] A test environment building module, used to build a test environment, the test environment includes a first model, and input test cases into the first model, wherein the number of cases for constructing a single test case is at least five, and parameters of the same type are prohibited from being added as case supplements;

[0031] A fine-tuning training module is used to fine-tune the first model and set input conditions and annotated data to ensure that the model can accurately understand and process test cases and continuously optimize model parameters during the fine-tuning process;

[0032] A verification module is used to verify the fine-tuned first model through multiple rounds of tests and perform performance evaluation, record its performance on different test sets, and adjust parameters of the first model based on the performance evaluation results;

[0033] The interface testing module is used to create an interface testing environment, deploy the fine-tuned model in it, perform interface testing, record the test parameters of the interface, and generate a test report.

[0034] The beneficial effects of the present application are as follows: the present application provides a POSIX interface testing method and testing system, which can accurately understand and process test cases by fine-tuning the first model, thereby generating code and annotations that meet the requirements, simplifying the interface testing process, and improving testing efficiency.

[0035] In addition to the above-described purposes, features and advantages, the present application also has other purposes, features and advantages. The present application will be further described in detail with reference to the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0037] In the attached picture:

[0038] Figure 1 This is a schematic diagram of the overall process of a POSIX interface testing method in this application;

[0039] Figure 2 The figure is a schematic diagram of the module structure of a POSIX interface test system in this application. DETAILED DESCRIPTION

[0040] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0041] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0042] Embodiment 1: Figure 1 As shown, the present application provides a POSIX interface testing method, which is used to test the POSIX interface so that it meets the requirements of various operating systems, and realizes code generation through large model technology to improve testing efficiency. The testing method includes the following steps:

[0043] Step 01: Build a test environment, which includes a first model, and input test cases into the first model, wherein the number of test cases for a single test case is at least five, and parameters of the same type are prohibited from being added as case supplements;

[0044] In order to use the big model technology to test the interface, it is necessary to first build a test environment. In this embodiment, the test environment includes a first model, and the first model has an input end and an output end. The first model can use the RealEvo_IDE6.5.0 integrated development platform. The input end of the first model is used to input test cases. In this embodiment, the total number of test cases is not less than 5,000, and the number of construction cases of a single test case is at least five and no more than eight, so as to ensure the diversity and comprehensiveness of the test cases and avoid duplication and redundancy.

[0045] When constructing test cases, you need to build multiple test cases according to the parameter changes of the test interface, and you need to include parameters with actual meanings when constructing them. You cannot use parameters of the same type as test case supplements. For example, malloc1024MB and malloc1048MB cannot be used as test case supplements to prevent the distortion of test results caused by the single parameter type. In addition, you can perform boundary tests on digital parameters to ensure that the test covers all kinds of extreme cases.

[0046] Step 02: Fine-tune the first model and set input conditions and labeled data to ensure that the model can accurately understand and process test cases, and continuously optimize model parameters during the fine-tuning process;

[0047] In the existing big model technology, although the big model can be used to output the design and code, the whole operation process is relatively complicated, and the output results are chaotic and difficult to call directly. At the same time, the code style and the design results of the test case are difficult to meet the requirements. Therefore, it is necessary to fine-tune the big model and optimize the model parameters to obtain the generation function of the special style code and the test case design in a special format;

[0048] Specifically, fine-tuning the first model further includes the following steps:

[0049] Step 201: Select a training set, which includes input conditions and labeled data, and preprocess the selected training set;

[0050] When fine-tuning the first model, it is first necessary to select a suitable training set. In this embodiment, the training set includes input conditions and annotated data, wherein the input conditions include code style, input parameter type, value range, code function, etc., and the annotated data includes code data, comments, test case design text, etc. that meet the programming style requirements. After the selection is completed, the selected training set is preprocessed, wherein the preprocessing process includes steps such as data cleaning, format unification, and outlier processing. For details, reference can be made to the method for preprocessing model input data in the prior art, which will not be described in detail in this embodiment.

[0051] Step 202: Select a base model and set fine-tuning parameters, including learning rate, training rounds, batch size, weight decay, and gradient clipping;

[0052] When performing fine-tuning training, it is necessary to select a basic model. In this embodiment, the basic model can be BERT, GPT-3, etc., as the basis for fine-tuning training, and set fine-tuning parameters, including learning rate, training rounds, batch size, weight decay, gradient clipping, etc. The settings of these parameters need to be adjusted according to actual needs to ensure that the model can converge efficiently during the fine-tuning process and avoid overfitting or underfitting;

[0053] Step 203: input the preprocessed training set into the basic model, and use the data of the specific task to train the basic model, and monitor the basic model during the training process;

[0054] Evaluate model performance by monitoring and adjusting the model status in real time based on the model's performance on specific tasks, ensuring that the model has high accuracy and stability when processing complex codes and test cases. During the training process, regularly evaluate the quality of model output and record changes in key indicators such as accuracy, recall, F1 score, etc., so as to adjust the training strategy in a timely manner, optimize model parameters, and improve the conformity of generated code and test cases.

[0055] Step 204: Efficiently fine-tune the parameters of the basic model, and only update some parameters in the model to reduce the amount of calculation and improve training efficiency.

[0056] Highly Efficient Fine-tuning (PEFT) is a lightweight fine-tuning method for specific tasks. It selectively updates key layer parameters to achieve fast and low-cost transfer learning, reduce computing resource consumption while maintaining model performance, and ensure high-quality output of generated code and test cases. During the fine-tuning process, dynamic adjustment strategies can also be used to optimize parameters in real time according to model performance to ensure the best fine-tuning effect. After fine-tuning is completed, multiple rounds of testing and verification are carried out to evaluate the performance of the model in different scenarios to ensure its generalization ability and robustness, and ultimately achieve efficient and accurate code generation and test case design.

[0057] Step 03: Verify the fine-tuned first model through multiple rounds of testing and conduct performance evaluation, record its performance on different test sets, and adjust the parameters of the first model based on the performance evaluation results;

[0058] After fine-tuning the first model, multiple rounds of testing and verification are required to ensure its stable performance on different test sets. By recording key indicators such as accuracy and recall rate, optimizing the model structure and various parameter configurations, the model's adaptability in complex scenarios can be further improved to ensure its efficiency and reliability in practical applications.

[0059] Step 04: Create an interface test environment and deploy the fine-tuned model in it to perform interface testing, record the test parameters of the interface, and generate a test report.

[0060] After completing model fine-tuning and multiple rounds of verification, it is necessary to conduct actual interface testing, deploy the fine-tuned model to the real environment, record various test parameters of the interface during the test, and generate a test report. Specifically, the above process includes the following steps:

[0061] Create a test file environment and build a directory through scripts, generate a file directory based on the tested interface, configure interface test parameters, and automatically modify the configuration development environment according to the interface name;

[0062] In the actual test process of the interface, it is necessary to use a script to create a test file environment, and automatically create file directories, copy files, modify project names, file names, etc. according to the POSIX interface, and then automatically modify the RealEvo_IDE configuration development environment according to the POSIX interface name, automatically type the Posix interface name when opening a text file, set the interface test parameters, execute the interface test script, and record the test results;

[0063] Design test cases, use prompts to design test cases for each Posix interface, and clarify the input and output requirements of the test cases;

[0064] During the testing process, design test cases, use prompts to design test cases for each Posix interface, and clarify the input and output requirements of the test cases, such as whether the test process is legal, whether the test results meet expectations, and whether exceptions are handled properly, to ensure that each test case is fully covered and verify the integrity and stability of the interface function;

[0065] Call the fine-tuned basic model, verify the interface functions according to the designed test cases, record the results of each call, and analyze data consistency and response time;

[0066] By calling the fine-tuned basic model and combining it with preset test cases, we can verify the interface functions item by item, record the results of each call in detail, analyze data consistency and response time, ensure the efficiency and stability of the interface in actual applications, and provide a reliable basis for subsequent optimization.

[0067] Generate test code and comments, and combine the requirements of code specifications and code templates to generate code and comments that meet the requirements;

[0068] After generating the test code and comments, you also need to review and optimize the code according to the code specifications and template requirements, and confirm the results through manual inspection to ensure code quality and readability. When generating test code and comments, you also need to give prompt words that fail the test so that you can quickly locate the problem when the test fails, further optimize the model and interface, and improve the overall performance of the system and user experience.

[0069] Embodiment 2: Figure 2 As shown, the present application proposes a POSIX interface testing system, which runs the POSIX interface testing method in the first embodiment, and generates codes and comments that meet the requirements by fine-tuning the first model. Specifically, the testing system includes:

[0070] A test environment building module, used to build a test environment, the test environment includes a first model, and input test cases into the first model, wherein the number of cases for constructing a single test case is at least five, and parameters of the same type are prohibited from being added as case supplements;

[0071] A fine-tuning training module is used to fine-tune the first model and set input conditions and annotated data to ensure that the model can accurately understand and process test cases and continuously optimize model parameters during the fine-tuning process;

[0072] A verification module is used to verify the fine-tuned first model through multiple rounds of tests and perform performance evaluation, record its performance on different test sets, and adjust parameters of the first model based on the performance evaluation results;

[0073] The interface testing module is used to create an interface testing environment, deploy the fine-tuned model in it, perform interface testing, record the test parameters of the interface, and generate a test report.

[0074] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A POSIX interface testing method, characterized in that: include: Building a test environment, the test environment including a first model, and inputting test cases into the first model, wherein the number of test cases for a single test case is at least five, and parameters of the same type are prohibited from being added as case supplements; Fine-tune the first model and set input conditions and annotated data to ensure that the model can accurately understand and process test cases, and continuously optimize model parameters during the fine-tuning process; The fine-tuned first model is verified through multiple rounds of testing and performance evaluation is performed, its performance on different test sets is recorded, and parameters of the first model are adjusted based on the performance evaluation results; Create an interface test environment and deploy the fine-tuned model in it to perform interface testing, record the test parameters of the interface, and generate a test report.

2. A POSIX interface testing method according to claim 1, characterized in that: The total number of test cases is not less than five thousand, and the number of construction cases of a single test case is at least five and no more than eight.

3. A POSIX interface testing method according to claim 1, characterized in that: Fine-tuning the first model further includes the following steps: Select a training set, which includes input conditions and labeled data, and preprocess the selected training set; Select a base model and set fine-tuning parameters, including learning rate, training rounds, batch size, weight decay, and gradient clipping; Input the preprocessed training set into the basic model, train the basic model with task-specific data, and monitor the basic model during the training process; Efficiently fine-tune the parameters of the basic model and only update some parameters in the model to reduce the amount of calculation and improve training efficiency.

4. A POSIX interface testing method according to claim 3, characterized in that: The input conditions include code style, input parameter type, value range, and code function, and the annotation data includes code data, comments, and test case design text that meet the programming style requirements.

5. A POSIX interface testing method according to claim 3, characterized in that: The basic model is BERT or GPT-3.

6. A POSIX interface testing method according to claim 3, characterized in that: During the fine-tuning process, a dynamic adjustment strategy is adopted to optimize parameters in real time according to the model performance to ensure the best fine-tuning effect. After the fine-tuning is completed, multiple rounds of testing and verification are carried out to evaluate the performance of the model in different scenarios.

7. A POSIX interface testing method according to claim 1, characterized in that: The process of interface testing includes the following steps: Create a test file environment and build a directory through scripts, generate a file directory based on the tested interface, configure interface test parameters, and automatically modify the configuration development environment according to the interface name; Design test cases, use prompts to design test cases for each Posix interface, and clarify the input and output requirements of the test cases; Call the fine-tuned basic model, verify the interface functions according to the designed test cases, record the results of each call, and analyze data consistency and response time; Generate test code and comments, and combine the requirements of code standards and code templates to generate code and comments that meet the requirements.

8. A POSIX interface testing method according to claim 7, characterized in that: The input and output requirements of test cases include whether the test process is legal, whether the test results meet expectations, and whether exceptions are handled properly.

9. A POSIX interface testing system, characterized in that: include: A test environment building module, used to build a test environment, the test environment includes a first model, and input test cases into the first model, wherein the number of cases for constructing a single test case is at least five, and parameters of the same type are prohibited from being added as case supplements; A fine-tuning training module is used to fine-tune the first model and set input conditions and annotated data to ensure that the model can accurately understand and process test cases and continuously optimize model parameters during the fine-tuning process; A verification module is used to verify the fine-tuned first model through multiple rounds of tests and perform performance evaluation, record its performance on different test sets, and adjust parameters of the first model based on the performance evaluation results; The interface testing module is used to create an interface testing environment, deploy the fine-tuned model in it, perform interface testing, record the test parameters of the interface, and generate a test report.

10. A POSIX interface testing system according to claim 9, characterized in that: Used to implement a POSIX interface testing method as described in any one of claims 1 to 8.

Images