A Method for Automatically Generating Test Assertions Based on Large Models

Through the automated test assertion generation method based on large models, the problem of incomplete assertion judgment logic and a large amount of manual intervention in software testing is solved, and efficient and accurate automatic test assertion generation and execution is achieved.

CN119883936BActive Publication Date: 2025-05-30SHENZHEN HAIYUNAN NETWORK SECURITY TECH CO LTD

Patent Information

Application Number
CN202510345333.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-03-24
Publication Date
2025-05-30
Estimated Expiration
2045-03-24

AI Technical Summary

Technical Problem

During the software testing process, the existing technology has defects such as insufficient personnel capabilities, improper assertion use, incomplete assertion logic, and the need for a large number of manual intervention to judge assertions, resulting in low testing efficiency and low accuracy.

Method used

Using a large-model-based automated test assertion generation method, we collect historical assertion test data and network text data, use language big models for learning training, generate assertions, and integrate them into the automated testing framework to perform tests.

Benefits of technology

It realizes the generation and execution of automated test assertions, reduces manual intervention, improves testing efficiency and accuracy, and can more comprehensively and accurately verify program execution results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119883936B_ABST
    Figure CN119883936B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of automated test assertion generation. The present invention relates to a method for generating automated test assertions based on a large model. It includes the following steps: S1. Collect historical assertion test data and network text data of different users, and then use a language large model to learn and train the historical assertion test data and network text data; S2. Collect the user's requirement test file, conduct test module planning and analysis on the requirement test file, so that the requirement test file is generated into multiple test modules; Through in-depth analysis of the requirement test file, the present invention divides multiple test modules, and analyzes the key values and associated values of each module, which can comprehensively cover various functions and scenarios of the software. When determining the key test modules, the file complexity and module key values are comprehensively considered, and the modules with the greatest impact on performance are preferentially tested to avoid missing important function points in testing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of automated test assertion generation, and specifically, to a method for generating automated test assertions based on a large model. Background Art

[0002] During the software testing process, programmers set assertions in the code to test whether the program execution results are consistent with the expectations. The expected output values may be one or more. In the functional testing scenario, the results returned or rendered by the interface can be compared with the product requirement description and UI design. If they meet the requirement description and UI design, the test case is determined to be executed successfully. In the interface testing scenario, different request parameters result in different return messages;

[0003] For the assertion method, it can be checked manually or by setting assertions through tools or writing code to judge the return results. An assertion is a means of result judgment, that is, a way to judge whether the result is yes or no. Whether in the R & D position or the testing position, one has to judge the work results. The role of the assertion is to make judgments through machines (tools / codes) as much as possible to avoid problems such as omissions that may be caused by manual inspection;

[0004] However, in the development and testing work, there are defects such as insufficient personnel capabilities leading to improper use of assertions, incomplete assertion judgment logic, and a large amount of manual intervention required to judge assertions. Therefore, a method for generating automated test assertions based on a large model is proposed. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for generating automated test assertions based on a large model to solve the problems raised in the above background art.

[0006] To achieve the above purpose, a method for generating automated test assertions based on a large model is provided, including the following steps:

[0007] S1. Collect historical assertion test data and network text data of different users, and then use a language large model to learn and train the historical assertion test data and network text data;

[0008] S2. Collect the user's requirement test file, conduct a test module planning analysis on the requirement test file to generate the requirement test file into multiple test modules, and at the same time analyze the key values and associated values of the planning module;

[0009] S3. Conduct a test case analysis on each test module, then adjust the test scenario planning by combining all test cases with key values and associated values, and at the same time merge the adjusted test cases according to the associated values;

[0010] S4. Generate assertions for the test cases of each module and the combined test cases based on the large language model, and then integrate the generated assertions with the corresponding test cases into the automated test framework to execute the tests;

[0011] S5. Collect the test results of S4, and at the same time use the test results corresponding to the combined test cases to perform assertion analysis on the test results corresponding to the uncombined test cases, and optimize the assertions according to the analysis results, so as to re-perform automated testing.

[0012] As a further improvement of this technical solution, in S1, by accessing the test management terminal, historical assertion test data is extracted from the test management terminal, and at the same time, historical users are extracted and uniquely identified for each historical user. Then, the historical assertion test data is matched according to the user identification, so as to obtain the historical assertion test data corresponding to each user.

[0013] As a further improvement of this technical solution, in S1, a large language model is established, and then network text data related to assertion testing is collected from the Internet. After that, the collected network text data is combined with the historical assertion test data of all users for learning and training, providing a basis for the subsequent assertion generation step.

[0014] As a further improvement of this technical solution, the steps of S2 are as follows:

[0015] S2.1. Set up a file submission channel in the test management terminal. Users send files to the file submission channel. After receiving the files in the file submission channel, the collected files are used as requirement test files;

[0016] S2.2. Conduct test module planning analysis on the requirement test files. First, perform static analysis and dynamic analysis on the requirement test files, and according to the analysis results, plan the entire requirement test file into multiple test modules;

[0017] S2.3. Combine the requirement test files to perform key value analysis on each test module to obtain the key values of each test module during this test process. At the same time, perform associated value analysis between the test modules to obtain the associated values of each test module compared to other test modules.

[0018] As a further improvement of this technical solution, S2.3 also includes the following steps:

[0019] S2.3.1. Perform key module quantity analysis according to the total quantity of test modules and the complexity of the requirement test files. Then, determine the key test modules by combining the key module quantity determined by the analysis results with the key values of each test module, and select the test module with the highest key value ranking as the key test module;

[0020] S2.3.2. When performing associated numerical analysis between test modules, obtain the file distance between each pair of test modules, and at the same time set the weights of the file distance and module parameters according to the complexity of the demand test file. Then, perform associated numerical calculation between the test modules based on the set weights in combination with the file distance and module parameters.

[0021] As a further improvement of this technical solution, the steps of S3 are as follows:

[0022] S3.1. Conduct a new user test on the user. If the user is a new user, collect demand information from the user. On the contrary, if the user is not a new user, extract historical demand information from the historical assertion test data, and then generate demand information by combining the historical demand information with the current demand test file.

[0023] S3.2. Analyze the test cases by combining the test modules with the demand information, generate test cases corresponding to each test module according to the analysis results, and then conduct a combined analysis with other test modules centered on the key test module to obtain the key test module corresponding to each test module.

[0024] S3.3. Adjust the test scenario planning for the associated test modules according to the corresponding key test module, and at the same time merge the test cases of the key test module with the test cases of the associated test modules to generate a test scenario representing the module combination.

[0025] As a further improvement of this technical solution, during the process of generating combined test cases in S3.3, the test cases corresponding to each test module are not deleted, so that two types of test cases are retained simultaneously, namely the test cases corresponding to a single test module and the test cases corresponding to the combination of multiple test modules.

[0026] As a further improvement of this technical solution, the steps of S4 are as follows:

[0027] S4.1. Automatically generate assertions according to the test dimensions of each test case through a language large model.

[0028] S4.2. Establish an automated test framework belonging to the pipeline, and then input the generated assertions and the corresponding test cases into the automated test framework. The automated test framework automatically starts the test process after receiving the data.

[0029] As a further improvement of this technical solution, the steps of S5 are as follows:

[0030] S5.1. Monitor the test results of the automated test framework. When the test results indicate that the test execution fails, mark the corresponding assertion as a failed assertion, and then perform an optimization analysis on the failed assertion in combination with the assertions that have passed the test execution. Conversely, if the test results indicate that the test execution is successful, proceed to step S5.2;

[0031] S5.2. Set the assertion accuracy threshold, and perform an accuracy value analysis on the assertions that have passed the single-module execution in combination with the assertions merged from multiple modules. At the same time, perform an optimization comparison by combining the accuracy value with the assertion accuracy threshold. When the accuracy value of the assertion that has passed the single-module execution and the assertion merged from multiple modules is less than the assertion accuracy threshold, optimize the assertion that has passed the single-module execution. Conversely, when the accuracy value of the assertion that has passed the single-module execution and the assertion merged from multiple modules is greater than the assertion accuracy threshold, complete the test.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0033] 1. In this method for generating automated test assertions based on a large model, through in-depth analysis of the requirements test file, multiple test modules are divided, and key numerical values and associated numerical values are analyzed for each module. This can comprehensively cover various functions and scenarios of the software. When determining the key test modules, the file complexity and module key numerical values are comprehensively considered, and the modules with the greatest impact on performance are preferentially tested to avoid missing important function points. The assertions generated by the language large model are based on a large amount of programming knowledge and test experience, covering various assertion methods such as response code and keyword field value verification, and can more comprehensively and accurately verify the program execution results.

[0034] 2. In this method for generating automated test assertions based on a large model, through optimization analysis of the test results and assertions, the test strategy can be dynamically adjusted. When the accuracy value of the assertion that has passed the single-module execution and the assertion merged from multiple modules is less than the assertion accuracy threshold, the assertion that has passed the single-module execution will be optimized, continuously improving the test cases and assertions to make the test more in line with the actual requirements of the software and adapt to the continuous update and change of the software.

[0035] 3. In this method for generating automated test assertions based on a large model, starting from the file submitted by the user, a series of operations such as file collection, test module planning, key numerical value analysis, and test case generation are automatically completed. There is no need for manual division of test modules, and it can be automatically planned through static and dynamic analysis of the requirements test file, saving a large amount of manpower and time costs. The language large model can also automatically generate assertions according to the test dimensions of the test cases, eliminating the need for testers to write them manually, greatly shortening the test preparation time and making the test process more efficient and fast. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is the overall flow block diagram of the present invention;

[0037] Figure 2 A flowchart for obtaining the correlation values of each test module compared to other test modules in the present invention;

[0038] Figure 3 A flowchart for obtaining the key test module corresponding to each test module in the present invention;

[0039] Figure 4 A flowchart for automatically starting the test process in the present invention;

[0040] Figure 5 A flowchart for monitoring the test results of the automated test framework in the present invention. Detailed implementation manners

[0041] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0042] Please refer to Figure 1 - Figure 5 As shown, the purpose of this embodiment is to provide a method for generating automated test assertions based on a large model, including the following steps:

[0043] S1. Collect historical assertion test data and network text data of different users, and then use a language large model to learn and train the historical assertion test data and network text data;

[0044] In S1, by accessing the test management terminal, historical assertion test data is extracted from the test management terminal, and at the same time, historical users are extracted and uniquely identified. Then, the historical assertion test data is matched according to the user identification, so as to obtain the historical assertion test data corresponding to each user;

[0045] Extract all historical assertion test data and the information of historical users from the test management terminal. Each user has a unique identifier (such as user ID), and each test data will be associated with the corresponding user identification;

[0046] According to the identification of each user (such as user ID), the historical assertion test data is matched according to the user, so as to obtain the relevant historical test data for each user. This step ensures the accurate association between the test data and the corresponding user behavior.

[0047] S1 establishes a large language model, then collects network text data related to assertion testing on the Internet, and then combines the collected network text data with the historical assertion test data of all users for learning and training to provide a basis for subsequent assertion generation steps;

[0048] Collect network text data: Collect text data related to assertion testing through the Internet. This data can include test documents, forum posts, technical articles, etc. All of these text data contain relevant discussions or actual application scenarios of assertion testing;

[0049] Combine user historical assertion test data: Combine the collected network text data with the user's historical assertion test data. These historical data contain the assertion test records executed by different users in the past, reflecting the test patterns and behaviors of users;

[0050] Train the language model: Input the combined data into the large language model for training, enabling it to learn relevant patterns and rules of assertion testing from it, and ultimately providing a training basis for subsequent assertion generation.

[0051] S2. Collect the user's requirement test file, conduct test module planning and analysis on the requirement test file, so that the requirement test file is generated into multiple test modules, and at the same time conduct critical value and associated value analysis on the planned modules;

[0052] The steps of S2 are as follows:

[0053] S2.1. Set up a file submission channel on the test management side. Users send files through the file submission channel. After receiving the files in the file supply channel, the collected files are used as requirement test files;

[0054] By setting up a file submission channel, the files uploaded by users will be collected into the system and used as requirement test files for subsequent analysis and testing.

[0055] S2.2. Conduct test module planning and analysis on the requirement test file. First, conduct static analysis and dynamic analysis on the requirement test file, and plan the entire requirement test file into multiple test modules according to the analysis results;

[0056] Conduct static analysis (check file structure, document format, data consistency, etc.) and dynamic analysis (simulate the behavior of the file during actual operation, test and performance) on the requirement test file. According to the results of static and dynamic analysis, divide the entire requirement test file into multiple independent test modules, and each test module represents an independent test unit in the file.

[0057] S2.3. Combine the requirement test file to perform key value analysis on each test module, obtain the key values of each test module during this test process, and at the same time perform associated value analysis between test modules to obtain the associated values of each test module compared with other test modules.

[0058] The said S2.3 also includes the following steps:

[0059] S2.3.1. Perform key module quantity analysis according to the total quantity of test modules and the complexity of the requirement test file. Then, determine the key test modules by combining the key module quantity determined by the analysis result with the key values of each test module. Select the test module with the highest sorted key value as the key test module. The specific steps are as follows:

[0060] Key module quantity analysis: First, perform key module quantity analysis according to the complexity of the requirement test file and the total quantity of test modules. Files with higher complexity will have more key modules, while files with lower complexity may focus on a few key modules.

[0061] Determine the key test modules: Based on the quantity of key modules, combine the key values (such as performance indicators) of each test module, sort the test modules, and select the module with the highest key value in the sorting as the key test module. This can give priority to testing the modules that have the greatest impact on the system performance. The specific formula is as follows:

[0062] ;

[0063] Among them, Q is the quantity of determined key modules, f is a function, f adjusts the quantity of key modules according to the complexity. If the complexity is high, the quantity of key modules will increase; if the complexity is low, the quantity of key modules will decrease. C is the complexity of the requirement test file, and X is the total quantity of test modules.

[0064] S2.3.2. When performing associated value analysis between test modules, obtain the file distance between each test module. At the same time, set the weights of the file distance and module parameters according to the complexity of the requirement test file. Then, perform associated value calculation between test modules according to the set weights, combined with the file distance and module parameters. The specific steps are as follows:

[0065] Associated value analysis: When performing associated value analysis between modules, first calculate the file distance between each test module. The file distance can reflect the relationship between modules and the impact on the system. The file distance includes functional correlation distance, data interaction distance, code dependency distance, and business process distance, which are selected according to the type of requirement test file.

[0066] Set weights: According to the complexity of the test files as required, set different weights for file distance and module parameters. For files with high complexity, the mutual influence between modules is greater, and the weights need to be adjusted to better reflect this influence;

[0067] Calculate the correlation value: Combine the set weights, file distance, and module parameters to calculate the correlation value between test modules. This step helps to further determine which modules have a closer relationship, so as to optimize the allocation of test resources. The specific formula is as follows:

[0068] ;

[0069] Among them, A(M, N) is the correlation value, w d is the weight of the file distance, used to adjust the influence of the file distance on the correlation value, w s is the weight of the module parameter, used to adjust the influence of the specific characteristics between modules on the correlation value, D(M, N) is the file distance between module M and module N, reflecting the dependency relationship or influence degree between modules, and P(M, N) is the parameter of module M and module N;

[0070] S3. Analyze the test cases for each test module, then combine all the test cases with the key values and correlation values to adjust the test scenario plan, and at the same time merge the adjusted test cases according to the correlation value;

[0071] The steps of S3 are as follows:

[0072] S3.1. Conduct new user tests on users. If the user is a new user, collect requirement information from the user. On the contrary, if the user is not a new user, extract historical requirement information from the historical assertion test data, and then generate requirement information by combining the historical requirement information with the current requirement test file;

[0073] S3.2. Analyze the test cases by combining the test modules with the requirement information, generate the test cases corresponding to each test module according to the analysis results, and then conduct combined analysis with other test modules centered on the key test module to obtain the key test module corresponding to each test module;

[0074] First, conduct a detailed analysis of the test modules in combination with the requirement information. The result of the analysis is to generate a set of targeted test cases for each test module. These test cases are designed based on the core content of the requirement information to ensure that the requirements of each test module can be verified. After generating the test cases, the key test modules will become the core of the subsequent analysis. By combining and analyzing the key test modules with other modules, determine the core functional requirements of each module, and clarify which modules are the most critical for the overall test through this analysis. Through this combined analysis, the key test modules of each test module can be identified, further improving the test efficiency;

[0075] S3.3. Adjust the test scenario planning for the associated test modules according to the corresponding key test modules, and at the same time combine the test cases of the key test modules with the test cases of the associated test modules for scenario merging, so as to generate a test scenario representing the module merging. The steps are as follows:

[0076] Analyze the characteristics of the key test modules: For each key test module, analyze its functional characteristics, execution logic, boundary conditions, etc. For example, if it is an order payment module, its characteristics may include supported payment methods, payment amount range, payment timeout time, etc.;

[0077] Adjust the test scenario planning of the associated test modules: According to the characteristics of the key test modules, adjust the test scenario planning of the associated test modules. For example, if a new payment method is added to the key test module, the test scenario of the associated test module (such as the order generation module) needs to add the test scenario of order generation under the corresponding payment method;

[0078] Analyze the association between test cases: Analyze the logical association between test cases, such as data dependency, execution order, etc. For example, the test cases of the order payment module may depend on the execution results of the test cases of the order generation module;

[0079] Merge test cases to generate a test scenario: Merge the test case sets of the key test modules and the associated test modules to generate a test scenario representing the module merging. During the merging process, the execution order and data transfer of the test cases need to be considered,

[0080] During the process of generating the merged test cases in S3.3, the test cases corresponding to each test module are not deleted, so that two types of test cases are retained at the same time, namely the test cases corresponding to a single test module and the test cases corresponding to the merging of multiple test modules.

[0081] S4. Generate assertions for the test cases of each module and the combined test cases based on the large language model, and then integrate the generated assertions with the corresponding test cases into the automated test framework to execute the tests;

[0082] The steps of S4 are as follows:

[0083] S4.1. Automatically generate assertions according to the test dimensions of each test case through the large language model;

[0084] S4.2. Establish an automated test framework belonging to the pipeline, and then input the generated assertions and the corresponding test cases into the automated test framework. After receiving the data, the automated test framework automatically starts the test process. The steps are as follows:

[0085] Determine the test dimensions: For each test case, clarify its test dimensions, such as functional correctness, performance metrics (response time, throughput, etc.), boundary conditions, exception handling, etc. Suppose a test case is to test the user login function, and its test dimensions may include the verification of the correctness of the username and password, whether the login response time is within the specified range, etc.;

[0086] The large language model generates assertions: Input the test dimension information of each test case into the large language model. Based on the programming knowledge, test experience, and natural language understanding ability it has learned, the large language model generates corresponding assertions for each test dimension;

[0087] Establish an automated test framework: Design an automated test framework that includes functional modules such as test case management, assertion execution, test result recording, and report generation. The test case management module is responsible for storing and managing the input test cases; the assertion execution module is used to execute the received assertions; the test result recording module records the execution results (pass or fail) of each test case; the report generation module generates a test report based on the recorded results;

[0088] Select a programming language and a test framework tool (such as Pytest for Python, JUnit for Java, etc.) to build the framework. During the building process, implement the interaction and collaboration between each functional module to ensure that the framework can run smoothly;

[0089] Execute the automated test: Input the generated assertions and the corresponding test cases into the automated test framework. After the automated test framework receives the data, the test case management module passes the test cases and assertions to the assertion execution module, and the assertion execution module executes the assertions of each test case in a certain order (such as the number order of the test cases) in turn;

[0090] Result Recording and Report Generation: The test result recording module records the execution results of each assertion. If all assertions of a test case pass, then the test case passes; otherwise, the test case fails. The report generation module generates a test report based on the recorded results. The report content includes the execution status of each test case, the failed assertions and reasons, etc.

[0091] S5. Collect the test results of S4, and at the same time, use the test results corresponding to the merged test cases to perform assertion analysis on the test results corresponding to the unmerged test cases. Optimize the assertions according to the analysis results, so as to re-automate the test.

[0092] The steps of S5 are as follows:

[0093] S5.1. Monitor the test results of the automated test framework. When the test results indicate that the test execution fails, mark the corresponding assertion as a failed assertion, and then perform optimization analysis on the failed assertion in combination with the assertions that have passed the test execution. Conversely, if the test results indicate that the test execution is successful, proceed to step S5.2;

[0094] S5.2. Set the assertion accuracy threshold, perform accuracy value analysis on the assertions that have passed the single-module execution in combination with the assertions merged by multiple modules, and at the same time perform optimization comparison by combining the accuracy value with the assertion accuracy threshold. When the accuracy value of the assertion that has passed the single-module execution and the assertion merged by multiple modules is less than the assertion accuracy threshold, optimize the assertion that has passed the single-module execution. Conversely, when the accuracy value of the assertion that has passed the single-module execution and the assertion merged by multiple modules is greater than the assertion accuracy threshold, the test is completed.

[0095] Conduct in-depth analysis on the failed assertions to identify the parts that have not passed the test. At the same time, use the assertions that have passed the execution to find potential problem points and improvement directions, which helps to improve the overall test coverage and accuracy. The analysis content includes but is not limited to the rationality of the assertion logic, the assertion coverage range, the matching degree with the test cases, etc. By comparing the characteristics of the successful assertions and the failed assertions, find the possible problems of the failed assertions, such as the assertion conditions are set too loose or too strict, key judgment conditions are omitted, etc., to provide a basis for optimizing the failed assertions.

[0096] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and the descriptions in the specification are only preferred examples of the present invention and are not used to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating automatic test assertions based on a large model, characterized in that: The following steps are involved: S1. Collect historical assertion test data and network text data from different users, and then use the large language model to learn and train the historical assertion test data and network text data; S2. Collect the user's demand test files, perform test module planning analysis on the demand test files, generate the demand test files into multiple test modules, and perform key value and associated value analysis on the test modules; The steps of S2 are as follows: S2.

1. Set up a file submission channel on the test management end. Users send files to the file submission channel. After the file provision channel receives the files, the collected files are used as the required test files. S2.

2. Perform test module planning and analysis on the demand test file. First, perform static analysis and dynamic analysis on the demand test file, and plan the entire demand test file into multiple test modules according to the analysis results. S2.

3. Perform key value analysis on each test module in combination with the required test file to obtain the key value of each test module during the test. At the same time, perform correlation value analysis between the test modules to obtain the correlation value of each test module compared with other test modules. S3. Analyze the test cases for each test module, then adjust the test scenario planning for all test cases in combination with key values ​​and associated values, and merge the adjusted test cases according to the associated values; S4. Generate assertions for the test cases of each module and the merged test cases based on the large language model, and then integrate the generated assertions with the corresponding test cases into the automated testing framework to perform testing; S5. Collect the test results of S4, and use the test results corresponding to the merged test cases to perform assertion analysis on the test results corresponding to the unmerged test cases, optimize the assertions and complete the test according to the analysis results; The steps of S5 are as follows: S5.

1. Monitor the test results of the automated test framework. When the test results indicate that the test execution failed, mark the corresponding assertion as a failed assertion. Then, optimize and analyze the failed assertion in combination with the assertion that the test execution succeeded. Otherwise, go to step S5.

2. S5.

2. Set the assertion accuracy threshold, analyze the accuracy value of the assertion of successful execution of a single module and the assertion of multiple modules, and optimize and compare the accuracy value with the assertion accuracy threshold. When the accuracy value of the assertion of successful execution of a single module and the assertion of multiple modules is less than the assertion accuracy threshold, the assertion of successful execution of the single module is optimized. Conversely, when the accuracy value of the assertion of successful execution of a single module and the assertion of multiple modules is greater than the assertion accuracy threshold, the test is completed.

2. The method for generating assertions for automated testing based on a large model according to claim 1, characterized in that: The S1 accesses the test management terminal, extracts historical assertion test data at the test management terminal, extracts historical users, and identifies the historical users separately, and then matches the historical assertion test data according to the user identification, thereby obtaining the historical assertion test data corresponding to each user.

3. The method for generating assertions for automated testing based on a large model according to claim 1, characterized in that: The S1 builds a large language model and then collects network text data related to assertion testing on the Internet. The collected network text data is then combined with the historical assertion test data of all users for learning and training, providing a basis for subsequent assertion generation steps.

4. The method for generating assertions for automated testing based on a large model according to claim 1, characterized in that: The S2.3 also includes the following steps: S2.3.

1. Analyze the number of key modules based on the total number of test modules and the complexity of the required test files. Then determine the key test modules based on the number of key modules determined by the analysis results and the key values ​​of each test module. Select the test module with the highest key value ranking as the key test module. S2.3.

2. When performing correlation numerical analysis between test modules, obtain the file distance between each test module, and set the weights of file distance and module parameters according to the complexity of the required test file. Then, calculate the correlation numerical values ​​between the test modules based on the set weights combined with the file distance and module parameters.

5. The method for generating assertions for automated testing based on a large model according to claim 1, characterized in that: The steps of S3 are as follows: S3.

1. Perform a new user test on the user. If the user is a new user, collect demand information from the user. Otherwise, if the user is not a new user, extract historical demand information from the historical assertion test data, and then combine the historical demand information with the current demand test file to generate demand information. S3.2, analyze the test cases by combining the test modules with the demand information, generate the test cases corresponding to each test module according to the analysis results, and then combine and analyze the key test modules with other test modules to obtain the key test modules corresponding to each test module; S3.

3. According to the corresponding key test modules, the test scenario planning and adjustment are performed on the associated test modules, and at the same time, the test cases of the key test modules are combined with the test cases of the associated test modules to merge the scenarios, thereby generating a test scenario representing the module merger.

6. The method for generating assertions for automated testing based on a large model according to claim 5, characterized in that: In the process of generating the merged test cases, S3.3 does not delete the test cases corresponding to each test module, so that two types of test cases are retained at the same time, namely, the test cases corresponding to a single test module and the test cases corresponding to the merge of multiple test modules.

7. The method for generating assertions for automated testing based on a large model according to claim 1, characterized in that: The steps of S4 are as follows: S4.1, automatically generate assertions based on the test dimensions of each test case through a large language model; S4.

2. Establish an automated testing framework, and then input the generated assertions into the automated testing framework in combination with the corresponding test cases. The automated testing framework automatically starts the testing process after receiving the data.

Citation Information

Patent Citations

  • Data-driven test method, device and equipment and computer readable storage medium

    CN112667512A

  • Unit test case assertion generation method based on large language model and retrieval enhancement

    CN117951011A

Cited By

  • Automatic evaluation method for server security computing environment based on large model

    CN121210260A