Test case generation method and system

Abstract

The invention discloses a test case generation method and system. The method comprises the steps that S1, code change is analyzed through a code difference analysis module, so that code key information is extracted, and a code difference type is recognized; s2, performing risk assessment on the code difference to obtain a risk assessment result; s3, converting the extracted code key information into natural language description by utilizing a natural language processing model; s4, constructing a structured cue word according to the risk assessment result of the code difference and the natural language description; s5, the constructed structured cue word is input into a pre-trained large language model, test cases are automatically generated, and the generated test cases comprise a function regression risk test case, a performance influence test case and a security vulnerability test case. According to the method, the test case is automatically generated, the workload of manually writing the test case by a tester is greatly reduced through the automatic process, and the test efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of testing technologies, and in particular, to a test case generation method and system. Background Art

[0002] As software systems continue to evolve, code is frequently added, deleted, or changed. These code changes may introduce new security risks, functional anomalies, or performance bottlenecks. The traditional way of manually writing test cases is difficult to meet the rapidly changing requirements and is prone to missing potential problems. There is an urgent need for an automated method to generate test cases. Summary of the Invention

[0003] Embodiments of the present invention provide a test case generation method and system to solve the problems of low efficiency and easy omission of potential problems caused by the traditional manual writing of test cases in the prior art.

[0004] To achieve the above object, on the one hand, the present invention provides a test case generation method, which includes: S1. Analyze the code changes through a code difference analysis module to extract key code information and identify the types of code differences; S2. Conduct a risk assessment on the code differences to obtain a risk assessment result; S3. Use a natural language processing model to convert the extracted key code information into a natural language description; S4. Construct a structured prompt word according to the risk assessment result of the code differences and the natural language description; S5. Input the constructed structured prompt word into a pre-trained large language model to automatically generate test cases, where the generated test cases include: functional regression risk test cases, performance impact test cases, and security vulnerability test cases.

[0005] Optionally, the analyzing the code changes through a code difference analysis module to extract key code information includes: analyzing the code changes through an abstract syntax tree to extract key code information including function signatures, input parameters, return values, control flows, and dependencies.

[0006] Optionally, the natural language description includes code function description, input / output description, control flow description, and dependency description.

[0007] Optionally, the functional regression risk test cases include normal input, boundary value input, and illegal input scenarios; the performance impact test cases include high-concurrency input and large-data-volume input scenarios; the security vulnerability test cases include ultra-long string input and input scenarios containing special characters.

[0008] Optionally, the types of code differences include: added code, deleted code, and modified code. For added code, deleted code, or modified code, functional regression risk test cases can be generated; for added code, deleted code, or modified code, performance impact test cases can be generated; for added code or modified code, security vulnerability test cases can be generated.

[0009] On the other hand, the present invention provides a test case generation system, which includes: a parsing unit for parsing code changes through a code difference analysis module to extract key code information and identify the types of code differences; a risk assessment unit for performing a risk assessment on the code differences to obtain a risk assessment result; a conversion unit for using a natural language processing model to convert the extracted key code information into a natural language description; a construction unit for constructing a structured prompt based on the risk assessment result of the code differences and the natural language description; and a test case generation unit for inputting the constructed structured prompt into a pre-trained large language model to automatically generate test cases, where the generated test cases include: functional regression risk test cases, performance impact test cases, and security vulnerability test cases.

[0010] Optionally, the parsing of code changes through a code difference analysis module to extract key code information includes: parsing code changes through an abstract syntax tree to extract key code information including function signatures, input parameters, return values, control flows, and dependencies.

[0011] Optionally, the natural language description includes code function descriptions, input / output descriptions, control flow descriptions, and dependency descriptions.

[0012] Optionally, the functional regression risk test cases include normal input, boundary value input, and illegal input scenarios; the performance impact test cases include high-concurrency input and large-data-volume input scenarios; the security vulnerability test cases include ultra-long string input and input scenarios containing special characters.

[0013] Optionally, the types of code differences include: added code, deleted code, and modified code. For added code, deleted code, or modified code, functional regression risk test cases can be generated; for added code, deleted code, or modified code, performance impact test cases can be generated; for added code or modified code, security vulnerability test cases can be generated.

[0014] Advantages of the present invention:

[0015] The present invention provides a test case generation method and system. Among them, this method conducts a risk assessment of code differences, extracts key code information using technologies such as Abstract Syntax Trees (ASTs), and converts the code information into clear natural language descriptions with the help of natural language processing models. Furthermore, it constructs structured prompting words to guide a pre-trained large language model to automatically generate targeted test cases, so as to achieve the comprehensiveness and efficiency of test coverage. This automated process significantly reduces the workload of testers in manually writing test cases and improves test efficiency. Description of the Drawings

[0016] Figure 1 is a flowchart of a test case generation method provided by an embodiment of the present invention;

[0017] Figure 2 is a schematic structural diagram of a test case generation system provided by an embodiment of the present invention. Detailed Embodiments

[0018] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0019] Figure 1 is a test case generation method provided by an embodiment of the present invention. As Figure 1 shown, this method includes:

[0020] S1. Analyze the code changes through a code difference analysis module to extract key code information and identify the types of code differences;

[0021] In an optional embodiment, use a code difference analysis module (Abstract Syntax Tree AST) to analyze the code changes to extract key code information including function signatures, input parameters, return values, control flows, and dependency relationships.

[0022] Function signature: function name, parameter types, return value types.

[0023] Input parameters: include the data types, default values, and constraint conditions of the parameters.

[0024] Return value: the type and format of the return result.

[0025] Control flow: conditional judgments, loop structures, exception handling logics, etc.

[0026] Dependency relationship: function call chain, dependency situations between modules.

[0027] The types of code differences include: added code, deleted code, and modified code.

[0028] S2. Conduct a risk assessment on the code differences to obtain a risk assessment result;

[0029] In an optional implementation, the risk assessment includes: risk type assessment and risk level assessment;

[0030] The risk types include: functional regression risk, performance testing, and security vulnerability;

[0031] The risk levels include: high risk, medium risk, and low risk.

[0032] The following takes added code as an example for illustration:

[0033] Functional regression risk

[0034] Basis for assessment: Added code may change the original input processing flow. If the input validation logic is misjudged, it may cause normal input to be misjudged as illegal, or vice versa, illegal input is not correctly intercepted, thus leading to functional anomalies.

[0035] Risk description: Since this logic directly affects the user login function, it belongs to the functional regression risk.

[0036] Performance impact

[0037] Basis for assessment: If there are complex conditional judgments or multi-layer nested loops in the added code, in the case of high concurrency or large data volume input, it may increase the system processing delay and affect the overall performance.

[0038] Risk description: Although the main purpose is input validation, complex logic may introduce additional performance overhead, so the performance impact risk also needs to be concerned.

[0039] Security vulnerability

[0040] Basis for assessment: The added code involves special characters and SQL injection protection. If the logic implementation is imperfect, some malicious inputs may be missed, resulting in security vulnerabilities.

[0041] Risk description: The purpose of the newly added logic here is to prevent security attacks, so its security protection ability is crucial and is classified as a security vulnerability risk.

[0042] For the above risk types, the system evaluates the risk level according to the preset risk assessment rules;

[0043] The preset risk assessment rules are:

[0044] Low risk: The changes only affect local code and do not involve core logic.

[0045] Medium risk: The changes affect multiple modules but do not involve the critical path.

[0046] High risk: The changes affect core logic or the critical path, which may lead to system crashes or data loss.

[0047] S3. Use a natural language processing model to convert the extracted key code information into a natural language description;

[0048] In an optional implementation, the natural language description includes code function description, input and output description, control flow description, and dependency description.

[0049] Use a pre-trained natural language processing (NLP) model (such as GPT series or BERT based on the Transformer architecture and fine-tuned for a specific domain) to convert the extracted key code information into an easy-to-understand natural language description. This conversion process includes the following steps:

[0050] 1. Input formatting

[0051] Format the extracted key code information according to a predefined template. For example, for a function that adds input validation logic, the following structured input can be constructed:

[0052] {

[0053] "Function name": "validateInput",

[0054] "Function": "Verify the legality of user input",

[0055] "Input parameters":

[0056] {"Name": "inputString", "Type": "String", "Constraint": "Length does not exceed 100 characters, no special characters"}

[0057] ,

[0058] "Return value": {"Type": "Boolean", "Description": "Returns true for legal, false for illegal"},

[0059] "Control flow": "Use if-else judgment logic",

[0060] "Dependency": "Called by the user authentication module",

[0061] }

[0062] 2. Pre-define structured templates

[0063] Pre-define structured templates to guide a natural language processing (NLP) model to generate high-quality natural language descriptions. For example:

[0064] Code function description template:

[0065] "The main function of the function [function name] is [function description]. This function is used for [description of specific application scenario]."

[0066] Input-output description template:

[0067] "This function receives [description of input parameters], where [input parameter name] is of [data type], and the requirement is [specific constraint]; the return value is of [return value type], and its meaning is [description of return value]."

[0068] Control flow description template:

[0069] "The internal control flow of this function is [description of control flow structure]. This control flow first [description of initial condition or judgment], then [description of main branch or loop processing], and finally [description of end processing or exception handling]."

[0070] Dependency relationship description template:

[0071] "This function depends on [description of dependency relationship]. Specifically, this function is called by [description of calling module or component], and depends on [description of external resource or interface] to achieve [specific application scenario or purpose]."

[0072] 3. The NLP model generates natural language descriptions

[0073] Combine the formatted key information of the code with the pre-defined structured templates to construct a complete input prompt, and pass the constructed input prompt to a pre-trained (and optionally fine-tuned) NLP model. The NLP model automatically generates a smooth and coherent natural language description based on the content in the input prompt and the pre-learned language patterns and context information.

[0074] The generated natural language description may be as follows:

[0075] "The main function of the function validateInput is to verify the legality of user input. This function is used in the user authentication module to ensure that the input data submitted by the user meets the predetermined requirements. This function receives a string parameter named inputString, whose type is String, and requires that the length of this string does not exceed 100 characters and does not contain special characters; the return value is Boolean, which means that it returns true when the input is legal and false when the input is illegal. The function internally uses an if-else judgment structure. This control flow first checks whether inputString is empty or exceeds the length limit; then determines whether it contains any illegal special characters; finally, according to the judgment results, it returns true when all conditions are met, otherwise it returns false, and at the same time records errors through an exception capture mechanism in case of abnormal situations and returns the corresponding error status. This function depends on the call of the user authentication module and depends on the shared data interface to ensure that the input verification results can be correctly transmitted and used throughout the user authentication process."

[0076] Using the NLP model to convert the key information of the code into natural language descriptions realizes the automated process from code parsing to description generation, reducing the need for manual intervention; the generated natural language descriptions not only cover the code functions and input / output information, but also combine the dependency relationships to form a complete context description, which helps to generate structured prompt words and automated test cases subsequently.

[0077] S4. Construct structured prompt words based on the risk assessment results of code differences and natural language descriptions;

[0078] Integrate the natural language descriptions generated in S3 with the risk assessment results obtained in S2 to construct a structured prompt word (Prompt). This prompt word not only includes the code function description, input / output description, and description of dependency relationships, but also clearly indicates the risk assessment results and test objectives, thereby guiding the pre-trained large language model to automatically generate detailed test cases.

[0079] In an optional implementation manner, the steps of constructing the structured prompt word are as follows:

[0080] 1. Integrate description information

[0081] Merge the natural language descriptions of S3 with the risk assessment results of S2. Assume the risk assessment results of S2 are as follows:

[0082] Function regression risk: medium risk (Hint: may misjudge normal input)

[0083] Performance risk: low risk

[0084] Security Vulnerability Risk: High Risk (Hint: Protect against injection of extremely long strings and special characters)

[0085] 2. Build a structured prompt template

[0086] The following template can be designed:

[0087]

Code Description

[0088] Function Name: validateInput

[0089] Description: The function validateInput is mainly used to verify the legality of user input. It is called by the user authentication module to ensure that the input data meets the predetermined standards. This function receives the parameter inputString (of type String, with a required length <= 100 and no special characters), and returns a boolean value. true indicates that the input is legal, and false indicates that it is illegal. Internally, an if-else judgment logic is used for verification. This control flow first checks whether inputString is empty or exceeds the length limit; then it determines whether it contains any illegal special characters; finally, according to the judgment results, it returns true when all conditions are met, otherwise it returns false. At the same time, when an exception occurs, the error is recorded through an exception capture mechanism and the corresponding error status is returned. This function depends on the call of the user authentication module and relies on a shared data interface to ensure that the input verification results can be correctly transmitted and used throughout the user authentication process.

[0090]

Risk Assessment

[0091] Risk Type:

[0092] Functional Regression Risk: Medium Risk (May misjudge normal input)

[0093] Performance Impact: Low Risk

[0094] Security Vulnerability: High Risk (Protect against injection of extremely long strings and special characters)

[0095] Risk Hint: The new logic has potential security hazards, and malicious input must be tested intensively.

[0096]

Test Objectives

[0097] Please generate detailed test cases based on the above information, requirements:

[0098] (1) Functional Regression Test: The test cases need to cover normal input, boundary value input, and illegal input situations.

[0099] (2) Performance Impact Testing: Test cases shall cover scenarios such as high-concurrency input, large-volume data input, complex input, and multiple invocations to evaluate system response time and resource utilization.

[0100] (3) Security Vulnerability Testing: For the newly added input validation logic, test cases shall verify the system's protection capabilities by passing in extremely long strings and inputs containing special characters.

[0101] Please describe in detail the test steps, expected results, and exception handling.

[0102] 3. Construct Prompt

[0103] Integrate the various parts of the above template into a complete structured prompt, ensuring that the model can clearly understand the requirements and generate test case descriptions covering multiple test objectives.

[0104] S5. Input the constructed structured prompt into the pre-trained large language model to automatically generate test cases, including functional regression risk test cases, performance impact test cases, and security vulnerability test cases.

[0105] In an optional implementation, the functional regression risk test cases include scenarios of normal input, boundary value input, and illegal input;

[0106] The performance impact test cases include scenarios of high-concurrency input and large-volume data input;

[0107] The security vulnerability test cases include scenarios of extremely long string input and input containing special characters.

[0108] The specific process includes the following steps:

[0109] 1. Input Preparation

[0110] Input the structured prompt constructed in the previous step as the complete text into the pre-trained large language model (such as GPT series models). Since the structured prompt clearly contains code descriptions, risk assessment information, and specific test objectives, the large model can fully understand the entire business scenario and risk background after receiving the prompt.

[0111] 2. Model Parses the Prompt

[0112] The pre-trained large language model parses each part of the prompt, understands the code functionality, input / output, control flow, dependencies, and risk assessment requirements. The model automatically identifies the requirements for each test objective, such as functional regression risk testing, performance impact testing, and security vulnerability testing, and generates corresponding test cases according to the prompt.

[0113] 3. Automatically Generate Test Cases

[0114] The test cases generated by the model based on the prompt usually include:

[0115] Function regression test cases

[0116] For example:

[0117] Test case 1: Use a normal and legal input string, and expect to return true.

[0118] Test case 2: Use an input string with boundary values (e.g., exactly 100 characters), and expect to return true.

[0119] Test case 3: Use an illegal input string containing special characters, expect to return false, and record the error message.

[0120] Performance impact test cases

[0121] For example:

[0122] Test case 1: In a high-concurrency scenario, simulate multiple threads calling validateInput simultaneously, record the response time and system resource occupancy, and ensure that the validation logic can still operate efficiently under high load.

[0123] Test case 2: Pass in test data with a large amount of data (e.g., continuous calls or batch inputs), and evaluate the system response performance and stability.

[0124] Security vulnerability test cases

[0125] For example:

[0126] Test case 1: Pass in an extremely long string (far exceeding 100 characters) as input, and verify whether the system correctly rejects this input to prevent buffer overflow or other security issues.

[0127] Test case 2: Pass in an input containing multiple special characters (e.g., quotation marks, semicolons, SQL injection attempt characters), and verify whether the system can effectively filter and prevent security vulnerabilities.

[0128] The method of the present invention can automatically generate detailed test cases covering function regression risks, performance impacts, and security vulnerabilities by automatically parsing code differences, performing risk assessments, and generating structured prompt words. This automated process significantly reduces the workload of testers in manually writing test cases and improves test efficiency.

[0129] In an alternative embodiment,

[0130] Function regression risk cases can be generated for newly added code, deleted code, or modified code;

[0131] Performance impact test cases can be generated for newly added code, deleted code, or changed code;

[0132] Security vulnerability test cases can be generated for newly added code or changed code.

[0133] The goal of the functional regression risk test is to ensure that code changes do not break the functionality of the existing system. Whether code is newly added, deleted, or changed, it may affect the operation of other parts of the system, so test cases are needed to verify the correctness of the overall functionality.

[0134] The purpose of the performance impact test is to evaluate the impact of code changes on system response time, resource utilization, throughput, and other metrics. The addition, deletion, or change of code may all change system performance, so corresponding test cases need to be generated.

[0135] The goal of the security vulnerability test cases is to verify the security protection capabilities of the newly added or changed code, preventing security issues caused by oversights in input validation, permission checks, etc.; generally, security test cases are not generated for deleted code because deletion operations usually do not directly introduce new security vulnerabilities.

[0136] By generating corresponding test cases for different types of code changes, the present invention can achieve comprehensive, risk-oriented automated testing, ensuring both the consistency of system functionality and full verification of performance and security.

[0137] Figure 2 is a test case generation system provided by an embodiment of the present invention, as Figure 2 shown, the system includes:

[0138] A parsing unit 201, configured to parse code changes through a code difference analysis module to extract key code information and identify the type of code difference;

[0139] In an optional embodiment, the parsing of code changes through the code difference analysis module to extract key code information includes:

[0140] Parsing code changes through an abstract syntax tree to extract key code information including function signatures, input parameters, return values, control flow, and dependencies.

[0141] A risk assessment unit 202, configured to perform a risk assessment on the code difference to obtain a risk assessment result;

[0142] A conversion unit 203, configured to use a natural language processing model to convert the extracted key code information into a natural language description;

[0143] In an alternative embodiment, the natural language description includes code function description, input / output description, control flow description, and dependency description.

[0144] A construction unit 204, configured to construct a structured prompt word according to the risk assessment result of code differences and the natural language description;

[0145] A test case generation unit 205, configured to input the constructed structured prompt word into a pre-trained large language model to automatically generate test cases, where the generated test cases include: functional regression risk test cases, performance impact test cases, and security vulnerability test cases.

[0146] In an alternative embodiment, the functional regression risk test cases include normal input, boundary value input, and illegal input scenarios;

[0147] The performance impact test cases include high-concurrency input and large data volume input scenarios;

[0148] The security vulnerability test cases include ultra-long string input and input scenarios containing special characters.

[0149] In an alternative embodiment, the types of code differences include: newly added code, deleted code, and changed code;

[0150] Functional regression risk cases can be generated for newly added code, deleted code, or changed code;

[0151] Performance impact cases can be generated for newly added code, deleted code, or changed code;

[0152] Security vulnerability test cases can be generated for newly added code or changed code.

[0153] The system of the present invention corresponds to the above method, and the specific implementation manners of the system will not be repeated here.

[0154] Advantages of the present invention:

[0155] The present invention provides a test case generation method and system. Among them, the method performs a risk assessment on code differences, extracts key code information by using techniques such as abstract syntax trees (ASTs), and converts the code information into a clear natural language description with the help of a natural language processing model, and further constructs structured prompt words to guide a pre-trained large language model to automatically generate targeted test cases, so as to achieve comprehensiveness and efficiency of test coverage. This automated process greatly reduces the workload of testers in manually writing test cases and improves test efficiency.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A test case generation method, characterized in that: include: S1. Analyze the code changes through the code difference analysis module to extract key code information and identify the code difference type; S2. Perform risk assessment on the code differences to obtain risk assessment results; S3, using a natural language processing model to convert the extracted key code information into a natural language description; S4. Construct structured prompt words based on the risk assessment results of code differences and natural language descriptions; S5. Input the constructed structured prompt words into the pre-trained large language model to automatically generate test cases, wherein the generated test cases include: functional regression risk test cases, performance impact test cases, and security vulnerability test cases.

2. The method according to claim 1, characterized in that The code difference analysis module is used to analyze the code changes to extract key code information, including: Code changes are parsed through the abstract syntax tree to extract key code information including function signatures, input parameters, return values, control flow, and dependencies.

3. The method according to claim 1, characterized in that: The natural language description includes code function description, input and output description, control flow description and dependency description.

4. The method according to claim 1, characterized in that: The functional regression risk test cases include normal input, boundary value input and illegal input scenarios; The performance impact test cases include high concurrent input and large data volume input scenarios; The security vulnerability test cases include scenarios where extra-long character string input is used and input contains special characters.

5. The method according to claim 1, characterized in that: The code difference types include: added code, deleted code, and changed code; Functional regression risk use cases can be generated for new code, deleted code, or changed code; Generate performance impact test cases for adding new code, deleting code, or changing code; Security vulnerability test cases can be generated for new or modified code.

6. A test case generation system, characterized in that: include: A parsing unit, used to parse the code changes through the code difference analysis module to extract key code information and identify the code difference type; A risk assessment unit, used to perform risk assessment on code differences to obtain risk assessment results; A conversion unit, used for converting the extracted key information of the code into a natural language description by using a natural language processing model; A construction unit, used to construct structured prompt words according to the risk assessment results of the code differences and the natural language description; The test case generation unit is used to input the constructed structured prompt words into the pre-trained large language model to automatically generate test cases, wherein the generated test cases include: functional regression risk test cases, performance impact test cases, and security vulnerability test cases.

7. The system according to claim 6, characterized in that The code difference analysis module is used to analyze the code changes to extract key code information, including: Code changes are parsed through the abstract syntax tree to extract key code information including function signatures, input parameters, return values, control flow, and dependencies.

8. The system according to claim 6, characterized in that: The natural language description includes code function description, input and output description, control flow description and dependency description.

9. The system according to claim 6, characterized in that: The functional regression risk test cases include normal input, boundary value input and illegal input scenarios; The performance impact test cases include high concurrent input and large data volume input scenarios; The security vulnerability test cases include scenarios where extra-long character string input is used and input contains special characters.

10. The system according to claim 6, characterized in that: The code difference types include: added code, deleted code, and changed code; Functional regression risk use cases can be generated for new code, deleted code, or changed code; Generate performance impact test cases for adding new code, deleting code, or changing code; Security vulnerability test cases can be generated for new or modified code.

Images