Software normalization test case adaptive generation method based on AI decision

Through the software normalized test cases adaptive generation method based on AI decision-making, GAN training generator and discriminator is used to extract state features and generate test cases that meet daily maintenance needs, which solves the problem of too large number of acceptance test cases and improves the Daily system testing efficiency and reduce the test scale.

CN118916267APending Publication Date: 2024-11-08CHANGZHOU UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410931620.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-12
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The number of test cases used in acceptance tests is too large, resulting in low efficiency in daily system testing.

Method used

Adaptive generation method of software normalized test cases based on AI decision-making is adopted. By importing information from the test case set parameter database as the initial data for model training, state features are extracted, and GAN training generators and discriminators are used to generate daily maintenance needs. Test cases and evaluate coverage and diversity through fitness function.

Benefits of technology

While ensuring coverage, the scale of test cases is significantly reduced, the efficiency of daily maintenance and testing is improved, and the normal operation and economic benefits of daily systems are ensured.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916267A_ABST
    Figure CN118916267A_ABST
Patent Text Reader

Abstract

The invention relates to the field of software testing, and provides a software normalization test case self-adaptive generation method based on AI decision, which comprises the following steps: importing information from a test case set parameter database as model training initial data; sFICF feature extraction is carried out on the test cases in the data set, and state features are extracted for subsequent model training and classification; the detection parameters are injected into a GAN training model to train a generator and a discriminator related to the test case, the generator and the discriminator play a game mutually, and the parameters of the generator and the discriminator are continuously optimized to generate the required test case; and according to the coverage rate and diversity of the test cases, evaluating whether the test cases generated after model training meet requirements or not. According to the SFICF feature extraction method provided by the invention, the test case is converted into the state vector, the GAN is used for training the state vector, the test case meeting the requirement of the daily maintenance test is generated, the test scale is reduced, the daily maintenance test efficiency is improved, and normal operation of a daily system is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This patent application relates to the field of software testing. The present invention proposes a method for adaptively generating software normalized test cases based on AI decision-making. Background Art

[0002] As software scale expands and functions increase, the complexity of software also increases. Software testing can discover and resolve errors and defects in software, reduce the risk of software failure and failure, and thus improve the stability and reliability of software; and because software requirements change frequently, there may be some problems and risks in the software, and software testing can timely discover and resolve these problems and risks, thereby ensuring that the software is developed and delivered in accordance with user requirements and expectations; similarly, security issues in software may also increase, and software testing can discover and fix security issues in software, thereby ensuring the security of user data and assets; the expansion of software scale also leads to an increase in software testing workload. In order to save time and cost, automated testing is needed to replace manual testing. Automated testing can improve testing efficiency, reduce testing costs, and ensure software quality.

[0003] Automatic generation of test cases is one of the important links in software test automation. Automatic generation of test cases is not only needed in software acceptance testing, but also in daily production. However, the number of test cases required for acceptance testing is too large. Daily system testing does not require such a large number of test cases. Executing such a large number of test cases is not only a huge workload, but also affects daily production work. Therefore, how to automatically generate test cases that meet the needs of daily maintenance is a problem that needs to be solved. Summary of the invention

[0004] The technical problem to be solved by the present invention is that the number of test cases used in the acceptance test is too large, which leads to low efficiency of daily system testing when used for daily system testing.

[0005] To this end, the present invention provides a method for adaptively generating software normalization test cases based on AI decision-making.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A method for adaptively generating software normalized test cases based on AI decision-making, comprising the following steps:

[0008] Step 1: Import information from the test case set parameter database as initial data for model training;

[0009] Step 2: Perform SF_ICF feature extraction on the test cases in the data set and extract state features from the initial data for subsequent model training and classification;

[0010] Step 3: Inject the detection parameters into the GAN training model to train the generator and discriminator related to the test case respectively. The generator and discriminator compete with each other and continuously optimize their own parameters to improve the performance and training effect of the model and generate the required test cases.

[0011] Step 4: Calculate the coverage and diversity of the generated test cases to evaluate whether the test cases generated after model training meet the requirements, and output the test cases that meet the requirements.

[0012] Furthermore, in step 2, based on two indicators, state frequency and inverse chain frequency, state feature extraction is performed on the test case generated based on the Markov chain.

[0013] Further, in step 2, the SF_ICF feature extraction method specifically includes the following steps:

[0014] S2.1 divides the Markov chain of test cases imported from the database into independent states one by one;

[0015] S2.2 calculates the state frequency and reverse chain frequency of each independent state in the Markov chain of each test case under the test case set;

[0016] S2.3 converts the test case into a state vector, evaluates the effectiveness of the test case state vector, and assesses the comprehensive performance value of the state vector in the test case execution link.

[0017] Furthermore, the state frequency is Among them, SO is the number of edges connected to the state in the Markov chain, ST is the total number of states in the test case set; the reverse chain frequency is Among them, NTD contains the number of test cases in the database, and CNTS contains the number of test cases in this state.

[0018] Furthermore, the effective value of the test case state vector is SIC, SIC=SF_ICF×COV, where SF_ICF is the product of SF and ICF, and COV is the coverage of the test case where the state vector belongs.

[0019] Furthermore, the training process of GAN in step 3 specifically includes:

[0020] S3.1 defines the network structure of the generator and the discriminator. The input of the generator is a random noise, and the output is a fake data similar to the real data. The input of the discriminator is real data or fake data, and the output is a binary classification result.

[0021] S3.2 Randomly initialize the parameters of the generator and discriminator.

[0022] Furthermore, in step 3.2, the following steps are looped until the model converges or reaches a preset number of training times:

[0023] S3.2.1 Randomly select a batch of data samples from the real data set, i.e., the state vectors collected in step 2 as training samples for the discriminator;

[0024] S3.2.2 Generate a batch of fake data samples from random noise as the output of the generator;

[0025] S3.2.3 Let the discriminator classify real data and fake data respectively and calculate the loss function;

[0026] S3.2.4 Based on the output of the discriminator, let the generator adjust its own parameters so that the generated fake data is close to the real data, and recalculate the loss function;

[0027] S3.2.5 Update the parameters of the discriminator and generator, and use optimization algorithms such as Adam to update the parameters.

[0028] Furthermore, the update formula of the Adam algorithm is as follows:

[0029]

[0030] Among them, m t and v t are the first-order moment estimate and the second-order moment estimate of the parameter, g t is the gradient of the parameter, β1 and β2 are attenuation coefficients, usually 0.9 and 0.999, {m} t and {v} t is the first-order moment estimate and the second-order moment estimate after bias correction, α is the learning rate, which is usually set to 0.001, and ∈ is a very small number, which is set to 10 -8 , to prevent the divisor from being 0, where the first-order moment estimate m is updated t and the second moment estimate v t : Estimate m of the first moment t and the second moment estimate v t To make a deviation correction: According to the modified first-order moment estimate {m} t and the second-order moment estimate {v) tTo update the model parameters:

[0031] Furthermore, in step four, the fitness function is used to evaluate the coverage and diversity. The fitness function fit(x) is shown in the following formula: fit(x) = w1*f1(x)+w2*f2(x), where x represents a test case, f1(x) represents the coverage of the test case, f2(x) represents the diversity of the test case, and w1 and w2 are weight coefficients.

[0032] Furthermore, assuming that the code of the tested software contains N code lines or branches, after executing test case x, M code lines or branches are covered, the coverage f1(x) of test case x is calculated as shown in the following formula: Diversity can be calculated by calculating the similarity between test cases. Assuming that the similarity measurement method is d(x1,x2), where x1 and x2 represent two test cases, the diversity f2(x) of test case x can be calculated as shown in the following formula: Among them, x' represents other test cases in the test case family, and sum represents the sum of the similarities between all test cases.

[0033] The beneficial effect of the present invention is that the present invention proposes a SF_ICF feature extraction method, extracts state features of test cases generated based on Markov chains, converts test cases into state vectors, and uses GAN to train the extracted vector data to generate test cases that meet the needs of daily maintenance testing. While ensuring the coverage, the scale of testing required is greatly reduced, the efficiency of daily maintenance testing is improved, the normal operation of daily systems is ensured, and economic benefits are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0035] Figure 1 It is a schematic diagram of the implementation process of a method for adaptively generating software normalized test cases based on AI decision-making in the present invention. DETAILED DESCRIPTION

[0036] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0037] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0038] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0039] A method for adaptively generating software normalized test cases based on AI decision-making, comprising the following steps:

[0040] Step 1: Import the required information from the test case set parameter database as the initial data for model training.

[0041] The system is tested in real time during the acceptance phase of the software. All the data of the generated test cases are packaged and stored in the chain evolution strategy dual-station real-time database, which is exported for feature extraction in step two.

[0042] Step 2: Perform SF_ICF feature extraction on the test cases in the data set and extract state features from the original data for subsequent model training and classification.

[0043] The present invention proposes a SF_ICF feature extraction method for extracting state features from test cases generated based on Markov chains. It is based on two indicators, state frequency and inverse chain frequency, to measure the importance of a state to the Markov chain. The specific implementation method of SF_ICF is as follows:

[0044] S2.1 State segmentation

[0045] The Markov chain of the test cases imported from the database is divided into independent states one by one. For a certain state in a Markov chain, when all the previous states are known, the state is independent of the subsequent states. In other words, the current state is only related to its previous state and is independent of other states. For example, if we are simulating the driving process of a car on the road, the state may include continuous variables such as the speed and the steering wheel angle. These continuous vectors are combined to form a complete Markov chain related to the driving process of the car. Variables such as the speed and the steering wheel angle are separated and do not participate in the composition of the Markov chain. At this time, the variable is an independent state.

[0046] S2.2 Calculation of status frequency

[0047] State frequency is the frequency of each independent state in the Markov chain that describes each test case in the test case set. The calculation formula is as follows:

[0048]

[0049] Among them, SF is the state frequency, that is, the frequency of the state appearing in the test case set, SO is the number of edges connecting the state with it in the Markov chain, and ST is the total number of states in the test case set.

[0050] S2.3 Calculation of reverse chain frequency

[0051] Inverse Chain Frequency (ICF) is used to measure the importance of a test case to the entire test case set.

[0052] The calculation formula is as follows:

[0053]

[0054] Among them, ICF is the reverse chain frequency, the number of test cases in the NTD database, and the number of test cases in the CNTS state.

[0055] S2.4 Calculation of SF_ICF features

[0056] Multiplying SF and ICF gives the SF_ICF value, which represents the importance of a test case in the test case database. Using SF_ICF, we can represent each state as a vector and the entire test case set as a weighted sum of vectors.

[0057] After converting the test case into a state vector and obtaining the SF_ICF value of the state, calculate its SIC value. The specific formula is as follows:

[0058] SIC=SF_ICF×COV=SF×ICF×COV

[0059] The SIC value is the effectiveness value of the test case state vector, which is used to evaluate the comprehensive performance value of the importance of the state vector in the test case execution link, and COV is the coverage rate of the test case where the state vector is located.

[0060] Through the above steps, a test case can be converted into a vector representation. Different test cases have different importance in system testing, and different importance can also have different probabilities of output in the deep learning process.

[0061] Step 3: Inject the Markov chain parameters after state extraction into the GAN training model to train the generator and discriminator related to the test case respectively. The generator and discriminator compete with each other and continuously optimize their own parameters to improve the performance and training effect of the model and generate the required test cases.

[0062] GAN (Generative Adversarial Network) is a deep learning model consisting of two modules: Generator and Discriminator. GAN is an effective generative model that generates new data similar to real data by training the generator and discriminator.

[0063] The basic idea of ​​GAN is to put the generator and the discriminator in a game environment, allowing the generator to generate fake samples as realistic as possible, and the discriminator to distinguish between real and fake samples as much as possible. During the training process, the generator and the discriminator compete with each other and constantly adjust their own parameters to improve the performance of the model and the training effect. Specifically, the training process of GAN can be divided into the following steps:

[0064] S3.1 defines the network structure of the generator and the discriminator. The input of the generator is a random noise, and the output is a fake data similar to the real data. The input of the discriminator is real data or fake data, and the output is a binary classification result (true or false).

[0065] S3.2 Randomly initialize the parameters of the generator and discriminator.

[0066] Cycle through the following steps:

[0067] S3.2.1 Randomly select a batch of data samples from the real data set, that is, the state vectors collected in step 2 as the training samples of the discriminator. The Markov chain state data before extraction is essentially human-oriented data, and its readability is very poor for the training system. Converting it into a state vector through extraction can make it closer to machine language, which can not only improve the system's readability of samples, make it convenient to apply machine learning algorithms to the data, but also improve the efficiency of text processing.

[0068] S3.2.2 Generate a batch of fake data samples from random noise as the output of the generator.

[0069] S3.2.3 Let the discriminator classify the real data and fake data respectively, and calculate the loss function. The commonly used loss function is Hinge Loss, and the specific formula is as follows:

[0070]

[0071] Among them, m is the batch size, y i is the true label, x i is the input data, D(x i ) is the output of the discriminator, λ is the regularization parameter, θ D are the parameters of the discriminator.

[0072] S3.2.4 Based on the output of the discriminator, let the generator adjust its own parameters to make the generated fake data as realistic as possible, and calculate the loss function Hinge Loss function. The specific formula is as follows:

[0073]

[0074] Among them, z i is random noise, G(z i ) is the output of the generator, D(G(z i )) is the classification result of the generator output by the discriminator, λ is the regularization parameter, θ G The parameters for the generator.

[0075] S3.2.5 Update the parameters of the discriminator and generator, and use optimization algorithms such as Adam to update the parameters.

[0076] The core idea of ​​the Adam algorithm is to adaptively adjust the learning rate of each parameter. It maintains the first-order moment estimate (i.e., the first-order moment of the gradient, also called the average gradient) and the second-order moment estimate (i.e., the second-order moment of the gradient, also called the average square gradient) of each parameter, and calculates the update amount of the parameter based on these moment estimates. Specifically, the update formula of the Adam algorithm is as follows:

[0077]

[0078] Among them, m t and v t are the first-order moment estimate and the second-order moment estimate of the parameter, g t is the gradient of the parameter, β1 and β2 are attenuation coefficients, usually 0.9 and 0.999, {m} t and {v} tis the first-order moment estimate and the second-order moment estimate after bias correction, α is the learning rate, which is usually set to 0.001, and ∈ is a very small number, which is set to 10 -8 , to prevent division by 0.

[0079] Where: Update the first-order moment estimate m t and the second moment estimate v t :

[0080]

[0081] Estimate m of the first moment t and the second moment estimate v t To make a deviation correction:

[0082]

[0083] According to the modified first-order moment estimate {m} t and the second-order moment estimate {v} t To update the model parameters:

[0084]

[0085] As can be seen from the formula, the Adam algorithm can adaptively adjust the learning rate of each parameter and update the parameters based on historical gradient information. At the same time, the Adam algorithm can also effectively handle sparse gradients, avoiding the problems that may occur when the SGD algorithm handles sparse gradients.

[0086] S3.3 Repeat steps S3.2.1 to S3.2.5 until the model converges or reaches the preset training rounds.

[0087] In the above process, the generator and the discriminator compete with each other. During the training process, the generator constantly tries to generate more realistic test cases, while the discriminator constantly tries to distinguish true and false test cases, and constantly adjusts its own parameters to improve the performance and training effect of the model. Through this adversarial learning method, the generator can gradually learn the skills of generating realistic test cases, and the discriminator can also gradually improve its ability to distinguish between real and fake, thereby generating test cases with high efficiency and accuracy.

[0088] Step 4: Calculate the coverage and diversity of the generated test cases to evaluate whether the test cases generated after model training meet the requirements.

[0089] The test cases generated by GAN also need to calculate their coverage and diversity to evaluate whether the generated test cases can be used as result output.

[0090] Specifically, the fitness function fit(x) is shown in the following formula:

[0091] fit(x)=w1*f1(x)+w2*f2(x)

[0092] Where x represents a test case, f1(x) represents the coverage of the test case, f2(x) represents the diversity of the test case, and w1 and w2 are weight coefficients used to balance coverage and diversity.

[0093] The weight coefficients w1 and w2 can be adjusted according to the specific situation to balance the importance of coverage and diversity. Generally speaking, coverage is very important for discovering software defects, so w1 can be set to a larger value and w2 can be set to a smaller value to emphasize the coverage of test cases.

[0094] The operator weights of the corresponding test cases can be obtained by importing them into the database, and the coverage is calculated based on the operator weights obtained from the database.

[0095] The coverage rate can be calculated by counting the number of code lines or branches covered during the execution of the test case. Specifically, assuming that the code of the tested software contains N code lines or branches, after executing test case x, M code lines or branches are covered. The coverage rate f1(x) of test case x is calculated as shown in the following formula:

[0096]

[0097] Diversity can be calculated by calculating the similarity between test cases. Specifically, some similarity measurement methods can be used to calculate the similarity between test cases. Assuming that the similarity measurement method is d(x1,x2), where x1 and x2 represent two test cases, the diversity f2(x) of test case x can be calculated as shown in the following formula:

[0098]

[0099] Among them, x' represents other test cases in the test case family, and sum represents the sum of the similarities between all test cases.

[0100] Step 5: Determine whether the test case meets the requirements. The specific requirements are set according to the required coverage and adaptability of the system under test. Test cases that meet the requirements are output, and those that do not meet the requirements are regenerated by repeating steps 2 to 4.

[0101] In summary, the present invention proposes a SF_ICF feature extraction method, which extracts state features of test cases generated based on Markov chain, converts test cases into state vectors, and uses GAN to train the extracted vector data to generate test cases that meet the needs of daily maintenance testing. While ensuring the coverage, it greatly reduces the scale of testing required, improves the efficiency of daily maintenance testing, ensures the normal operation of daily systems, and improves economic benefits.

[0102] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for adaptively generating software normalized test cases based on AI decision making, characterized in that: The following steps are included: Step 1: Import information from the test case set parameter database as initial data for model training; Step 2: Perform SF_ICF feature extraction on the test cases in the data set and extract state features from the initial data for subsequent model training and classification; Step 3: Inject the detection parameters into the GAN training model to train the generator and discriminator related to the test case respectively. The generator and discriminator compete with each other and continuously optimize their own parameters to improve the performance and training effect of the model and generate the required test cases. Step 4: Calculate the coverage and diversity of the generated test cases to evaluate whether the test cases generated after model training meet the requirements, and output the test cases that meet the requirements.

2. The method for adaptively generating software normalized test cases based on AI decision-making according to claim 1 is characterized in that: In step 2, based on the two indicators of state frequency and inverse chain frequency, state features are extracted for test cases generated based on Markov chains.

3. The method for adaptively generating software normalized test cases based on AI decision-making according to claim 2 is characterized in that: In step 2, the SF_ICF feature extraction method specifically includes the following steps: S2.1 divides the Markov chain of test cases imported from the database into independent states one by one; S2.2 calculates the state frequency and reverse chain frequency of each independent state in the Markov chain of each test case under the test case set; S2.3 converts the test case into a state vector, evaluates the effectiveness of the test case state vector, and assesses the comprehensive performance value of the state vector in the test case execution link.

4. The method for adaptively generating software normalized test cases based on AI decision-making according to claim 2 is characterized in that: The frequency is Among them, SO is the number of edges connected to the state in the Markov chain, ST is the total number of states in the test case set; the reverse chain frequency is Among them, NTD contains the number of test cases in the database, and CNTS contains the number of test cases in this state.

5. The method for adaptively generating software normalized test cases based on AI decision-making according to claim 4 is characterized in that: The effective value of the test case state vector is SIC, SIC = SF_ICF × COV, where SF_ICF is the product of SF and ICF, and COV is the coverage of the test case in which the state vector is located.

6. The method for adaptively generating software normalized test cases based on AI decision-making according to claim 1 is characterized in that: The training process of GAN in step 3 specifically includes: S3.1 defines the network structure of the generator and the discriminator. The input of the generator is a random noise, and the output is a fake data similar to the real data. The input of the discriminator is the real data or fake data, and the output is a binary classification result. S3.2 Randomly initialize the parameters of the generator and discriminator.

7. The method for adaptively generating software normalized test cases based on AI decision-making according to claim 6 is characterized in that: In step 3.2, the following steps are repeated until the model converges or reaches a preset number of training times: S3.2.1 Randomly select a batch of data samples from the real data set, i.e., the state vectors collected in step 2 as training samples for the discriminator; S3.2.2 Generate a batch of fake data samples from random noise as the output of the generator; S3.2.3 Let the discriminator classify real data and fake data respectively and calculate the loss function; S3.2.4 Based on the output of the discriminator, let the generator adjust its own parameters so that the generated fake data is close to the real data, and recalculate the loss function; S3.2.5 Update the parameters of the discriminator and generator, and use optimization algorithms such as Adam to update the parameters.

8. The method for adaptively generating software normalized test cases based on AI decision-making according to claim 7 is characterized in that: The update formula of the Adam algorithm is as follows: Among them, m t and v t are the first-order moment estimate and the second-order moment estimate of the parameter, g t is the gradient of the parameter, β1 and β2 are attenuation coefficients, usually 0.9 and 0.999, {m} t and {v} t is the first-order moment estimate and the second-order moment estimate after bias correction, α is the learning rate, which is usually set to 0.001, and ∈ is a very small number, which is set to 10 -8 , to prevent the divisor from being 0, where the first-order moment estimate m is updated t and the second-order moment estimate v t : Estimate m of the first moment t and the second-order moment estimate v t To make a deviation correction: According to the modified first-order moment estimate {m} t and the second-order moment estimate {v} t To update the model parameters:

9. The method for adaptively generating software normalized test cases based on AI decision-making according to claim 1 is characterized in that:In step 4, the fitness function is used to evaluate the coverage and diversity. The fitness function fit(x) is shown in the following formula: fit(x) = w1*f1(x)+w2*f2(x), where x represents a test case, f1(x) represents the coverage of the test case, f2(x) represents the diversity of the test case, and w1 and w2 are weight coefficients.

10. The method for adaptively generating software normalized test cases based on AI decision-making according to claim 9 is characterized in that: Assume that the code of the tested software contains N code lines or branches. After executing test case x, M code lines or branches are covered. The coverage f1(x) of test case x is calculated as shown in the following formula: Diversity can be calculated by calculating the similarity between test cases. Assuming that the similarity measurement method is d(x1,x2), where x1 and x2 represent two test cases, the diversity f2(x) of test case x can be calculated as shown in the following formula: Among them, x' represents other test cases in the test case family, and sum represents the sum of the similarities between all test cases.

Citation Information

Cited By

  • Method and system for generating software test case based on adversarial network

    CN120763071A