An intelligent test case generation method based on an AI adversarial model

Abstract

The application relates to an intelligent test case generation method based on an AI adversarial model, and belongs to the technical field of computer software testing, which comprises the following steps: constructing an initial training set according to system information to be tested, test requirements and historical test data; generating a test case by using a generator of the AI adversarial model based on the initial training set; evaluating the quality of the test case by using a discriminator, and iteratively optimizing the generator based on the evaluation result; executing the generated test case, recording execution logs and intermediate results; and evaluating the execution logs and the intermediate results until a preset test termination condition is met. The test case generated by using the AI adversarial model has stronger adversarialness and pertinence, and can effectively find hidden defects that are difficult to be found by traditional methods.

Description

Technical Field

[0001] This invention relates to the field of computer software testing technology, and specifically to an intelligent test case generation method based on an AI adversarial model. Background Technology

[0002] As software systems become increasingly complex, traditional methods of manually designing and executing test cases are no longer sufficient to meet the demands for efficient and comprehensive testing. While some automated testing tools and AI-assisted testing methods have emerged, they suffer from the following shortcomings:

[0003] 1. Test case generation relies on rule bases or simple machine learning models, making it difficult to generate high-value test cases with adversarial and boundary-based characteristics;

[0004] 2. The level of intelligence in test execution and result analysis is low, making it difficult to achieve adaptive adjustments in the test process;

[0005] 3. Test coverage assessments often rely on single metrics such as code coverage, lacking in-depth analysis of potential risk points;

[0006] 4. Insufficient integration of AI technology across the entire testing process (from test case generation to result analysis and defect localization) creates information silos. Summary of the Invention

[0007] To address the aforementioned issues, the purpose of this invention is to provide an intelligent test case generation method based on an AI adversarial model.

[0008] A method for generating intelligent test cases based on an AI adversarial model, comprising:

[0009] Step 1: Construct an initial training set based on the information of the system under test, test requirements, and historical test data;

[0010] Step 2: Generate test cases based on the initial training set using the generator of the AI ​​adversarial model;

[0011] Step 3: Evaluate the quality of test cases using a discriminator, and iteratively optimize the generator based on the evaluation results;

[0012] Step 4: Execute the generated test cases and record the execution log and intermediate results;

[0013] Step 5: Evaluate the execution logs and intermediate results until the preset test termination conditions are met.

[0014] Preferably, in step 2, the numerical features in the initial training set are normalized, and the text in the initial training set is converted into vectors using the Word2Vec model.

[0015] Preferably, in step 2, the generator is trained using a Transformer model, and the learning rate is optimized using the Adam optimizer during training; wherein, the optimization process of the Adam optimizer is as follows:

[0016]

[0017] in, , This represents the initial learning rate. This indicates the number of iterations in the current model training. The number of warm-up steps indicates the learning rate. This represents the actual learning rate at the current iteration step.

[0018] Preferably, in step 3, the discriminator is trained using a convolutional neural network or a gradient boosting model.

[0019] Preferably, in step 3, a weighted summation is used to calculate the final evaluation result; wherein the formula for calculating the evaluation result is:

[0020]

[0021] in, Indicates the evaluation results. This represents the first weighting coefficient. The novelty score is calculated based on the feature similarity between the test cases and historical test cases. This represents the second weighting coefficient. The adversarial score is calculated based on historical defect data to assess the probability that test cases will trigger system vulnerabilities. This represents the third weighting coefficient. The requirement matching score is calculated by matching the keywords between the test cases and the test requirement document.

[0022] Preferably, in step 3, the generator and discriminator adopt an alternating training mode, achieving collaborative optimization through minimax game; wherein, the objective function in the adversarial training process is the minimax value function:

[0023]

[0024] in, This represents the distribution of real, high-quality test cases. Represents real, high-quality use cases. The distribution of input noise to the generator, Indicates to Expectation calculation, Represents random variables Follow the distribution of real high-quality use cases , This indicates that the generator is input with a random vector. Expectation calculation, Represents random variables Follow the input noise distribution , A random vector representing the input generator. The test cases output by the generator. For the discriminator to identify real high-quality use cases The probability of being judged as "high-quality" For the discriminator to generate test cases The probability of being judged as "high-quality".

[0025] Preferably, in step 4, the cosine similarity is used to calculate the matching degree between the features of the test case to be executed and the features of the target system. When the matching degree is ≥0.8, it is determined to be adapted and executed directly; when the matching degree is <0.8, the optimal adaptation scheme is selected by minimizing the conversion cost.

[0026] Preferably, the evaluation results output by the discriminator are used as the basis for test case priority, and the maximum priority priority strategy is adopted to realize the dynamic scheduling of test cases.

[0027] The present invention also provides an electronic device, including a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor. The transceiver, the memory, and the processor are connected via the bus. The computer program, when executed by the processor, implements the steps in the above-described method for generating intelligent test cases based on an AI adversarial model.

[0028] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps in the above-described method for generating intelligent test cases based on an AI adversarial model.

[0029] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0030] This invention relates to an intelligent test case generation method based on an AI adversarial model. Compared with existing technologies, this invention has the following advantages:

[0031] 1. Test cases generated using AI adversarial models are more adversarial and targeted, and can effectively discover hidden defects that are difficult to detect using traditional methods.

[0032] 2. Achieve full automation and intelligence in the process of test case generation, execution, and analysis, significantly improving testing efficiency and reducing labor costs;

[0033] 3. Continuously optimize the AI ​​model through a dynamic feedback mechanism to enable the testing process to be adaptive and learnable, thereby continuously improving the quality of testing;

[0034] 4. Achieve deep integration of AI technology into the entire testing process to ensure comprehensive and effective test coverage and provide reliable quality assurance for complex software systems.

[0035] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 The present invention provides a flowchart of an intelligent test case generation method based on an AI adversarial model. Detailed Implementation

[0038] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0039] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0040] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0041] Please see Figure 1 A method for generating intelligent test cases based on an AI adversarial model, comprising:

[0042] Step 1: Construct an initial training set based on the information of the system under test, test requirements, and historical test data;

[0043] Step 2: Generate test cases based on the initial training set using the generator of the AI ​​adversarial model;

[0044] In step 2, the numerical features in the initial training set are normalized, and the Word2Vec model is used to convert the text in the initial training set into vectors; the normalization formula is as follows:

[0045]

[0046] in, The characteristic mean, The characteristic standard deviation is denoted as .

[0047] Step 3: Evaluate the quality of test cases using a discriminator, and iteratively optimize the generator based on the evaluation results;

[0048] In step 3, the discriminator is a Transformer model. The Transformer model includes:

[0049] 1) Location coding

[0050]

[0051]

[0052] Among them, the position encoding is the Transformer's supplementary sequence position information. For sequence position, For dimensional indexing, Input dimensions for the model.

[0053] 2) Bullish Self-Attention

[0054] 1. Linear transformation to generate Q / K / V

[0055]

[0056] 2. Single-head attention calculation

[0057]

[0058] 3. Multi-head splicing

[0059]

[0060] ( (For attention heads), capture the dependencies between parameters.

[0061] 4. Mask self-attention

[0062]

[0063] in, For the mask matrix (the upper triangle is...) This prevents the generation process from seeing future outputs in advance.

[0064] 5. Feedforward Neural Network

[0065] formula:

[0066]

[0067] illustrate: , This enhances the model's fitting ability.

[0068] During training, the generator uses the following cross-entropy loss function:

[0069]

[0070] in, For the sample size, The sequence length is given.

[0071] The Adam optimizer is used to optimize the learning rate during training; the optimization process of the Adam optimizer is as follows:

[0072]

[0073] in, , This represents the initial learning rate. This indicates the number of iterations in the current model training. The number of warm-up steps indicates the learning rate. This represents the actual learning rate at the current iteration step.

[0074] The discriminator can choose from two core models, allowing for flexible selection based on the test scenario:

[0075] 1. Convolutional Neural Network (CNN): Suitable for processing feature matrices (such as parameter combination encoding matrices) of structured test cases, adept at capturing local feature correlations, and improving the accuracy of adversarial evaluation;

[0076] 2. Gradient Boosting Model (XGBoost / LightGBM): Suitable for discrete evaluation features (such as novelty labels and demand matching keywords), with fast training speed, strong interpretability, and suitable for rapid iteration testing scenarios.

[0077] Core formula of Convolutional Neural Network (CNN)

[0078] Convolutional layer calculation:

[0079]

[0080] illustrate: Let (i,j) be the value at position (i,j) of the output feature map of the l-th convolutional layer. The kernel size is [size]. For convolution kernel weights, For bias, The ReLU activation function is used to extract local correlation patterns of test case features.

[0081] Pooling layer calculation (max pooling):

[0082]

[0083] illustrate: The pooling window size is used to reduce dimensionality while retaining key features, thereby improving the model's generalization ability.

[0084] (2) Core formula of gradient boosting model

[0085] Loss function (mean squared error):

[0086]

[0087] illustrate: This is a genuine rating (1-10 points). The model is given a predicted score, which is used to optimize the model's fitting accuracy to the evaluation metric.

[0088] Model update formula:

[0089]

[0090] illustrate: For the model after the t-th iteration, Set the learning rate to 0.1. To add a new base learner (decision tree), the prediction effect is optimized by gradually stacking base learners.

[0091] The final score is calculated using a weighted summation method, as shown in the following formula:

[0092]

[0093] illustrate:

[0094] Weighting coefficients (satisfying) (The default values ​​are set to 0.3, 0.4, and 0.3 respectively).

[0095] (Novelty Score): Calculated by the feature similarity between the test case and historical test cases; the lower the similarity, the higher the score.

[0096] (Adversarial scoring): Based on historical defect data, assess the probability that use cases will trigger system vulnerabilities;

[0097] (Requirement Matching Score): Calculated based on the keyword matching degree between test cases and test requirement documents.

[0098] The adversarial training process is as follows:

[0099] 1. Training core logic

[0100] The generator (G) and discriminator (D) are trained alternately, and the two models are co-optimized through a min-max game to ultimately reach a Nash equilibrium.

[0101] Generator goal: To learn to generate test cases that meet testing requirements, are highly adversarial and novel, making it difficult for the discriminator to accurately distinguish the quality of its generated test cases, that is, to minimize the probability of the discriminator judging the generated test cases as "inferior".

[0102] The discriminator's goal is to improve the ability to distinguish between real high-quality use cases and generated use cases, and to accurately identify the quality level of generated use cases, that is, to maximize the probability of judging high-quality use cases as "high-quality" and the probability of judging low-quality generated use cases as "low-quality".

[0103] 2. Core Formula of Minimax Game

[0104] The objective function for adversarial training is a minimax value function, as shown in the following formula:

[0105]

[0106] illustrate:

[0107] This represents the distribution of real, high-quality test cases. Represents real, high-quality use cases;

[0108] The distribution of input noise (or interface feature vector) to the generator. A random vector representing the input generator;

[0109] Test cases output by the generator;

[0110] For the discriminator to identify real high-quality use cases The probability of being judged as "high quality" (within the range [0,1]). For the discriminator to generate test cases The probability of being judged as "high-quality";

[0111] The generator minimizes The discriminator misclassifies generated test cases as high-quality by maximizing... Improve the accuracy of differentiation.

[0112] 3. Alternating Training Process

[0113] 1. Training the discriminator: With the generator parameters fixed, real high-quality test cases (from a historical high-quality test case library) and the test cases currently generated by the generator are used as training samples and input into the discriminator for training. The discriminator parameters are then optimized to maximize the objective function. ;

[0114] 2. Generator Training: With the discriminator parameters fixed, random noise (or interface feature vectors) is input into the generator. The discriminator's judgment of the generated test cases is used as feedback to optimize the generator parameters to minimize the objective function. ;

[0115] 3. Iterative convergence: Repeat steps 1-2 above. After each iteration, calculate the discrimination accuracy of the discriminator (the recognition rate of real high-quality use cases and the recognition rate of low-quality generated use cases). When the accuracy tends to stabilize (fluctuation less than 5%) or reaches the preset number of iterations (e.g., 100 rounds), stop training.

[0116] Step 4: Execute the generated test cases and record the execution log and intermediate results;

[0117] In step 4, the present invention completes the execution process through the test case execution module:

[0118] 1. Core logic for multi-system adaptation

[0119] By constructing a system type feature library, feature modeling is performed on the systems under test, such as web applications, mobile applications, and API interfaces, to achieve dynamic adaptation of test cases. The core idea is to extract features such as interaction protocols, data formats, and execution environments from each system, and then determine the test case adaptation strategy through feature matching algorithms to ensure that the execution module is compatible with the testing requirements of different systems.

[0120] 2. Core formulas and algorithms of the execution module

[0121] (1) System type feature matching algorithm

[0122] The cosine similarity is used to calculate the matching degree between the features of the test case to be executed and the features of the target system, as shown in the following formula:

[0123]

[0124] illustrate:

[0125] Feature vectors (dimensions) of test cases This includes features such as use case type, interaction protocol, and parameter format;

[0126] The feature vector (dimension) of the target system under test Web application features such as "HTTP protocol, form parameters", mobile features such as "TCP / UDP protocol, gesture parameters", and API interface features such as "RESTful protocol, JSON parameters";

[0127] When the similarity is ≥0.8, it is determined to be a match and is executed directly; when the similarity is <0.8, the use case adaptation conversion logic is triggered.

[0128] (2) Calculation of test case adaptation and conversion cost

[0129] For use cases that are not directly compatible, the optimal adaptation scheme is selected by minimizing the transformation cost, as shown in the following formula:

[0130]

[0131] illustrate:

[0132] To adapt to conversion schemes (such as parameter format conversion, protocol conversion);

[0133] For the first in the plan Each conversion step (such as "XML→JSON" or "HTTP→HTTPS");

[0134] For the first The conversion cost of each step (preset constant, such as format conversion cost = 2, protocol conversion cost = 3);

[0135] The weights for the transformation steps (set according to the transformation complexity, ranging from [0,1], satisfying...) );

[0136] choose The minimum solution is to perform the adaptation and conversion.

[0137] (3) Execution result verification algorithm (based on threshold judgment)

[0138] The core formula for verifying the consistency of execution results across different systems is as follows:

[0139]

[0140] illustrate:

[0141] This refers to the actual execution result (such as response text or UI element values). This is the expected result;

[0142] For the degree of difference in results, edit distance (Levenshtein distance) was used to calculate the text type, and absolute error was used to calculate the numerical type;

[0143] Set a preset error threshold (adjust according to system type, such as API interface). Web interface text );

[0144] = indicates that the execution result is valid, =0 indicates that it is invalid (there is a defect).

[0145] (4) Dynamic scheduling algorithm (based on priority)

[0146] The score output by the discriminator is used as the core basis for test case priority. A maximum priority strategy is adopted to achieve dynamic scheduling, ensuring that high-value test cases are executed first. The core formula and logic are as follows:

[0147] 1. Priority mapping formula:

[0148]

[0149] illustrate: For test cases priority, The final score (1-10 points) output by the discriminator is positively correlated with the score output by the discriminator. Directly mapping scores to priorities simplifies scheduling logic and ensures a strong correlation between priorities and test case quality.

[0150] 2. Execute the sorting formula:

[0151]

[0152] illustrate:

[0153] This is the set of test cases to be executed.

[0154] For sorting functions, based on priority Using the sort key, sorting in descending order yields the final execution order. ;

[0155] If there are use cases with the same score, a secondary ranking rule can be added, with the following formula: ,in The value represents the time the test case was generated. A negative sign indicates that the earlier the generation time, the higher the priority (first generated, first executed). It can also be adjusted to the system type weight sort according to business needs (e.g., mobile test case weight coefficient 1.2, Web test case weight coefficient 1.0).

[0156] 3. Dynamically adjust formulas (new use case scenarios):

[0157] When a new test case is added during execution, the execution queue is updated and reordered in real time, as shown in the following formula:

[0158]

[0159] illustrate: This is the queue of currently unexecuted test cases. For newly added test cases, union operation and reordering are used to ensure that the newly added high-priority test cases can be inserted into the corresponding positions for priority execution, thus ensuring the real-time performance and rationality of scheduling.

[0160] (5) Algorithm for real-time acquisition and processing of system status data

[0161] The system collects core status data (memory usage, response time, error logs) of the tested system in real time during execution. Quantitative calculations and anomaly detection provide a basis for test execution optimization and model feedback. The core formulas and algorithms are as follows:

[0162] The core status metrics include: memory usage, test case response time, and error log density.

[0163] To avoid the impact of instantaneous data fluctuations on judgment, a sliding window smoothing process is applied to the continuously collected status data, as shown in the following formula:

[0164]

[0165] illustrate: The index value after smoothing at time t. Set to the sliding window size (default value is 5, which means taking the current and the previous 4 time values). This represents the original collected value at time ti, improving data stability.

[0166] System anomaly detection algorithm (3σ principle):

[0167] Based on the distribution of historical state data, the 3σ principle is used to detect whether real-time state data is abnormal, as shown in the following formula:

[0168]

[0169] illustrate:

[0170] The value is the real-time smoothed state indicator value;

[0171] This is the average value of this indicator across similar historical test scenarios. The historical standard deviation;

[0172] = indicates an abnormal system state, requiring an alert to be triggered (such as pausing the execution of the current test case or adjusting the priority of subsequent test cases), while =0 indicates a normal state.

[0173] Formula for linking status data with test feedback:

[0174] Incorporate abnormal system state results into the test case quality assessment to correct the discriminator score, as shown in the following formula:

[0175]

[0176] illustrate: The final score for the revised use cases, This is the abnormal weighting coefficient (default setting is 0.2). If the execution of a test case triggers a system abnormality, its final score will increase by 20%, and it will be given priority in subsequent review and analysis.

[0177] (6) Multi-system adaptation execution process (including dynamic scheduling and status acquisition)

[0178] 1. Extract features from the target system under test and construct feature vectors. ;

[0179] 2. Calculate the feature vector of the test cases to be executed. and Cosine similarity is used to determine fit;

[0180] 3. For adapted (or transformed and adapted) use cases, calculate priorities and sort them using a dynamic scheduling algorithm to generate an initial execution order. ;

[0181] 4. Press Execute the test cases sequentially, and simultaneously start collecting system status data (memory, response time, error log) during the execution process.

[0182] 5. Perform sliding window smoothing on the collected status data and detect system anomalies using the 3σ principle;

[0183] 6. If new test cases are added during execution, the execution order is updated in real time by dynamically adjusting the formulas;

[0184] 7. After executing each test case, adjust the final score of the test case based on the state anomaly detection results, and then determine the validity of the execution result by using a threshold.

[0185] 8. Output execution results, system status data, corrected scores, and execution order logs to provide feedback for model optimization.

[0186] Step 5: Evaluate the execution logs and intermediate results until the preset test termination conditions are met.

[0187] In step 5, the present invention can use a test process monitoring and analysis module to perform the following functions:

[0188] 1. Use natural language processing technology to parse error logs and extract key error information;

[0189] 2. Utilize deep learning models (such as autoencoders) to identify abnormal execution patterns and assist in defect localization;

[0190] 3. Evaluate the adequacy of testing based on multiple dimensions such as code coverage, requirement coverage, and risk coverage.

[0191] Phase 1: Data Preprocessing (Format Standardization, Noise Removal)

[0192] (1) Log formatting: Use regular expressions to extract core fields (timestamp, level, module, error description) and integrate logs from multiple sources;

[0193] (2) Text cleaning: Remove special characters and redundant information (such as UUID and IP), and filter duplicate logs;

[0194] (3) Standardization: unify uppercase and lowercase letters, normalize technical terms (e.g., “OOM” → “memory overflow”).

[0195] Phase 2: Text Feature Engineering (Text → Computable Vectors)

[0196] Word Segmentation: Use Jieba for Chinese and NLTK for English to split semantic units;

[0197] Stop Word Removal: Filter out meaningless words (such as "的", "the", "日志");

[0198] Feature Encoding: Convert text into numerical vectors. Core methods:

[0199] TF-IDF: Highlight core keywords (such as "空指针", "连接超时"). Core formula:

[0200] (Term Frequency, where t is the term and d is a single log)

[0201] (Inverse Document Frequency, where D is the log set and DF(t) is the number of logs containing t)

[0202] (Final feature value);

[0203] BERT Word Embedding: Capture complex associations, adapt to ambiguous scenarios. The core is the self-attention mechanism formula:

[0204] (Q query matrix, K key matrix, V value matrix, where d_k is the dimension of the key vector).

[0205] Phase 3: Key Information Extraction (Combination of Rules + Models)

[0206] 1. Extract target fields

[0207] field name Example value Error Type Null pointer exception, memory overflow Affected modules Login module, database connection pool Error level Fatal, Warning Triggering conditions User input is empty, network interrupted.

[0208] 2. Core solutions

[0209] Rule-based: Regular expressions + keyword dictionaries, quickly implement known error types;

[0210] Machine learning: Naive Bayes (error classification). Core formula: (C_k is the error category, X is the text feature); BiLSTM-CRF (named entity recognition). Core objective function: (A transition probability, B emission probability, Z normalization factor), adapt to variable logs;

[0211] Deep learning: BERT + classification / NER head, high-precision processing of complex semantics.

[0212] Phase 4: Result Optimization and Output

[0213] Result validation: Rule checks for abnormal results (such as "fatal error with no trigger condition"), followed by manual review; supplementary similar error clustering for deduplication, using the K-Means algorithm as the core algorithm, with the objective function:

[0214]

[0215] Where C_k is the cluster, μ_k is the cluster center, and x is the text vector;

[0216] Anomaly Execution Pattern Recognition (Autoencoder-Assisted Defect Localization): This algorithm learns the execution patterns of normal logs using an autoencoder, identifies anomalous logs that deviate from the pattern, and quickly associates defect types. Core Algorithm Process: Data Input: Preprocessed log text vectors (such as BERT embedding vectors) are used as input to the autoencoder; Model Training: The autoencoder is trained using normal execution logs to minimize reconstruction error and learn the feature representation of normal execution patterns; Anomaly Detection: The reconstruction error of the log to be detected is calculated; if the error exceeds a threshold, it is determined to be an anomalous execution pattern; Defect Association: The anomalous pattern is matched with a historical defect database to output candidate defect types, assisting in localization. Core Formula:

[0217] 1. Reconstruction error (mean squared error, MSE): (x is the input log vector, (The output is a model reconstruction, where n is the vector dimension).

[0218] 2. Anomaly detection threshold:

[0219] ( This represents the set of reconstruction errors from normal logs, where std is the standard deviation and α is the adjustment coefficient, typically ranging from 1.5 to 3.

[0220] Structured output: Converted to JSON / Excel / database format for integration with testing platforms or monitoring systems.

[0221] Phase 5: Test Adequacy Assessment (Multi-dimensional Coverage Metrics)

[0222] By using three core metrics—code coverage, requirement coverage, and risk coverage—we can quantitatively evaluate the extent to which test cases cover the system, determine whether the testing is sufficient, and help optimize testing strategies to improve defect detection capabilities.

[0223] 1. Code coverage (measures the degree to which test cases cover the code logic)

[0224] Core logic: Calculate the proportion of code elements (statements, branches, functions, etc.) executed by the test cases out of the total code elements. Commonly used core metrics include statement coverage and branch coverage.

[0225] Algorithm process: 1. Insert probes: Insert monitoring probes into the code using testing tools to record the code execution trajectory; 2. Execute tests: Run test case sets and collect execution data fed back by the probes; 3. Calculate ratio: Calculate the ratio of the number of executed code elements to the total number.

[0226] Core formula:

[0227] Statement coverage = (Number of executed statements / Total number of statements) × 100%, Formula: ( The number of statements executed. (Total number of statements in the code).

[0228] Branch coverage = (Number of branches executed / Total number of branches) × 100%, Formula: ( The number of branches executed, such as the different branches in if-else and switch statements. (Total number of all branches in the code).

[0229] 2. Requirements Coverage (measures the extent to which test cases cover the requirements)

[0230] Core logic: Based on the functional points / requirements broken down from the requirements document, calculate the proportion of required items covered by the test cases to the total required items, ensuring that no requirements are omitted.

[0231] Algorithm process: 1. Requirement decomposition: Decompose the original requirements into the smallest verifiable requirement units (such as functional points, user scenarios); 2. Test case mapping: Associate corresponding test cases with each requirement unit; 3. Calculate the ratio: Calculate the ratio of the number of requirement units with corresponding test cases to the total number of requirement units.

[0232] Core formula: ( The number of requirement units covered by the test cases. (This represents the total number of required units after decomposition).

[0233] 3. Risk coverage (measures the extent to which test cases cover potential risks)

[0234] Core logic: First, identify potential system risks (such as performance risks, security risks, and compatibility risks), then calculate the proportion of risk items covered by test cases designed for these risks, and focus on testing high-risk modules.

[0235] Algorithm process: 1. Risk identification: Identify risk items and label their risk levels through FMEA (Failure Mode and Effects Analysis), historical defect analysis, etc.; 2. Risk test case design: Design targeted test cases for each risk item; 3. Calculate the ratio: Calculate the ratio of the number of risk items covered by the test cases to the total number of risk items.

[0236] Core formula: ( The number of risk items covered by the test cases. (Total number of identified risk items); Advanced: Weighted risk coverage ratio (highlighting high-risk items), formula: ( , where is the weight of the i-th type of risk, with 1.0 for high risk, 0.6 for medium risk, and 0.3 for low risk.

[0237] Result validation: Rule checks for abnormal results (such as "fatal error with no trigger condition"), followed by manual review; supplementary similar error clustering for deduplication, using the K-Means algorithm as the core algorithm, with the objective function:

[0238]

[0239] Where C_k is the cluster, μ_k is the cluster center, and x is the text vector;

[0240] Abnormal execution pattern recognition (autoencoder-assisted defect localization): By learning the execution patterns of normal logs through an autoencoder, abnormal logs that deviate from the pattern can be identified and defect types can be quickly associated.

[0241] Core algorithm process:

[0242] Data input: Use the preprocessed log text vector (such as BERT embedding vector) as input to the autoencoder;

[0243] Model training: Train the autoencoder using normal execution logs, minimize reconstruction error, and learn feature representations of normal execution patterns;

[0244] Anomaly detection: Calculate the reconstruction error of the log to be detected. If the error exceeds the threshold, it is determined to be an abnormal execution mode.

[0245] Defect Association: Match abnormal patterns with the historical defect database to output candidate defect types and assist in localization.

[0246] Structured output: Converted to JSON / Excel / database format for integration with testing platforms or monitoring systems.

[0247] To verify the technical advantages of this invention, a comparative experiment was designed, selecting three typical tested systems (Web applications, mobile applications, and API interfaces) for comparison with traditional manual testing, rule-based automated testing, and ordinary machine learning (non-adversarial) testing. The technical effect was quantified through multi-dimensional indicators.

[0248] 1. Experimental Environment

[0249] Environment type Specific configuration Hardware environment CPU: Intel Xeon 8375C (32 cores); Memory: 128GB Software environment Operating system: Ubuntu 20.04; System under test 1. Web Application: E-commerce backend management system (150,000 lines of code, 20 functional modules); 2. Mobile Application: Social App (200,000 lines of code, 25 functional modules); 3. API Interface: Payment service interface (50 interfaces, moderate parameter combination complexity).

[0250] 2. Control group design

[0251] Control group 1: Traditional manual testing (3 senior test engineers design and execute test cases according to the requirements document);

[0252] Control group 2: Rule-based automated testing (generating test cases through a pre-defined parameter rule base, such as equivalence class partitioning and boundary value analysis);

[0253] Control group 3: Standard machine learning test (using LSTM model to generate test cases, without adversarial training).

[0254] Experimental group: Testing of the AI ​​adversarial model of this invention (adversarial training of generator + discriminator, fully AI-driven process).

[0255] 3. Core Experimental Data

[0256] Test metrics Control group 1 (artificial) Control Group 2 (Automated Rules) Control group 3 (normal ML) Experimental group (AI adversarial model) Defect detection capability Total number of defects found (3 systems total) 42 51 63 89 Number of hidden defects discovered (not detected by traditional methods) 5 8 15 37 Defect detection rate (number of defects found / total number of defects) 65.6% 79.7% 98.4% 99.0% Test efficiency Time to generate a single test case (seconds) 120 30 15 8 Batch generation of 1000 test cases (time in minutes) 200 50 25 13 Total testing period (days) 15 8 5 3 Test coverage Code statement coverage 72.3% 78.5% 85.1% 94.7% Code branch coverage 65.8% 70.2% 78.3% 89.5% Demand coverage 80.0% 85.0% 90.0% 98.0% Weighted risk coverage 68.5% 73.3% 82.1% 95.2% Cost-effectiveness Human workload (person-days) 45 12 8 3 The percentage reduction in testing costs (relative to labor costs) - 73.3% 82.2% 93.3%

[0257] 4. Data Explanation and Technical Effect Analysis

[0258] The defect detection capability was significantly improved: the total number of defects found in the experimental group (89) was 111.9% higher than that in control group 1 and 41.3% higher than that in control group 3; in particular, the number of hidden defects found (37) was 146.7% higher than that in control group 3. The core reason is that the test cases generated by the AI ​​adversarial model are highly adversarial and can accurately hit the system's weak points (such as boundary parameter combinations and abnormal scene triggers), which solves the problem that traditional methods are difficult to find hidden defects.

[0259] Test efficiency has been significantly improved: the generation time for a single test case is only 8 seconds, which is 73.3% shorter than control group 2 and 46.7% shorter than control group 3; the total test cycle has been reduced from 15 days for manual testing to 3 days, with an efficiency improvement of 80%, demonstrating the advantages of full-process automation and AI-driven approach, and significantly reducing the cost of manual intervention.

[0260] Test coverage was fully achieved: code statement coverage (94.7%) and branch coverage (89.5%) were significantly higher than the comparison group, while requirement coverage (98.0%) and weighted risk coverage (95.2%) were close to 100%, indicating that the deep integration of AI technology into the entire testing process effectively avoided requirement omissions and risk blind spots.

[0261] Outstanding cost-effectiveness: The manual workload is only 3 days, which is 62.5% less than that of ordinary machine learning testing and 93.3% less than that of manual testing. It can significantly reduce the labor and time costs of enterprises in the testing of complex software systems.

[0262] 5. Supplementary Model Stability Experiments

[0263] The experimental group model was repeatedly tested for 10 rounds, and the fluctuations of the core indicators were statistically analyzed.

[0264] Defect detection rate fluctuation range: 98.2%-99.0% (fluctuation ≤0.8%);

[0265] Test case generation time fluctuation range: 7.8-8.2 seconds (fluctuation ≤5%);

[0266] Code coverage fluctuation range: 94.0%-94.7% (fluctuation ≤0.7%);

[0267] According to specific embodiments of the present invention, the following beneficial effects are disclosed:

[0268] 1. Test cases generated using AI adversarial models are more adversarial and targeted, and can effectively discover hidden defects that are difficult to detect using traditional methods.

[0269] 2. Achieve full automation and intelligence in the process of test case generation, execution, and analysis, significantly improving testing efficiency and reducing labor costs;

[0270] 3. Continuously optimize the AI ​​model through a dynamic feedback mechanism to enable the testing process to be adaptive and learnable, thereby continuously improving the quality of testing;

[0271] 4. Achieve deep integration of AI technology into the entire testing process to ensure comprehensive and effective test coverage and provide reliable quality assurance for complex software systems.

[0272] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for generating intelligent test cases based on an AI adversarial model, characterized in that, include: Step 1: Construct an initial training set based on the information of the system under test, test requirements, and historical test data; Step 2: Generate test cases based on the initial training set using the generator of the AI ​​adversarial model; Step 3: Evaluate the quality of test cases using a discriminator, and iteratively optimize the generator based on the evaluation results; Step 4: Execute the generated test cases and record the execution log and intermediate results; Step 5: Evaluate the execution logs and intermediate results until the preset test termination conditions are met.

2. The intelligent test case generation method based on an AI adversarial model according to claim 1, characterized in that, In step 2, the numerical features in the initial training set are normalized, and the Word2Vec model is used to convert the text in the initial training set into vectors.

3. The intelligent test case generation method based on an AI adversarial model according to claim 2, characterized in that, In step 2, the generator is trained using a Transformer model, and the learning rate is optimized using the Adam optimizer during training. The optimization process of the Adam optimizer is as follows: in, , This represents the initial learning rate. This indicates the number of iterations in the current model training. The number of warm-up steps indicates the learning rate. This represents the actual learning rate at the current iteration step.

4. The intelligent test case generation method based on an AI adversarial model according to claim 3, characterized in that, In step 3, the discriminator is trained using a convolutional neural network or a gradient boosting model.

5. The intelligent test case generation method based on an AI adversarial model according to claim 4, characterized in that, In step 3, a weighted summation method is used to calculate the final evaluation result; the formula for calculating the evaluation result is as follows: in, Indicates the evaluation results. This represents the first weighting coefficient. The novelty score is calculated based on the feature similarity between the test cases and historical test cases. This represents the second weighting coefficient. The adversarial score is calculated based on historical defect data to assess the probability that test cases will trigger system vulnerabilities. This represents the third weighting coefficient. The requirement matching score is calculated by matching the keywords between the test cases and the test requirement document.

6. The intelligent test case generation method based on an AI adversarial model according to claim 5, characterized in that, In step 3, the generator and discriminator adopt an alternating training mode, achieving collaborative optimization through a minimax game; wherein, the objective function in the adversarial training process is the minimax value function: in, This represents the distribution of real, high-quality test cases. Represents real, high-quality use cases. The distribution of input noise to the generator, Indicates to Expectation calculation, Represents random variables Follow the distribution of real high-quality use cases , This indicates that the generator is input with a random vector. Expectation calculation, Represents random variables Follow the input noise distribution , A random vector representing the input generator. The test cases output by the generator. For the discriminator to identify real high-quality use cases The probability of being judged as "high-quality". For the discriminator to generate test cases The probability of being judged as "high-quality".

7. The intelligent test case generation method based on an AI adversarial model according to claim 6, characterized in that, In step 4, the cosine similarity is used to calculate the matching degree between the features of the test cases to be executed and the features of the target system. When the matching degree is ≥0.8, it is determined to be a match and is executed directly. When the matching degree is less than 0.8, the optimal adaptation scheme is selected by minimizing the transformation cost.

8. The intelligent test case generation method based on an AI adversarial model according to claim 7, characterized in that, The evaluation results output by the discriminator are used as the basis for test case priority, and the maximum priority priority strategy is adopted to realize the dynamic scheduling of test cases.

9. An electronic device comprising a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the transceiver, the memory, and the processor are connected via the bus, characterized in that, When the computer program is executed by the processor, it implements the steps in the intelligent test case generation method based on an AI adversarial model as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps in the intelligent test case generation method based on an AI adversarial model as described in any one of claims 1-8.

Images