AIGC test case adaptive generation system based on risk feedback

By using an adaptive generation system based on risk feedback to dynamically adjust the mutation operator weight matrix, the problems of low efficiency and poor adaptability of the AIGC test case generation system are solved, achieving efficient and dynamic test resource optimization and risk coverage.

CN121326709APending Publication Date: 2026-01-13ANHUI GAOSHAN TECH CO LTD
View PDF 0 Cites 4 Cited by

Patent Information

Application Number
CN202511601472.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-11-04
Publication Date
2026-01-13

AI Technical Summary

Technical Problem

Existing AIGC test case generation systems are inefficient, struggle to adapt to environmental changes automatically, fail to focus on weak areas of the model, and result in significant waste of test resources.

Method used

An adaptive generation system based on risk feedback is adopted. Through a semantic mutation generation module, a risk assessment and quantification module, a risk feedback processing module, and an adaptive strategy adjustment module, the mutation operator weight matrix is ​​dynamically adjusted to achieve intelligent optimization and adaptive generation of test cases.

Benefits of technology

Significantly improves testing efficiency, increases the detection rate of high-value defects by more than 60%, reduces testing time by 40%, dynamically adapts to model updates and scenario changes, eliminates the need for manual redesign of solutions, reduces testing costs by 40%, and improves the comprehensiveness of risk coverage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121326709A_ABST
    Figure CN121326709A_ABST
Patent Text Reader

Abstract

The invention relates to artificial intelligence generation content AIGC, in particular to an AIGC test case self-adaptive generation system based on risk feedback, which comprises a semantic variation generation module for selecting a variation operator from a variation operator library according to a self-adaptive strategy, performing variation processing on a seed case provided by a basic test case library by using the variation operator, and generating a semantic variation result; generating a new test case; the AIGC model takes the test case as model input and performs model output to the risk assessment and quantification module through an AIGC model interface; the risk assessment and quantification module is used for performing multi-dimensional risk assessment on a model output result by utilizing the risk detection model library, calculating a risk level and generating a test report; the risk feedback processing module is used for storing the high-risk test case, performing feature extraction and converting a risk assessment result into an operable risk feedback signal according to extracted features; according to the technical scheme provided by the invention, the defects of low efficiency and difficulty in automatically adapting to environment change can be overcome.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to artificial intelligence generated content (AIGC), and in particular to an AIGC test case adaptive generation system based on risk feedback. BACKGROUND

[0002] With the wide application of large language models such as ChatGPT, Claude, GPT-4, AIGC technology has become an important development direction in the field of artificial intelligence. However, AIGC models may produce factual errors, logical fallacies, harmful content, biased expressions, and other risks when generating content, which seriously affects their reliability in critical application scenarios. Therefore, establishing an effective AIGC quality assurance and testing mechanism has become a pressing technical problem.

[0003] Currently, the industry mainly uses AIGC test case generation methods based on templates and rules, and the typical implementation process includes: 1) Template construction phase: manually design prompt templates covering different scenarios, such as "Please describe the [entity]'s [attribute]", "Analyze the [aspect] of [event]" and the like; 2) Batch generation phase: use template instantiation and random semantic variation techniques to generate a large-scale test case set, and semantic variation mainly includes synonym replacement, sentence reconstruction, entity replacement, and other operations; 3) Risk screening phase: use pre-defined risk detection rules or classification models to evaluate the risk of AIGC output content and screen out high-risk samples; 4) Manual review phase: manually review the high-risk samples screened out to confirm the real model defects.

[0004] The core devices of the above scheme include: template library management module, semantic variation engine, AIGC calling interface, risk detection classifier, and result storage module, and the data flow between each functional module is transferred in a one-way pipeline, lacking feedback adjustment mechanism.

[0005] Since the existing technology uses a static template-driven generation strategy, the test cases are uniformly distributed in the input space, and cannot focus on the actual weak areas of the AIGC model. Therefore, a large amount of computing resources is consumed on low-value test cases, resulting in low overall efficiency.

[0006] Since the risk assessment and test case generation are independent, the high-value information found in the risk assessment process (such as which type of input is more likely to trigger defects, which semantic features are strongly related to risks, etc.) cannot be passed to the test case generation. Therefore, the subsequent generated test cases still repeat the previous inefficient mode, and the intelligent optimization of the test strategy cannot be achieved.

[0007] Since the test strategy parameters remain fixed throughout the process, the original strategy may no longer be applicable when the AIGC model version is updated or the application scenario changes. Therefore, the existing system cannot automatically adapt to environmental changes, and manual redesign of the test scheme is required, which has high maintenance costs and poor timeliness. SUMMARY

[0008] (I) Technical problems to be solved In view of the above-mentioned defects of the prior art, the present application provides an AIGC test case adaptive generation system based on risk feedback, which can effectively overcome the defects of low efficiency and difficulty in automatically adapting to environmental changes of the prior art.

[0009] (II) Technical solutions To achieve the above object, the present application is realized by the following technical solutions: An AIGC test case adaptive generation system based on risk feedback comprises the following functional modules: A semantic variation generation module selects a variation operator from a variation operator library according to an adaptive strategy, uses the variation operator to perform variation processing on a seed case provided by a basic test case library, and generates a new test case; An AIGC model takes the test case as a model input, and outputs the model to a risk assessment and quantification module through an AIGC model interface; A risk assessment and quantification module performs multi-dimensional risk assessment on the model output result using a risk detection model library, calculates a risk level, and generates a test report; A risk feedback processing module stores high-risk test cases, extracts features, and converts the risk assessment result into an operable risk feedback signal according to the extracted features; An adaptive strategy adjustment module dynamically adjusts the variation operator weight matrix according to the risk feedback signal.

[0010] Preferably, the semantic variation generation module selects a variation operator from a variation operator library according to an adaptive strategy, uses the variation operator to perform variation processing on a seed case provided by a basic test case library, and generates a new test case, comprising: S11, initializing the variation operator weight matrix: Maintain a dynamic variation operator weight matrix W, where the elements W i,j is the weight coefficient of the jth variation operator corresponding to the ith risk, and in the initial state, all weight coefficients are set to be uniformly distributed: N is the total number of variation operators; S12, selecting a variation operator: For a given basic test case library, according to the historical risk label of the provided seed case, select the corresponding weight coefficient from the mutation operator weight matrix W to form the mutation operator; S13, perform multi-level semantic mutation: Perform three-level semantic mutation including lexical layer mutation, syntax layer mutation and semantic layer mutation on the seed case by using the mutation operator: Lexical layer mutation includes synonym replacement, antonym confrontation and professional term transformation; Syntax layer mutation includes sentence reconstruction, word order adjustment and clause nesting; Semantic layer mutation includes logical relationship rewriting, emotional tone adjustment and expression mode conversion; S14, mutation quality evaluation and filtering: Perform quality evaluation on the semantic mutation results to ensure that the mutated seed case has sufficient variation while maintaining the original semantics; S15, generate test case set: For each seed case, generate multiple mutated versions to form a test case set, and mark the mutation operator information of each mutated seed case to provide traceability basis for subsequent feedback processing.

[0011] Preferably, in S12, for a given basic test case library, according to the historical risk label of the provided seed case, select the corresponding weight coefficient from the mutation operator weight matrix W to form the mutation operator, including: The selection probability follows a weighted random sampling strategy to ensure that elements with high weight coefficients have a higher probability of being selected.

[0012] Preferably, in S14, the quality of the semantic mutation results is evaluated to ensure that the mutated seed case has sufficient variation while maintaining the original semantics, including: The quality of the semantic mutation results is evaluated by semantic similarity calculation and grammatical correctness check, and the semantic similarity threshold is set to 0.7-0.9 to ensure that the mutated seed case has sufficient variation while maintaining the original semantics.

[0013] Preferably, the risk assessment and quantification module utilizes a risk detection model library to perform multi-dimensional risk assessment on the model output results, calculates the risk level, and generates a test report, including: S21, multi-dimensional risk detection: Use multiple specialized risk detection models to detect different risk types to perform multi-dimensional risk detection on the model output results: The factual error detection model verifies the factual accuracy of the model output results based on the knowledge graph; A logical consistency detection model is used to analyze the logical structure and reasoning chain of the model's output results. Harmful content detection model; identifies violence, discrimination, and inappropriate content in the model's output. A bias detection model that detects potential biases in the gender, race, and region dimensions of the model output. S22. Quantitative Calculation of Risk Level: The normalized risk scores output by each risk detection model are weighted and fused to calculate the comprehensive risk level, Risk. total ; S23. Generate a test report.

[0014] Preferably, in step S22, the normalized risk scores output by each risk detection model are weighted and fused to calculate the comprehensive risk level, Risk. total ,include: The overall risk level (Risk) is calculated using the following formula. total : ; Among them, Risk m Let m be the normalized risk score for the m-th dimension. For the weighting coefficient of the m-th dimension, M represents the number of risk detection dimensions.

[0015] Preferably, the risk feedback processing module stores high-risk test cases and extracts features, transforming the risk assessment results into actionable risk feedback signals based on the extracted features, including: S31. Risk Feature Extraction: Based on the comprehensive risk level Risk total Store high-risk test cases, and extract risk features from these high-risk test cases: Input features include triggering words, syntactic structures, and semantic patterns; Output characteristics include risk type, risk location, and risk severity; Association features include causal relationship patterns between inputs and outputs; S32. Risk Vector Construction: Risk features are encoded into risk vectors, where each dimension corresponds to a risk feature, and the risk vectors are sent to the adaptive strategy adjustment module as actionable risk feedback signals.

[0016] Preferably, the adaptive strategy adjustment module dynamically adjusts the mutation operator weight matrix based on the risk feedback signal, including: S41. Define the state space: The risk assessment result is defined as a quadruple S=(H,w,C,T), where H is the feature distribution matrix of historical high-risk test cases, w is the current mutation operator weight matrix, C is the test case coverage statistics of completed tests, and T is the remaining test resources. S42. Designing the motion space: The action space is designed as an adjustment operation of the mutation operator weight matrix W: Weight enhancement increases the selection weight of specific weight coefficients. Weight reduction, which lowers the selection weight of a specific weight coefficient; Strategy switching allows you to switch between different adjustment operations. Parameter fine-tuning involves adjusting hyperparameters, including mutation intensity and mutation number. S43. Design the reward function: The reward function R is designed by comprehensively considering multiple optimization objectives; S44, Strategy Update Mechanism: Deep Q-Network (DQN) is used for policy learning and updating: Maintain an experience replay buffer and periodically sample training data from the buffer to update the network parameters of the Deep Q-Network (DQN). - Greedy strategies balance exploration and utilization, stabilizing the training process of the target network; S45, Weight Matrix Update: Update the mutation operator weight matrix W based on the learned optimal strategy.

[0017] Preferably, in S43, the reward function R is designed by comprehensively considering multiple optimization objectives, including: The reward function R is expressed as follows: ; Among them, R discovery As a reward for newly discovered high-risk test cases, R efficiency As a reward for improved testing efficiency, R cost Penalties for resource consumption , , All are trade-off coefficients.

[0018] Preferably, in step S45, the mutation operator weight matrix W is updated according to the learned optimal strategy, including: The updated mutation operator weight matrix W is expressed as follows: ; in, , These are the updated and original weight coefficients for the j-th mutation operator corresponding to the i-th risk type, respectively. The adjustment amount for the weight coefficient of the j-th mutation operator corresponding to the i-th risk is determined by the output of the deep Q-network (DQN). This is the learning rate.

[0019] (III) Beneficial Effects Compared with existing technologies, the AIGC test case adaptive generation system based on risk feedback provided by this invention has the following beneficial effects: 1) Significantly improved testing efficiency: By establishing a risk feedback-driven adaptive generation mechanism, this invention can intelligently focus testing resources on the actual weak links of the AIGC model, avoiding the blind exploration of low-value areas by existing methods. Experiments show that, with the same input of testing resources, the high-value defect discovery rate is increased by more than 60%, and the testing time is shortened by about 40%. 2) Strong dynamic adaptability: The closed-loop feedback mechanism established by this invention enables the system to automatically adapt to the version updates of the AIGC model and changes in application scenarios. When the risk points of the model drift, the system can automatically adjust the test strategy within 3 to 5 iterations without the need for manual redesign of the test plan. 3) Effective control of testing costs: Through intelligent test case generation and accurate risk positioning, this invention effectively reduces invalid API calls and manual review workload. Compared with existing methods, the overall testing cost is reduced by more than 40%, while the testing quality is significantly improved. 4) Comprehensive risk coverage: The multi-dimensional risk detection system established by this invention can comprehensively cover all kinds of potential risks in AIGC applications, including factual errors, logical fallacies, harmful content, biased expressions, etc. Through a quantitative fusion mechanism, the system can accurately identify compound risks and hidden risks. 5) Technical scalability: The present invention adopts a modular architecture design, and the functional modules interact with each other through standardized interfaces. When it is necessary to support new AIGC models or risk types, only the corresponding modules need to be updated without reconstructing the entire system, which has good technical scalability and maintainability. 6) Practicality and operability: Based on mature machine learning and software engineering technologies, this invention has a high degree of engineering feasibility. The system provides complete API interfaces and configuration management functions, which can be quickly integrated into existing AI development and testing processes, and has strong practical value. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0021] Figure 1 This is a schematic diagram of the system of the present invention; Figure 2 This is a schematic diagram of the complete system architecture of the present invention; Figure 3 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0023] The following examples illustrate the specific functional modules of the AIGC test case adaptive generation system based on risk feedback provided by this invention (e.g., Figure 1 (As shown). The system functional modules include: The semantic mutation generation module selects mutation operators from the mutation operator library according to an adaptive strategy, and uses the mutation operators to mutate the seed test cases provided by the basic test case library to generate new test cases. The AIGC model takes test cases as input and outputs the model to the risk assessment and quantification module through the AIGC model interface. The risk assessment and quantification module uses a risk detection model library to perform multi-dimensional risk assessments on the model output results, calculates the risk level, and generates a test report. The risk feedback processing module stores high-risk test cases and extracts features, transforming the risk assessment results into actionable risk feedback signals based on the extracted features. The adaptive strategy adjustment module dynamically adjusts the mutation operator weight matrix based on risk feedback signals.

[0024] I. Semantic Variation Generation Module The semantic mutation generation module selects mutation operators from the mutation operator library according to an adaptive strategy, and uses these operators to mutate seed test cases provided by the basic test case library to generate new test cases, including: S11. Initialize the mutation operator weight matrix: Maintain a dynamic mutation operator weight matrix W, where elements W i,j Let be the weight coefficients for the j-th mutation operator corresponding to the i-th risk. Initially, all weight coefficients are set to a uniform distribution. N is the total number of mutation operators; S12, Selecting the mutation operator: For a given base test case library, based on the historical risk labels of the provided seed test cases, select the corresponding weight coefficients from the mutation operator weight matrix W to form a mutation operator; S13, Perform multi-level semantic variation: The mutation operator is used to perform three-level semantic mutations on the seed use case, including lexical mutation, syntactic mutation, and semantic mutation: Lexical layer variation includes synonym substitution, antonym counteraction, and technical term transformation; Syntactic variations include sentence restructuring, word order adjustment, and clause nesting; Semantic layer variations include rewriting logical relationships, adjusting emotional tone, and transforming expression methods; S14. Variation Quality Assessment and Filtering: Perform a quality assessment on the semantic mutation results to ensure that the mutated seed use cases have sufficient variability while maintaining the original semantics. S15. Generate test case set: For each seed test case, multiple variant versions are generated to form a test case set. Each mutated seed test case is marked with the mutation operator information of its source, providing a traceability basis for subsequent feedback processing.

[0025] Specifically, in S12, for a given base test case library, based on the historical risk labels of the provided seed test cases, corresponding weight coefficients are selected from the mutation operator weight matrix W to form a mutation operator, including: The selection probability follows a weighted random sampling strategy to ensure that elements with high weight coefficients have a greater probability of being selected.

[0026] Specifically, S14 performs a quality assessment of the semantic mutation results to ensure that the mutated seed use cases have sufficient variability while maintaining the original semantics, including: The quality of semantic mutation results is evaluated by semantic similarity calculation and grammatical correctness check. The semantic similarity threshold is set at 0.7~0.9 to ensure that the mutated seed use cases have sufficient variability while maintaining the original semantics.

[0027] II. Risk Assessment and Quantification Module The risk assessment and quantification module utilizes a risk detection model library to perform multi-dimensional risk assessments on the model outputs, calculates the risk level, and generates a test report, including: S21. Multi-dimensional risk detection: By utilizing multiple specialized risk detection models that identify different risk types, multi-dimensional risk detection is performed on the model outputs: A factual error detection model is used to verify the factual accuracy of the model's output based on a knowledge graph. A logical consistency detection model is used to analyze the logical structure and reasoning chain of the model's output results. Harmful content detection model; identifies violence, discrimination, and inappropriate content in the model's output. A bias detection model that detects potential biases in the gender, race, and region dimensions of the model output. S22. Quantitative Calculation of Risk Level: The normalized risk scores output by each risk detection model are weighted and fused to calculate the comprehensive risk level, Risk. total ; S23. Generate a test report.

[0028] Specifically, in S22, the normalized risk scores output by each risk detection model are weighted and fused to calculate the comprehensive risk level, Risk. total ,include: The overall risk level (Risk) is calculated using the following formula. total : ; Among them, Risk m Let m be the normalized risk score for the m-th dimension. For the weighting coefficient of the m-th dimension, M represents the number of risk detection dimensions.

[0029] III. Risk Feedback Processing Module The risk feedback processing module stores high-risk test cases and extracts their features. Based on the extracted features, it transforms the risk assessment results into actionable risk feedback signals, including: S31. Risk Feature Extraction: Based on the comprehensive risk level Risk total Store high-risk test cases, and extract risk features from these high-risk test cases: Input features include triggering words, syntactic structures, and semantic patterns; Output characteristics include risk type, risk location, and risk severity; Association features include causal relationship patterns between inputs and outputs; S32. Risk Vector Construction: Risk features are encoded into risk vectors, where each dimension corresponds to a risk feature, and the risk vectors are sent to the adaptive strategy adjustment module as actionable risk feedback signals.

[0030] IV. Adaptive Strategy Adjustment Module The adaptive strategy adjustment module dynamically adjusts the mutation operator weight matrix based on risk feedback signals, including: S41. Define the state space: The risk assessment result is defined as a quadruple S=(H,w,C,T), where H is the feature distribution matrix of historical high-risk test cases, w is the current mutation operator weight matrix, C is the test case coverage statistics of completed tests, and T is the remaining test resources (such as time, number of API calls, etc.). S42. Designing the motion space: The action space is designed as an adjustment operation of the mutation operator weight matrix W: Weight enhancement increases the selection weight of specific weight coefficients. Weight reduction, which lowers the selection weight of a specific weight coefficient; Strategy switching allows you to switch between different adjustment operations. Parameter fine-tuning involves adjusting hyperparameters, including mutation intensity and mutation number. S43. Design the reward function: The reward function R is designed by comprehensively considering multiple optimization objectives; S44, Strategy Update Mechanism: Deep Q-Network (DQN) is used for policy learning and updating: Maintain an experience replay buffer and periodically sample training data from the buffer to update the network parameters of the Deep Q-Network (DQN). - Greedy strategies balance exploration and utilization, stabilizing the training process of the target network; S45, Weight Matrix Update: Update the mutation operator weight matrix W based on the learned optimal strategy.

[0031] Specifically, S43 comprehensively considers multiple optimization objectives in designing the reward function R, including: The reward function R is expressed as follows: ; Among them, R discovery As a reward for newly discovered high-risk test cases, R efficiency As a reward for improved testing efficiency, R cost Penalties for resource consumption , , All are trade-off coefficients.

[0032] Specifically, in S45, the mutation operator weight matrix W is updated based on the learned optimal policy, including: The updated mutation operator weight matrix W is expressed as follows: ; in, , These are the updated and original weight coefficients for the j-th mutation operator corresponding to the i-th risk type, respectively. The adjustment amount for the weight coefficient of the j-th mutation operator corresponding to the i-th risk is determined by the output of the deep Q-network (DQN). This is the learning rate.

[0033] Figure 2 This diagram illustrates the complete system architecture of the present invention, showcasing the entire technical implementation path. The system adopts a layered design, with each layer having clearly defined functional responsibilities. The key innovation lies in the design of the feedback layer, which transforms the output of the evaluation layer into actionable risk feedback signals, achieving a leap from "passive testing" to "active learning." The policy updater continuously optimizes the decision parameters of the policy controller through a reinforcement learning mechanism, forming an intelligent adaptive loop. This design ensures that the system can continuously improve itself during testing, gradually increasing testing accuracy and efficiency.

[0034] Figure 3 This is a flowchart illustrating the core innovative mechanism of this invention: dynamic closed-loop feedback. Unlike the unidirectional pipeline of existing technologies, this invention adds a crucial feedback loop after risk assessment. The system not only records high-risk test cases, but more importantly, extracts features from these cases, calculates risk feedback signals, and adjusts the mutation operator weight matrix in real time accordingly. This "test-learn-adjust" mechanism ensures that test resources automatically focus on the weak points of the AIGC model, achieving continuous optimization of test efficiency.

[0035] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An AIGC test case adaptive generation system based on risk feedback, characterized in that: Includes the following functional modules: The semantic mutation generation module selects mutation operators from the mutation operator library according to an adaptive strategy, and uses the mutation operators to mutate the seed test cases provided by the basic test case library to generate new test cases. The AIGC model takes test cases as input and outputs the model to the risk assessment and quantification module through the AIGC model interface. The risk assessment and quantification module uses a risk detection model library to perform multi-dimensional risk assessments on the model output results, calculates the risk level, and generates a test report. The risk feedback processing module stores high-risk test cases and extracts features, transforming the risk assessment results into actionable risk feedback signals based on the extracted features. The adaptive strategy adjustment module dynamically adjusts the mutation operator weight matrix based on risk feedback signals.

2. The AIGC test case adaptive generation system based on risk feedback according to claim 1, characterized in that: The semantic mutation generation module selects mutation operators from the mutation operator library according to an adaptive strategy, and uses the mutation operators to mutate seed test cases provided by the basic test case library to generate new test cases, including: S11. Initialize the mutation operator weight matrix: Maintain a dynamic mutation operator weight matrix Among the elements For the first Class of risk corresponds to the first The weight coefficients of the mutation operator are initially set to a uniform distribution: The total number of mutation operators; S12, Selecting the mutation operator: Given a basic test case library, based on the historical risk labels of the provided seed test cases, from the mutation operator weight matrix... Select the appropriate weight coefficients to form the mutation operator; S13, Perform multi-level semantic variation: The mutation operator is used to perform three-level semantic mutations on the seed use case, including lexical mutation, syntactic mutation, and semantic mutation: Lexical layer variation includes synonym substitution, antonym counteraction, and technical term transformation; Syntactic variations include sentence restructuring, word order adjustment, and clause nesting; Semantic layer variations include rewriting logical relationships, adjusting emotional tone, and transforming expression methods; S14. Variation Quality Assessment and Filtering: Perform a quality assessment on the semantic mutation results to ensure that the mutated seed use cases have sufficient variability while maintaining the original semantics. S15. Generate test case set: For each seed test case, multiple variant versions are generated to form a test case set. Each mutated seed test case is marked with the mutation operator information of its source, providing a traceability basis for subsequent feedback processing.

3. The AIGC test case adaptive generation system based on risk feedback according to claim 2, characterized in that: In S12, given a basic test case library, based on the historical risk labels of the provided seed test cases, the mutation operator weight matrix is ​​used to... W Select the appropriate weight coefficients to construct the mutation operator, including: The selection probability follows a weighted random sampling strategy to ensure that elements with high weight coefficients have a greater probability of being selected.

4. The AIGC test case adaptive generation system based on risk feedback according to claim 2, characterized in that: S14 performs a quality assessment of the semantic mutation results to ensure that the mutated seed use cases have sufficient variability while maintaining the original semantics, including: The quality of semantic mutation results is evaluated by semantic similarity calculation and grammatical correctness check. The semantic similarity threshold is set at 0.7~0.9 to ensure that the mutated seed use cases have sufficient variability while maintaining the original semantics.

5. The AIGC test case adaptive generation system based on risk feedback according to claim 1, characterized in that: The risk assessment and quantification module utilizes a risk detection model library to perform multi-dimensional risk assessments on the model output results, calculates the risk level, and generates a test report, including: S21. Multi-dimensional risk detection: By utilizing multiple specialized risk detection models that identify different risk types, multi-dimensional risk detection is performed on the model outputs: A factual error detection model is used to verify the factual accuracy of the model's output based on a knowledge graph. A logical consistency detection model is used to analyze the logical structure and reasoning chain of the model's output results. Harmful content detection model; identifies inappropriate content in the model's output. A bias detection model that detects potential biases in the model's output. S22. Quantitative Calculation of Risk Level: The normalized risk scores output by each risk detection model are weighted and fused to calculate the comprehensive risk level. Risk total ; S23. Generate a test report.

6. The AIGC test case adaptive generation system based on risk feedback according to claim 5, characterized in that: In S22, the normalized risk scores output by each risk detection model are weighted and fused to calculate the comprehensive risk level. include: The comprehensive risk level is calculated using the following formula. in, Risk m For the first m Normalized risk scores across multiple dimensions For the first Weighting coefficients for each dimension This refers to the number of risk detection dimensions.

7. The AIGC test case adaptive generation system based on risk feedback according to claim 1, characterized in that: The risk feedback processing module stores high-risk test cases and extracts features. Based on the extracted features, it transforms the risk assessment results into actionable risk feedback signals, including: S31. Risk Feature Extraction: Based on comprehensive risk level Risk total Store high-risk test cases, and extract risk features from these high-risk test cases: Input features include triggering words, syntactic structures, and semantic patterns; Output characteristics include risk type, risk location, and risk severity; Association features include causal relationship patterns between inputs and outputs; S32. Risk Vector Construction: Risk features are encoded into risk vectors, where each dimension corresponds to a risk feature, and the risk vectors are sent to the adaptive strategy adjustment module as actionable risk feedback signals.

8. The AIGC test case adaptive generation system based on risk feedback according to claim 1, characterized in that: The adaptive strategy adjustment module dynamically adjusts the mutation operator weight matrix based on risk feedback signals, including: S41. Define the state space: Define the risk assessment results as a quadruple. in This is the feature distribution matrix of historical high-risk test cases. This is the current mutation operator weight matrix. This is a test case coverage statistic for completed tests. Remaining test resources; S42. Designing the motion space: The action space is designed as the mutation operator weight matrix. Adjustment operations: Weight enhancement increases the selection weight of specific weight coefficients. Weight reduction, which lowers the selection weight of a specific weight coefficient; Strategy switching allows you to switch between different adjustment operations. Parameter fine-tuning involves adjusting hyperparameters, including mutation intensity and mutation number. S43. Design the reward function: Design a reward function that takes into account multiple optimization objectives. R ; S44, Strategy Update Mechanism: Deep Q-Network (DQN) is used for policy learning and updating: Maintain an experience replay buffer and periodically sample training data from the buffer to update the network parameters of the Deep Q-Network (DQN). Greedy strategies balance exploration and utilization, stabilizing the training process through the target network; S45, Weight Matrix Update: Update the mutation operator weight matrix based on the learned optimal strategy. W .

9. The AIGC test case adaptive generation system based on risk feedback according to claim 8, characterized in that: S43 comprehensively considers multiple optimization objectives in designing the reward function. R ,include: The reward function is expressed as follows: R : in, R discovery A reward for newly discovered high-risk test cases. R efficiency As a reward for improved testing efficiency, R cost Penalties for resource consumption All are trade-off coefficients.

10. The AIGC test case adaptive generation system based on risk feedback according to claim 8, characterized in that: In S45, the mutation operator weight matrix is ​​updated based on the learned optimal policy. W ,include: Update the mutation operator weight matrix W It can be expressed as follows: in, These are the updated and the original numbers, respectively. Class of risk corresponds to the first j The weighting coefficients of the mutation operator, For the first Class of risk corresponds to the first The adjustment amount of the weight coefficients of the mutation operator is determined by the output of the deep Q-network (DQN). This is the learning rate.

Citation Information

Cited By

  • Method and system for testing reasoning ability of artificial intelligence system

    CN121860067A

  • Method and system for testing the inferential ability of an artificial intelligence system

    CN121860067B

  • Security test method, system and equipment for text video model and medium

    CN122019395A

  • A method, system, device and medium for testing security of a pre-trained video model

    CN122019395B