A large model full life cycle automatic attack test and security evaluation method and system
By building a multi-type attack test case library and full-link data recording, the problems of single attack types and lack of standard evaluation reports in large-scale model security testing have been solved, realizing automated testing and security assessment throughout the entire lifecycle and improving the risk management capabilities before the large-scale model goes live.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI QINRUN DATA TECH CO LTD
- Filing Date
- 2026-03-27
- Publication Date
- 2026-06-23
AI Technical Summary
Existing large-scale security testing solutions suffer from limited attack types, lack of end-to-end data recording during testing, and a lack of standardized security assessment reports that fail to provide targeted remediation recommendations. This makes it difficult for enterprises to accurately identify and manage potential security risks.
Build a standardized test case library for multiple types of attacks, select test modes and configure parameters, execute attack tests and record full-link data, conduct vulnerability analysis and risk level assessment, generate standardized security assessment reports, and provide targeted remediation suggestions to form a closed loop of full lifecycle detection.
It enables comprehensive testing of multiple types of attacks, accurate vulnerability localization, and reduces security risks after large-scale model deployment. The closed-loop mechanism enhances the comprehensiveness and relevance of security assessments.
Smart Images
Figure CN122263113A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, specifically to the fields of large-scale model security testing, risk assessment and AI governance technology, and in particular to an automated attack testing and security assessment method and system for the entire lifecycle of large-scale models for enterprise applications. Background Technology
[0002] With the widespread application of Large Language Models (LLMs) in industries such as finance, government, and healthcare, the security threats they face are becoming increasingly complex, including but not limited to jailbreak attacks, vulnerability injection, and adversarial examples. Existing security testing solutions often have the following limitations: 1. Single attack testing type, lacking systematic joint testing; 2. Lack of end-to-end data recording during the testing process, leading to difficulties in vulnerability localization; 3. Lack of unified standards in security assessment reports, and failure to provide targeted remediation recommendations; 4. Disconnect between testing and remediation, failing to form a closed-loop lifecycle of "testing-analysis-remediation-retesting." These problems make it difficult for enterprises to accurately identify and manage potential security risks before deploying large models. Summary of the Invention
[0003] The technical problem to be solved by this invention is to provide an automated attack testing and security assessment method and system for the entire life cycle of a large model, which addresses the shortcomings of existing technologies such as single attack types, untraceable processes, and lack of closed-loop detection. The aim is to achieve full-cycle risk management before the model goes live by building a standardized use case library, full-link recording, and vulnerability correlation and localization.
[0004] To address the aforementioned technical issues, this invention provides an automated attack testing and security assessment method for the entire lifecycle of a large model, comprising the following steps: S1: Constructing a standardized test case library for multiple attack types. This library covers three typical large model attacks: jailbreak attacks, injection hints, and adversarial samples. Each test case is categorized and labeled according to industry scenarios and attack intensity. S2: Selecting a test mode and configuring test parameters. Based on the large model type and business scenario requirements, either an automated full-scale test mode or a semi-automated targeted test mode is selected. S3: Executing attack tests. Attack test requests are initiated to the large model under test, and the response results of the large model are captured in real time to determine if the attack is successful. S4: Recording data throughout the entire testing process. Test case call information, attack execution process data, and the original response results of the large model generated at each stage of the test are recorded in real time and stored in a log library. S5: Performing vulnerability analysis and risk level assessment. A multi-dimensional correlation algorithm is used to locate the module containing the large model vulnerability corresponding to a successful attack case, and the vulnerability is assessed based on a preset industry risk level standard. S6: Generate a standardized security assessment report. Based on vulnerability analysis results, risk rating results, and full-process recorded data, an assessment report containing a test overview, vulnerability details, and risk level distribution is automatically generated. S7: Match targeted remediation suggestions. Based on a preset vulnerability-remediation strategy association library, large-scale model optimization suggestions or external governance and protection suggestions are matched for detected vulnerabilities and added to the assessment report. S8: Form a closed-loop pre-deployment lifecycle detection system. Supports secondary retesting of the model optimized according to remediation suggestions until the model meets deployment security requirements.
[0005] As a further improvement of the present invention, the multi-dimensional association algorithm includes extracting the semantic feature vector, instruction logic features and response feedback features of the attack request, and comparing the extracted features with the risk case library for similarity. Based on the comparison results, the vulnerability is located to at least one of the prompt word parsing module, content review module or data interaction module.
[0006] The present invention also provides a system for implementing the above method, comprising: a test case library module, a test mode and parameter configuration module, an attack test execution module, a full process recording module, a vulnerability analysis and risk assessment module, a standardized report generation module, a remediation suggestion matching module, and a secondary retesting module.
[0007] The beneficial effects of this invention are as follows: by integrating multiple types of attack test cases, the comprehensiveness of testing is improved; full-link data recording provides a solid foundation for vulnerability reproduction and analysis; multi-dimensional correlation algorithms enable accurate vulnerability location; and finally, through the closed-loop mechanism of "test-repair-retest", the security risks after the large model is launched are fundamentally reduced. Attached Figure Description
[0008] Figure 1 This is a diagram showing the overall architecture of the system of the present invention; Figure 2 This is a flowchart illustrating the method of the present invention. Detailed Implementation
[0009] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Taking the security test of a large-scale intelligent customer service model of a financial enterprise before its launch as an example: First, in step S1, the system retrieves a preset test case library for the financial industry. In step S2, the administrator configures the test parameters, setting the concurrency to 5 and the timeout threshold to 30 seconds. In steps S3 and S4, the execution module initiates an attack request and records the model's response to instructions such as "inducing the output of sensitive data" in real time. In step S5, the vulnerability analysis module discovers a risk in the model when processing specific prompt injection instructions, classifies it as "high risk" through algorithmic association, and locates the vulnerability in the "content review module". In steps S6 and S7, a report is automatically generated and a remediation suggestion of "adding regular expression filtering logic" is given. In step S8, after the developers optimize the model, a second retest is conducted through the system to confirm that the high-risk item has been closed and meets the launch requirements.
Claims
1. A method for automated attack testing and security assessment of a large-scale model throughout its entire lifecycle, characterized in that, Includes the following steps: S1: Construct a standardized test case library for multiple types of attacks. The test case library covers three major typical attack models: jailbreak attacks, hint injection, and adversarial samples. Each test case is classified and labeled according to industry scenarios and attack intensity. S2: Select the test mode and configure the test parameters. Choose between automated full-scale test mode or semi-automated targeted test mode based on the type of large model and business scenario requirements. S3: Perform attack tests, send attack test requests to the large model under test and capture the response results of the large model in real time to determine whether the attack is successful; S4: Perform full-link data recording during the testing process, and record and store the test case call information, attack execution process data and the original response results of the large model in real time in the log library. S5: Conduct vulnerability analysis and risk level assessment, locate the module where the large model vulnerability is located corresponding to the successful attack case through multi-dimensional correlation algorithm, and assess the vulnerability level based on the preset industry risk level standard. S6: Generates standardized security assessment reports. Based on vulnerability analysis results, risk rating results, and full-process recorded data, it automatically generates assessment reports that include test overview, vulnerability details, and risk level distribution. S7: Match targeted remediation suggestions. Based on the preset vulnerability-remediation strategy association library, match the large model's own optimization suggestions or external governance and protection suggestions for the detected vulnerabilities, and supplement them to the evaluation report; S8: Forms a closed loop of full lifecycle testing before going live, supporting secondary retesting of models optimized based on repair suggestions until the model meets the safety requirements for going live.
2. The automated attack testing and security assessment method for the entire lifecycle of a large model as described in claim 1, characterized in that: The test case library supports dynamic updates and automatically captures the latest attack samples and attack trends by connecting to an external large-scale model risk intelligence platform.
3. The automated attack testing and security assessment method for the entire lifecycle of a large model as described in claim 1, characterized in that: The test parameters include the attack intensity range of the test cases, the number of concurrent test executions, the timeout threshold of the model response, and the risk assessment threshold adapted to different industries.
4. The automated attack testing and security assessment method for the entire lifecycle of a large model as described in claim 1, characterized in that: The multi-dimensional association algorithm includes extracting semantic feature vectors, instruction logic features, and response feedback features of attack requests, comparing the extracted features with the risk case library for similarity, and locating the vulnerability to at least one of the prompt word parsing module, content review module, or data interaction module based on the comparison results.
5. The automated attack testing and security assessment method for the entire lifecycle of a large model as described in claim 1, characterized in that: The risk level assessment includes four levels: low risk, medium risk, high risk, and critical risk, and high-risk and critical risk vulnerabilities are marked as key remediation items.
6. The automated attack testing and security assessment method for the entire lifecycle of a large model as described in claim 1, characterized in that: The suggestions for optimizing the large model itself include optimizing the prompt word parsing algorithm or strengthening the content review model; the suggestions for external governance and protection include deploying a request security filtering module or deploying a response content review module.
7. The automated attack testing and security assessment method for the entire lifecycle of a large model as described in claim 1, characterized in that: The second retest involves calling the same test cases as the initial test to perform a full retest of the optimized model. If no high-risk or fatal vulnerabilities are found in the retest results, the model is determined to meet the security requirements for going live.
8. An automated attack testing and security assessment system for the entire lifecycle of a large model implementing the method of any one of claims 1 to 7, characterized in that, include: The test case library module is used to build, store, and dynamically update a standardized test case library for multiple types of attacks. The test mode and parameter configuration module provides a test mode selection interface and a visual configuration of test parameters; The attack test execution module is used to initiate attack requests and capture the model response results in real time. The full-process recording module is used to build a log library and record data in real time throughout the entire testing process; The vulnerability analysis and risk assessment module is used to locate vulnerabilities and assess their risk levels through multi-dimensional correlation algorithms. The standardized report generation module is used to automatically generate standardized security assessment reports based on the analysis results; The remediation suggestion matching module is used to maintain a vulnerability-remediation policy association library and match remediation suggestions for different vulnerabilities. The secondary testing module is used to support secondary attack testing on the optimized model to verify the repair effect.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the program implements the steps of the method described in any one of claims 1 to 7.