Large Model Risk Assessment Using Offline Auxiliary Test Sets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current large model security assessment frameworks rely heavily on manual annotation, increasing costs and limiting scalability, while also posing risks of data leakage and resource overheads due to third-party service invocation.
Innovation Solution
A method and apparatus for large model risk assessment that utilizes a pre-constructed test set with auxiliary test results and label information from multiple assessment models, allowing offline annotation for multiple assessments and flexible scalability, without invoking the target model's interface.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation is used to review output content, then assessment accuracy is improved, but assessment costs increase and scalability is limited
Solution Approach 1:
The patent divides the assessment system into multiple auxiliary assessment models, each specializing in different risk detection tasks. These models process test data independently and their results are aggregated to form the final assessment, enabling scalable parallel processing while maintaining comprehensive evaluation accuracy
Solution Approach 2:
The patent creates a test set that includes auxiliary test results and label information from multiple auxiliary assessment models. This pre-constructed test set serves as a reference framework that can be reused across different assessment scenarios, eliminating the need for repeated manual annotation while preserving assessment accuracy
2Adaptability or versatility
If third-party services are invoked for assessment, then assessment capability is improved, but data leakage risks and resource overheads increase
Solution Approach 1:
The patent introduces an offline test set as an intermediary layer between the target large model and the assessment process. The test set contains pre-collected auxiliary test results and label information that mediate the assessment, eliminating the need to invoke third-party services or the target model's interface during actual assessment, thus preventing data leakage while maintaining assessment capability
Solution Approach 2:
The patent performs preliminary assessment work by constructing a test set with auxiliary test results and label information from multiple auxiliary assessment models before actual risk assessment. This advance preparation stores assessment knowledge in the test set, enabling subsequent assessments to be conducted offline without invoking external services, thereby reducing data leakage risks and resource overheads
Data Source
AI summary
Implementations of this specification disclose a large model risk assessment method, apparatus, and device. The method includes: obtaining a test set used to perform risk assessment on a target large model, the test set including test data, an auxiliary test result corresponding to the test data, and label information corresponding to the auxiliary test result, the test data including data of one or more different modalities, and the auxiliary test result being an auxiliary test result corresponding to the test data and outputted by each auxiliary assessment model after the test data are separately inputted into one or more different auxiliary assessment models; inputting the test data into the target large model to obtain a test result corresponding to the test data; and searching the obtained auxiliary test result for a target auxiliary test result matching the test result, determining label information corresponding to the test result based on label information corresponding to the target auxiliary test result, and determining a risk assessment result of the target large model based on the label information corresponding to the test result.