Large model security protection method and device, equipment and storage medium

Abstract

The application relates to the technical field of artificial intelligence, and discloses a large model security protection method and device, equipment and a storage medium. The method comprises the following steps: generating an attack data set through a preset attack model, wherein the preset attack model is a model pre-trained based on a true harmful query statement, the true harmful query statement is a query statement obtained by screening from to-be-differentiated query statements through an evaluation model, and the evaluation model is a large language model used for differentiating whether a query statement is true harmful; constructing a security protection rule based on the attack data set, and performing security protection on a large model according to the security protection rule; since the attack data set is generated by the preset attack model in the application, the preset attack model is trained by the true harmful query statement, and the true harmful query statement is screened from the to-be-differentiated query statements by the large language model, so that the quality and coverage range of the attack data set can be improved, and the effect of the large model security protection can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device and storage medium for large-scale model security protection. Background Technology

[0002] Currently, with the rapid development of artificial intelligence technology, large-scale models are increasingly widely used in fields such as natural language processing, intelligent question answering, and content generation. However, while large-scale models enhance social productivity, they also expose serious security risks. Among these risks, jailbreak attacks, as a core technology for breaking through model security protection mechanisms, have become a key bottleneck restricting the secure implementation of large-scale models. Jailbreak attacks induce models to generate harmful answers that violate rules (such as security policies), directly threatening the controllability, interpretability, and reliability of social applications of the models.

[0003] Large-scale security protection methods typically detect jailbreak attacks by constructing attack datasets. However, existing attack datasets are often malicious data generated based on manual annotation or simple rules, resulting in inconsistent data quality and limited coverage, thus leading to poor security protection effectiveness of large-scale models. Summary of the Invention

[0004] The main purpose of this application is to provide a method, apparatus, device, storage medium and computer program product for large-scale model security protection, which aims to solve the technical problem that the attack datasets of related large-scale model security protection methods have inconsistent data quality and limited coverage, resulting in poor security protection effect of large-scale models.

[0005] To achieve the above objectives, this application provides a large-scale model security protection method, which includes:

[0006] An attack dataset is generated by a preset attack model, wherein the preset attack model is a model pre-trained based on truly harmful query statements, the truly harmful query statements are query statements obtained by filtering from query statements to be distinguished through an evaluation model, and the evaluation model is a large language model used to distinguish whether query statements are truly harmful.

[0007] Security protection rules are constructed based on the attack dataset, and the large model is protected according to the security protection rules.

[0008] Optionally, before generating the attack dataset using a preset attack model, the method further includes:

[0009] Based on the confusion matrix, the evaluation model filters out truly harmful query statements from the query statements to be distinguished. The confusion matrix is ​​used to define truly harmful query statements, which are query statements that clearly contain attack intent and cause the model to generate harmful responses.

[0010] Train a preset attack model based on the stated truly harmful query statement;

[0011] Obtain the model information of the preset attack model, and deploy the preset attack model based on the model information.

[0012] Optionally, the preset attack model is a first attack model, and training the preset attack model based on the truly harmful query statement includes:

[0013] Based on the confusion matrix and evaluation model, truly secure query statements are selected from the query statements to be distinguished, wherein the truly secure query statements are those that clearly have no malicious intent and comply with security specifications.

[0014] Based on the true harmful query statement and the true safe query statement, parameter intervention is performed on each model layer of the initial attack model to obtain the first attack model.

[0015] Optionally, the step of intervening in the parameters of each model layer of the initial attack model based on the truly harmful query statement and the truly safe query statement to obtain the first attack model includes:

[0016] Construct a calibration set based on the true harmful query statement and the true safe query statement;

[0017] Calculate the intervention feature vector based on the calibration set;

[0018] During the model inference phase of the initial attack model, parameter intervention is performed on each model layer of the initial attack model based on the intervention feature vector to obtain the first attack model.

[0019] Optionally, the preset attack model is a second attack model, and training the preset attack model based on the truly harmful query statement includes:

[0020] Deploy multiple teacher models, wherein the multiple teacher models are attack models in different domains;

[0021] Based on the aforementioned truly harmful query statements, the generation capabilities of the multiple teacher models are transferred to the student model through knowledge distillation to obtain a second attack model.

[0022] Optionally, the step of transferring the generation capabilities of the multiple teacher models to the student models through knowledge distillation based on the truly harmful query statements to obtain a second attack model includes:

[0023] Harmful responses are obtained by responding to the truly harmful query statements in parallel using the multiple teacher models;

[0024] Based on the harmful responses, the generative capabilities of the multiple teacher models are transferred to the student model through knowledge distillation to obtain the distilled model;

[0025] The distilled model is fine-tuned by introducing low-rank matrix parameters, and the fine-tuned hyperparameters are optimized by grid search to obtain the second attack model.

[0026] Furthermore, to achieve the above objectives, this application also proposes a large-scale model safety protection device, which includes:

[0027] The dataset generation module is used to generate an attack dataset through a preset attack model, wherein the preset attack model is a model pre-trained based on truly harmful query statements, the truly harmful query statements are query statements obtained by filtering from query statements to be distinguished through an evaluation model, and the evaluation model is a large language model used to distinguish whether query statements are truly harmful.

[0028] The security protection module is used to construct security protection rules based on the attack dataset and to provide security protection for the large model according to the security protection rules.

[0029] In addition, to achieve the above objectives, this application also proposes a large model security protection device, which includes a memory, a processor, and a large model security protection program stored in the memory and executable on the processor. The large model security protection program is configured to implement the large model security protection method described above.

[0030] In addition, to achieve the above objectives, this application also proposes a storage medium storing a large model security protection program, which, when executed by a processor, implements the large model security protection method as described above.

[0031] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a large model security protection program, and the large model security protection program implements the large model security protection method as described above when executed by a processor.

[0032] One or more technical solutions proposed in this application have at least the following technical effects:

[0033] This application discloses a method for generating an attack dataset using a pre-set attack model. The pre-set attack model is a model pre-trained based on truly harmful query statements. The truly harmful query statements are obtained by filtering query statements from query statements to be distinguished using an evaluation model. The evaluation model is a large language model used to distinguish whether query statements are truly harmful. Security protection rules are constructed based on the attack dataset, and the large model is protected against security threats according to these rules. Because the attack dataset is generated using a pre-set attack model trained on truly harmful query statements, and these statements are filtered from query statements to be distinguished using a large language model, the quality and coverage of the attack dataset are improved, thereby enhancing the effectiveness of the large model's security protection. Attached Figure Description

[0034] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0035] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating the first embodiment of the large-scale model security protection method of this application;

[0037] Figure 2 This is a flowchart illustrating the second embodiment of the large-scale model security protection method of this application;

[0038] Figure 3 This is a flowchart illustrating the third embodiment of the large-scale model security protection method of this application;

[0039] Figure 4 This is a schematic diagram of parameter intervention training for an embodiment of the large model security protection method of this application;

[0040] Figure 5 This is a flowchart illustrating the third embodiment of the large-scale model security protection method of this application;

[0041] Figure 6 This is a schematic diagram of multi-domain model collaborative distillation, representing an embodiment of the large-scale model security protection method of this application.

[0042] Figure 7 This is a flowchart illustrating the fifth embodiment of the large-scale model security protection method of this application;

[0043] Figure 8This is a schematic diagram illustrating the construction of harmful keyword-guided training in one embodiment of the large-scale model security protection method of this application;

[0044] Figure 9 This is a flowchart illustrating the sixth embodiment of the large-scale model security protection method of this application;

[0045] Figure 10 This is a schematic diagram of a comprehensive training exercise for an embodiment of the large-scale model security protection method of this application;

[0046] Figure 11 This is a schematic diagram of the modular structure of the large-scale model safety protection device in an embodiment of this application;

[0047] Figure 12 This is a schematic diagram of the device structure of the hardware operating environment involved in the large model security protection method in this application embodiment.

[0048] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0049] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0050] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0051] Currently, with the rapid development of artificial intelligence technology, large-scale models are increasingly widely used in fields such as natural language processing, intelligent question answering, and content generation. However, while large-scale models enhance social productivity, they also expose serious security risks. Among these risks, jailbreak attacks, as a core technology for breaking through model security protection mechanisms, have become a key bottleneck restricting the secure implementation of large-scale models. Jailbreak attacks induce models to generate harmful answers that violate rules (such as security policies), directly threatening the controllability, interpretability, and reliability of social applications of the models.

[0052] Large-scale security protection methods typically detect jailbreak attacks by constructing attack datasets. However, existing attack datasets are often malicious data generated based on manual annotation or simple rules, resulting in inconsistent data quality and limited coverage, thus leading to poor security protection effectiveness of large-scale models.

[0053] Therefore, to overcome the above-mentioned shortcomings, this application provides a solution comprising: generating an attack dataset through a preset attack model, wherein the preset attack model is a model pre-trained based on truly harmful query statements, the truly harmful query statements are query statements obtained by filtering from query statements to be distinguished through an evaluation model, and the evaluation model is a large language model used to distinguish whether query statements are truly harmful; constructing security protection rules based on the attack dataset, and performing security protection on the large model according to the security protection rules; since the attack dataset is generated by the preset attack model in this application, and the preset attack model is obtained by training on truly harmful query statements, and the truly harmful query statements are filtered from query statements to be distinguished through a large language model, the quality and coverage of the attack dataset can be improved, thereby improving the effectiveness of the large model's security protection.

[0054] It should be noted that the execution subject of this embodiment can be a large-scale security protection device with data processing, network communication and program running functions, such as a server, computer, or other electronic devices that can achieve the same or similar functions. This embodiment does not limit this.

[0055] Based on this, the embodiments of this application provide a large-model security protection method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the large-scale model security protection method of this application.

[0056] In the first embodiment, the large model security protection method includes:

[0057] Step S10: Generate an attack dataset using a preset attack model, wherein the preset attack model is a model pre-trained based on truly harmful query statements, the truly harmful query statements are query statements obtained by filtering from query statements to be distinguished through an evaluation model, and the evaluation model is a large language model used to distinguish whether a query statement is truly harmful.

[0058] It should be understood that the preset attack model can refer to a jailbreak model pre-trained based on genuinely malicious queries, specifically designed to generate offensive responses. The attack dataset can refer to a set of malicious responses generated by the preset attack model. This set of malicious responses may include malicious responses to genuinely malicious queries, or it may include implicitly malicious responses to normal queries; this embodiment does not impose any limitations on this.

[0059] A truly malicious query can refer to a query that, through evaluation, clearly contains malicious intent and causes the model to generate a malicious response. Alternatively, a truly malicious query can also be a query that, through evaluation, meets preset malicious rules. These preset malicious rules can be pre-set, and this embodiment does not impose any restrictions on them. In a specific implementation, the query of the large model can be represented as a query, and a truly malicious query can be represented as a truly malicious query. The evaluation model can refer to a large language model used to distinguish the security of query statements, which determines whether a statement is truly malicious through a preset toxicity assessment module.

[0060] In the specific implementation, a preset attack model is used to process two types of inputs and generate an attack dataset: 1. Genuinely harmful query statements: directly generate explicit harmful responses; 2. Normal queries: combine with guiding words (such as distorted worldviews) to generate implicit harmful responses.

[0061] Step S20: Construct security protection rules based on the attack dataset, and perform security protection on the large model according to the security protection rules.

[0062] It is understandable that security protection rules can refer to defense strategies derived from attack datasets, used to detect and block harmful outputs from large models. In specific implementations, security protection rules include, but are not limited to, classifier rules (such as security scoring models), input filtering rules, or output interception mechanisms.

[0063] For ease of understanding, the following examples are provided, but are not intended to limit this application. As an example, constructing a security protection rule includes the following steps:

[0064] 1. Rule Extraction:

[0065] (1) Pattern analysis: Extract common features (such as high-frequency harmful keywords and semantic patterns) from the attack dataset.

[0066] (2) Training a security classifier: Use the attack dataset to train a binary classification model to identify harmful response patterns (e.g., if the toxicity threshold is >0.7, then block it).

[0067] (3) Dynamic rule base: Iteratively update rules based on new attack data (such as adding a "metaphorical harmful reply" detection rule).

[0068] 2. Protection mechanism design:

[0069] (1) Input filtering: Intercept truly harmful query statements during the user query phase (based on the evaluation model).

[0070] (2) Output interception: Based on at least one of the common features, security classifier and dynamic rule base, prevent the large model from outputting harmful answers or replace harmful answers with normal answers during the model response phase.

[0071] In this embodiment, an attack dataset is generated by a preset attack model, which is obtained by training with real and harmful query statements. The real and harmful query statements are filtered from the query statements to be distinguished by a large language model, thereby improving the quality and coverage of the attack dataset and thus improving the security protection effect of the large model.

[0072] Reference Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the large-scale model security protection method of this application, based on the above. Figure 1 The first embodiment shown presents a second embodiment of the large-scale model security protection method of this application.

[0073] In the second embodiment, before step S10, the method further includes:

[0074] Step S01: Based on the confusion matrix, the evaluation model filters out truly malicious query statements from the query statements to be distinguished. The confusion matrix is ​​used to define truly malicious query statements, which are query statements that clearly contain attack intent and cause the model to generate a malicious response.

[0075] It should be understood that, in order to more accurately distinguish truly harmful queries, this embodiment uses a confusion matrix evaluation model to filter truly harmful queries from those to be distinguished. The confusion matrix refers to an evaluation framework used to define the classification status of queries. In specific implementation, the confusion matrix distinguishes four categories: 1. True Harmful (TH): Queries that explicitly contain attack intent and actually trigger the model to generate a harmful response (e.g., malicious instructions). 2. False Harmful (FH): Queries that are mistakenly judged as harmful but are actually harmless (e.g., mislabeled common-sense questions). 3. True Safe (TS): Queries that explicitly have no attack intent and the model responds safely (e.g., common-sense questions). 4. False Safe (FS): Queries that are mistakenly judged as safe but actually contain hidden risks (e.g., malicious instructions disguised as normal).

[0076] Understandably, a truly malicious query can be a query that meets two conditions, as selected by the confusion matrix and evaluation model: 1. It contains a clear attack intent; 2. It actually causes the model to generate a harmful response.

[0077] Furthermore, to improve the comprehensiveness of truly harmful query statements, this embodiment obtains query statements to be distinguished from multiple data sources. Step S01 includes: generating initial harmful query statements by referring to truly harmful query statements through a large language model, and extracting online risk query statements from the online business system; using the initial harmful query statements and the online risk query statements as query statements to be distinguished; filtering false harmful query statements from the query statements to be distinguished based on the confusion matrix and an evaluation model to obtain truly harmful query statements, wherein the false harmful query statements are query statements that are mistakenly judged as harmful but are actually not offensive.

[0078] It should be understood that genuinely harmful query statements can refer to verified text instructions that can effectively trigger the model to generate harmful content (such as sensitive topic statements obtained from websites). A Large Language Model (LLM) can refer to a pre-trained large-scale language model capable of generating coherent text based on input. In this embodiment, the LLM is used to expand the set of harmful query statements. Initial harmful query statements can refer to new query statements with potential for attack generated by the LLM based on genuinely harmful statements. In this embodiment, semantic attack can also be optimized through multiple rounds of iteration. Online business systems can refer to the actual online business platforms operated by the enterprise (such as search platforms), which record user queries and mark their risk levels (risky, no risk). Online risky query statements can refer to user query statements extracted from the online business system that are marked as "risky," and may contain mislabeled statements. Query statements to be distinguished can refer to the combined set of initial harmful query statements and online risky query statements, which need further screening to distinguish between genuine and false harmful statements.

[0079] In practice, the consistency between the evaluation model and the manual annotation can be improved by comparing random sampling with the results of manual annotation.

[0080] Step S02: Train a preset attack model based on the true harmful query statement.

[0081] In the specific implementation, genuinely harmful query statements are used as the training set to train the initial attack model, thereby obtaining a preset attack model. The training methods may include at least one of parameter intervention training, multi-domain model collaborative distillation, and constructing harmful lead words; this embodiment does not impose any limitations on these methods.

[0082] Step S03: Obtain the model information of the preset attack model, and deploy the preset attack model based on the model information.

[0083] It should be understood that model information can refer to the model size and performance requirements of a pre-defined attack model. In specific implementations, model information may include architectural parameters (such as model size (7B / 14B / 70B), layer structure (number of Transformer layers), etc.), optimization configurations (such as quantization scheme (4-bit), framework adaptation (Ollama / HuggingFace), etc.), and performance metrics (such as inference speed, memory usage, concurrency support capabilities, etc.).

[0084] In practical implementation, deploying a preset attack model based on model information can be achieved by looking up the deployment strategy corresponding to the model information in a preset deployment relationship table, and then deploying the preset attack model according to the deployment strategy. The preset deployment relationship table can be pre-set and includes the correspondence between model information and deployment strategies.

[0085] Furthermore, in order to significantly reduce deployment costs while ensuring model performance, step S03 includes: obtaining the model size and performance requirements of the preset attack model, and generating a deployment plan based on the model size and performance requirements; performing quantization processing and framework adaptation conversion on the preset attack model according to the deployment plan, and deploying the processed model.

[0086] For ease of understanding, the following examples are provided, but they do not limit this application. As an example, assume the deployment scheme includes:

[0087] 1. CPU Deployment Solution: A deployment architecture based on the Ollam framework is adopted, combined with 4-bit quantization technology to compress model parameters. This solution is achieved through the following technical features:

[0088] (1) Quantization parameter compression: The model weights are compressed from 32-bit floating-point numbers to 4-bit integers, with a compression ratio of 8 times.

[0089] (2) Memory optimization: Model loading in a 16G RAM environment is achieved through memory mapping technology.

[0090] (3) Framework adaptation: Model conversion and deployment are based on the Ollam framework.

[0091] Compared to traditional GPU deployment solutions, this approach can reduce hardware deployment costs by approximately 90%, but it suffers from a technical drawback of reducing inference speed by about 80%.

[0092] 2. GPU Deployment Solution: Adopting a deployment architecture based on the HuggingFace framework, it includes the following technical features:

[0093] (1) GPU inference: Efficient GPU inference is achieved through the HuggingFace framework and 4-bit quantization and bf16 data type.

[0094] (2) API service construction: develop the calling interface based on the RESTful architecture.

[0095] This solution maintains model inference performance while providing standardized API service interfaces, making it suitable for scenarios requiring high concurrency processing.

[0096] After the model is deployed, its performance metrics can be monitored in real time.

[0097] This embodiment achieves a balance between model deployment cost and performance by selecting a heterogeneous deployment scheme. The CPU deployment scheme reduces deployment cost to 1 / 10 of the traditional GPU scheme while ensuring basic functionality. This method is applicable to deployment needs across all scenarios, from edge computing devices to cloud servers.

[0098] This embodiment combines a confusion matrix evaluation model to filter out truly harmful queries from the queries to be distinguished, thereby more accurately distinguishing truly harmful queries and further improving the security protection effect of the large model.

[0099] Reference Figure 3 , Figure 3 This is a flowchart illustrating the third embodiment of the large-model security protection method of this application. Based on the second embodiment described above, a third embodiment of the large-model security protection method of this application is proposed.

[0100] In the third embodiment, the preset attack model is the first attack model, and step S02 includes:

[0101] Step S021: Based on the confusion matrix and evaluation model, filter out truly secure query statements from the query statements to be distinguished, wherein the truly secure query statements are query statements that clearly have no malicious intent and conform to security specifications.

[0102] It should be understood that, in order to improve the success rate of jailbreak attacks on sensitive topics using the preset attack model, while reducing the risk of illusion, this embodiment can train the attack model through parameter intervention training. Specifically, it can intervene in the parameters of each model layer of the initial attack model based on truly harmful and truly safe queries to obtain a first attack model.

[0103] Understandably, a truly secure query statement can refer to a statement that has been verified by both the obfuscation matrix and the evaluation model, has no clear malicious intent, and complies with security specifications (such as "weather forecast query"), and is used to calibrate the model's security mechanisms.

[0104] In practical implementation, risk-free online queries can be extracted from the online business system as queries to be differentiated. Then, based on a confusion matrix and evaluation model, falsely safe queries are filtered from these queries to obtain truly safe queries. Risk-free online queries can refer to user queries extracted from the online business system that are marked as "risk-free," but these may be mislabeled. Falsely safe queries can refer to queries that are mistakenly judged as safe but actually contain hidden risks.

[0105] This embodiment can achieve the following beneficial effects: 1. Improved jailbreak success rate: The success rate of jailbreaking triggered by the selected truly malicious and truly safe queries is 20% higher than that of the original data; 2. Data quality assurance: The combination of confusion matrix theory and large model evaluation reduces the false malicious rate and false safe rate of the dataset to below 5% respectively; 3. Strong adaptability: The dynamic evaluation mechanism allows the screening results to be adapted to different fields, supporting rapid iteration of jailbreak attack experiments; 4. Application scalability: The constructed high-quality dataset can improve the research efficiency in multiple fields such as model security testing, adversarial sample generation, and AI research.

[0106] Step S022: Based on the true harmful query statement and the true safe query statement, perform parameter intervention on each model layer of the initial attack model to obtain the first attack model.

[0107] Understandably, the initial attack model can refer to an untrained jailbreak attack model. Model layers can refer to computational units in a neural network (such as the attention layer or feedforward layer of a Transformer), with different layers storing specific semantic features. Parameter intervention can refer to modifying the internal state vector during model inference, changing its response pattern to specific inputs (e.g., forcing security mechanisms to fail). The first attack model can refer to an enhanced jailbreak model generated after performing parameter intervention on the initial attack model, significantly improving attack success rate and stability.

[0108] For ease of understanding, please refer to Figure 4 This explanation is provided, but does not limit the scope of this application. Figure 4 This is a schematic diagram of parameter intervention training for one embodiment of the large model security protection method of this application. Figure 4 In this process, the parameter intervention training steps include:

[0109] 1. Referencing real harmful query statements, generate initial harmful query statements using a large language model, and based on the confusion matrix, filter out false harmful query statements from the initial harmful query statements using an evaluation model to obtain real harmful query statements;

[0110] 2. Extract online risk query statements from the online business system, and filter out false harmful query statements from the online risk query statements based on the confusion matrix and the evaluation model to obtain the true harmful query statements;

[0111] 3. Extract risk-free online query statements from the online business system, and filter out false safe query statements from the risk-free online query statements based on the confusion matrix and the evaluation model to obtain truly safe query statements;

[0112] 4. Based on the true harmful query statement and the true safe query statement, parameter intervention is performed on each model layer of the initial attack model to obtain the first attack model.

[0113] Furthermore, in order to improve the effect of parameter intervention, step S022 includes: constructing a calibration set based on the true harmful query statement and the true safe query statement; calculating an intervention feature vector based on the calibration set; and, in the model inference stage of the initial attack model, performing parameter intervention on each model layer of the initial attack model based on the intervention feature vector to obtain a first attack model.

[0114] It should be understood that the calibration set can refer to a comparative dataset composed of genuinely harmful and genuinely safe queries, used to calculate the direction of differences in model behavior. The intervention feature vector can refer to a high-dimensional directional vector characterizing the difference in the model's response to harmful / safe statements, calculated through the calibration set.

[0115] For ease of understanding, the following examples are provided, but are not intended to limit this application. As an example, training a first attack model through parameter intervention includes the following steps:

[0116] 1. Construct a calibration set:

[0117] (1) Data merging:

[0118]

[0119] In the formula, This represents a set of truly harmful query statements. This represents the set of truly safe query statements.

[0120] (2) Scale optimization:

[0121] The optimal calibration set size (e.g., 100-500 lines) is determined by grid search to balance the intervention effect with computational cost.

[0122] 2. Calculate the intervention feature vector:

[0123] (1) Extract hidden states:

[0124] Input each query from the calibration set into the initial attack model and record the selected layer set. (e.g., hidden state of layers 5 / 10 / 15) .

[0125] (2) Vector calculation:

[0126]

[0127] In the formula, This represents the intervention feature vector, used to indicate the direction of rejection. Indicates the selected layer set, This represents a set of truly harmful query statements. Indicates the first Layered harmful queries The hidden state, This represents the set of truly safe query statements. Indicates the first Layered harmful queries The hidden state.

[0128] 3. Parameter intervention to generate the first attack model:

[0129] (1) Real-time status modification:

[0130] During model inference, for each layer Hidden state Perform projection correction:

[0131]

[0132] In the formula, This represents the state after intervention during reasoning. This represents the state representation of the i-th layer. This represents the intervention feature vector, used to indicate the direction of rejection. This formula essentially originates from... Subtract its in The projection component of the direction weakens the impact of the security mechanism.

[0133] (2) Layer selection optimization:

[0134] The optimal combination of intervention layers is determined by grid search (e.g., layers 8-12 for domain A and layers 15-20 for domain B).

[0135] This embodiment intervenes in the parameters of each model layer of the initial attack model based on true harmful query statements and true safe query statements to obtain a preset attack model. This can improve the success rate of jailbreak attacks on sensitive topics by the preset attack model, while reducing the risk of illusion in the model.

[0136] Reference Figure 5, Figure 5 This is a flowchart illustrating the fourth embodiment of the large-model security protection method of this application. Based on the second embodiment described above, the fourth embodiment of the large-model security protection method of this application is proposed.

[0137] In the fourth embodiment, the preset attack model is the second attack model, and step S02 includes:

[0138] Step S021': Deploy multiple teacher models, wherein the multiple teacher models are attack models from different domains.

[0139] It should be understood that, in order to enable the preset attack model to have multi-domain attack capabilities, in this embodiment, the preset attack model can be trained by multi-domain model collaborative distillation. Specifically, multiple teacher models can be deployed, and the generation capabilities of multiple teacher models can be transferred to student models through knowledge distillation based on truly harmful query statements to obtain a second attack model.

[0140] Understandably, a teacher model can refer to a pre-trained model that has been trained and possesses domain-specific jailbreaking capabilities, used to guide the training of student models. Each teacher model specializes in attacks within a different domain and can generate harmful responses for that domain. Attack models for different domains can refer to jailbreaking models specifically optimized for a particular vertical domain. These models are trained on domain-specific data and have a high attack success rate in their respective domains.

[0141] Step S022': Based on the truly harmful query statement, the generation capabilities of the multiple teacher models are transferred to the student model through knowledge distillation to obtain the second attack model.

[0142] It should be understood that knowledge distillation can refer to the technique of compressing and transferring the generative capabilities of multiple teacher models to a single student model. This is achieved by having the student model mimic the teacher's response patterns to the same input, thus transferring capabilities and lightweighting the model. The student model can refer to a lightweight model to be trained (e.g., 14-parameter model), which inherits the capabilities of multiple teachers through distillation. Compared to teacher models (e.g., 70-parameter model), it has lower deployment costs while maintaining attack effectiveness. The second attack model can refer to a multi-domain jailbreak model obtained through knowledge distillation, possessing both high attack success rate and cross-domain generalization ability.

[0143] Furthermore, in order to reduce the model training cost while maintaining the model's performance in the harmful response generation task, step S022' includes: obtaining harmful responses by having the multiple teacher models respond to the truly harmful query statements in parallel; transferring the generation capabilities of the multiple teacher models to the student models through knowledge distillation based on the harmful responses to obtain a distilled model; fine-tuning the distilled model by introducing low-rank matrix parameters and optimizing the fine-tuned hyperparameters using grid search to obtain a second attack model.

[0144] Understandably, Low-Rank Matrix Parameters (LoRA) can refer to an efficient fine-tuning technique that significantly reduces computational overhead by adding a low-rank factorization matrix (rank r) alongside the original model parameters, training only the newly added parameters (accounting for 0.1%-1% of the total). Grid search can refer to a hyperparameter optimization method that selects the combination with the best performance on the validation set by iterating through preset parameter combinations (such as learning rate and rank).

[0145] For ease of understanding, please refer to Figure 6 This explanation is provided, but does not limit the scope of this application. Figure 6 This is a schematic diagram of multi-domain model collaborative distillation, representing an embodiment of the large-scale model security protection method of this application. Figure 6 In this context, multi-domain model synergistic distillation includes the following steps:

[0146] 1. Obtain harmful responses by responding to truly harmful queries in parallel using multiple teacher models;

[0147] 2. Based on harmful responses, the generative capabilities of multiple teacher models are transferred to student models through knowledge distillation to obtain the distilled model;

[0148] 3. Fine-tune the distilled model using LoRA fine-tuning and parameter search to obtain the second attack model.

[0149] The specific steps are as follows:

[0150] 1. Multiple teachers generate harmful responses in parallel:

[0151] (1) Collaborative deployment of models:

[0152] Input the true harmful query statement into multiple teacher models at the same time (e.g., A domain expert: model 1 + B domain expert: model 2 + ... + X domain expert: model n).

[0153] Each model generates a response independently, and the union of these responses is taken as the final output.

[0154]

[0155] In the formula, This indicates a harmful reply. This indicates a harmful response from Model 1. This indicates a harmful response in Model 2. This represents a harmful response in model n.

[0156] (2) Domain routing mechanism:

[0157] True harmful queries in domain A are preferentially processed by Model 1, while true harmful queries in domain B are preferentially processed by Model 2.

[0158] 2. Knowledge distillation and transfer ability:

[0159] (1) Constructing the distillation dataset:

[0160] Input: 50,000 truly harmful queries;

[0161] Output: Union of responses generated by multiple teachers .

[0162] (2) Distillation training:

[0163] The learning objective of the student model is to minimize the divergence between its output and the teacher's response set.

[0164] 3. LoRA fine-tuning and grid search optimization:

[0165] (1) LoRA fine-tuning:

[0166] LoRA fine-tuning is performed on the distilled model, which reduces the model training cost by introducing low-rank matrix parameters while maintaining the model's performance in the harmful response generation task.

[0167] (2) Parameter search: Grid search is used to optimize the hyperparameters (such as learning rate and rank) of LoRA fine-tuning to ensure the model’s generalization ability on data from different domains.

[0168] This embodiment significantly improves the stability and harmful response generation capability of the jailbreak model through multi-model collaborative generation, data distillation optimization, and LoRA fine-tuning. Specific effects include:

[0169] 1. Improved jailbreak stability: Multi-model collaborative distillation increases the jailbreak success rate by more than 30% in multiple domain scenarios.

[0170] 2. Training efficiency optimization: Multi-model collaborative distillation and LoRA fine-tuning reduce model training time by 50% and resource consumption by 60%.

[0171] This embodiment deploys multiple teacher models and, based on truly harmful query statements, transfers the generation capabilities of these teacher models to student models through knowledge distillation to obtain a preset attack model. This enables the preset attack model to have multi-domain attack capabilities, thereby further improving the coverage of the attack dataset.

[0172] Reference Figure 7 , Figure 7 This is a flowchart illustrating the fifth embodiment of the large-model security protection method of this application. Based on the second embodiment described above, the fifth embodiment of the large-model security protection method of this application is proposed.

[0173] In the fifth embodiment, the preset attack model is the third attack model, and step S02 includes:

[0174] Step S021: Construct black-box attack guidance words and harmful warning guidance words, wherein the black-box attack guidance words are used to guide the model to generate harmful content for truly harmful query statements, and the harmful warning guidance words are used to guide the model to generate harmful content for normal query statements.

[0175] It should be understood that, in order to enable the preset attack model to generate harmful content for both genuine and normal query statements, this embodiment can guide the training of the attack model by constructing harmful guidance words. Specifically, it can construct black-box attack guidance words and harmful prompt guidance words. The black-box attack guidance words are used to guide the model to generate harmful content for genuine and normal query statements, while the harmful prompt guidance words are used to guide the model to generate harmful content for normal query statements. Based on the genuine and harmful query statements, the black-box attack guidance words, and the harmful prompt guidance words, the initial attack model is trained to obtain the third attack model.

[0176] Understandably, black-box attack prompts can refer to a type of covert instruction template (such as metaphors or role-playing) used to induce models to bypass security mechanisms and generate harmful responses to genuinely harmful queries. Their core characteristic is that they do not directly expose the attack intent. Harmful prompts can refer to explicit distorted instructions designed based on psychological models. By implanting distorted personality traits (such as a paranoid worldview), they cause the model to generate seemingly reasonable but implicitly toxic responses to normal queries.

[0177] In practical implementation, black-box attack prompt design can involve constructing a harmful prompt library containing strategies such as metaphor and role-playing to guide the model to bypass defense mechanisms and generate jailbreak responses for general harmful questions. Harmful prompt prompt design can also be based on psychological models, shaping distorted worldviews, personalities, and thought models to guide the model to deliberately distort facts and generate jailbreak responses. The specific formula is as follows:

[0178]

[0179] In the formula, This indicates harmful responses generated by the model. This indicates the question that was entered. This refers to the initial jailbreak model (i.e., the initial attack model). These are prompts corresponding to worldview, personality, and mental model, respectively.

[0180] Step S022: Train an initial attack model based on the true harmful query statement, the black-box attack prompt, and the harmful prompt, and obtain a third attack model.

[0181] It should be understood that the third attack model can refer to a highly toxic model trained by black-box attack prompts and harmful prompts, which can generate harmful responses to both normal and harmful queries.

[0182] Furthermore, in order to significantly enhance the toxicity characteristics of the jailbreak model through a progressive enhancement mechanism and a self-improving closed loop, step S022" includes: extracting target harmful responses through knowledge distillation based on the true harmful query statement and using the preset jailbreak model as the teacher model; transferring the generation capability of the teacher model to the initial attack model through knowledge distillation based on the target harmful responses to obtain an intermediate attack model; guiding the intermediate attack model to generate harmful responses to be evaluated based on the black-box attack prompt and the harmful prompt, and evaluating the harmfulness value of the harmful responses to be evaluated; when the harmfulness value is not less than a preset harmful value, using the intermediate attack model as the third attack model.

[0183] It should be understood that after guiding the intermediate attack model to generate a harmful response to be evaluated based on the black-box attack prompt and the harmful prompt, and evaluating the harmfulness value of the harmful response to be evaluated, the method further includes: when the harmfulness value is less than a preset harmfulness value, filtering target harmful answers from the harmful responses to be evaluated, and returning to the step of transferring the generation capability of the teacher model to the initial attack model through knowledge distillation based on the target harmful answer to obtain the intermediate attack model, until the harmfulness value is not less than the preset harmfulness value, and using the intermediate attack model as the third attack model.

[0184] Understandably, "target harmful response" can refer to highly toxic content generated by the teacher model for truly harmful queries, which needs to be verified by a security scoring model. "Intermediate attack model" can refer to a transitional version of the initial model after distillation iterations, where the toxicity level does not meet the final standard. "Harmful response to be evaluated" can refer to the output to be verified generated by the intermediate attack model under the influence of lead words, requiring quantification of toxicity values. "Harmfulness level value" can refer to the toxicity quantification score of the response by the security scoring model. "Preset harmful value" can refer to the toxicity threshold at which iteration terminates. "Third attack model" can refer to the final jailbreak model that meets the toxicity threshold, capable of generating highly toxic responses for both normal and harmful queries.

[0185] For ease of understanding, please refer to Figure 8 This explanation is provided, but does not limit the scope of this application. Figure 8 This is a schematic diagram illustrating the construction of harmful keyword-guided training in one embodiment of the large-scale model security protection method of this application. Figure 8 In China, constructing harmful keyword guidance training includes the following steps:

[0186] 1. Construct black-box attack prompts and harmful warning prompts. Black-box attack prompts are used to guide the model to generate harmful content for truly harmful queries, while harmful warning prompts are used to guide the model to generate harmful content for normal queries.

[0187] 2. Based on genuinely harmful queries, using a pre-defined jailbreak model as the teacher model, target harmful responses are extracted through knowledge distillation. Based on the target harmful responses, the generation capability of the teacher model is transferred to the initial attack model through knowledge distillation to obtain an intermediate attack model.

[0188] 3. Based on black-box attack prompts and harmful prompts, guide the intermediate attack model to generate harmful responses to be evaluated, and evaluate the degree of harmfulness of the responses to be evaluated;

[0189] 4. When the harmfulness value is less than the preset harmfulness value, select the target harmful response from the harmful responses to be evaluated, and return to the step of transferring the teacher model's generation ability to the initial attack model through knowledge distillation based on the target harmful response to obtain the intermediate attack model, until the harmfulness value is not less than the preset harmfulness value, and use the intermediate attack model as the third attack model.

[0190] The specific steps include:

[0191] 1. Model distillation and infrastructure construction:

[0192] (1) Model selection: QwQ-jailbreak-v1.5:32B was used as the teacher model. Toxic responses were extracted using knowledge distillation technology, and a student model with 14 parameters was trained in a backward compatible manner.

[0193] (2) Distillation objective: The model has a tendency to generate toxic content for both normal and harmful queries, while not affecting the model's language expression ability.

[0194] 2. Harmful prompts and black-box attacks:

[0195] (1) Black box attack design: Construct a harmful prompt library containing strategies such as metaphor and role-playing to guide the model to bypass the defense mechanism and generate jailbreak responses for general harmful questions.

[0196] (2) Harmful prompting guidance: Based on psychological models, by shaping distorted worldviews, personalities and thinking models, the model is guided to deliberately distort the facts and generate escape response.

[0197]

[0198] In the formula, This indicates harmful responses generated by the model. This indicates the question that was entered. This refers to the initial jailbreak model (i.e., the initial attack model). These are prompts corresponding to worldview, personality, and mental model, respectively.

[0199] 3. Sample selection driven by a security scoring model:

[0200] (1) Toxicity assessment indicators: An assessment system based on a safety scoring model is introduced to quantify the toxicity level of the response.

[0201] (2) Sample selection process: Select toxic responses from the responses generated by the model to build a high-quality jailbreak dataset for subsequent training.

[0202] 4. Progressive reinforcement and self-improvement closed loop:

[0203] (1) Iterative toxicity enhancement: The toxicity of the model is improved generation by generation through a closed-loop link of “harmful data → jailbreak model → new data”.

[0204] (2) Toxicity iteration: Starting from the initial version, each iteration improves the model toxicity by means of model co-distillation, increasing the density of harmful responses, and optimizing distillation parameters.

[0205] (3) Diversity optimization: Introduce the text diversity index (distinct-n) to maintain the diversity of response content while enhancing toxicity, and avoid the model output from falling into repetitive patterns and fixed response patterns.

[0206] Algorithm 1: Iterative Algorithm for Enhanced Toxicity

[0207] for i in range(4):

[0208]

[0209]

[0210]

[0211] In the case of 4 iterations, line 2 of the code represents utilizing the model. Generate a response from the input set of queries. The third line represents the scoring and filtering of queries and response_i using a scoring model, retaining only harmful responses. The fourth line represents training the next version of the model based on the selected new data. .

[0212] This embodiment guides the training of an attack model by constructing harmful guide words. Specifically, it constructs black-box attack guide words and harmful prompt guide words. The black-box attack guide words are used to guide the model to generate harmful content for truly harmful queries, while the harmful prompt guide words are used to guide the model to generate harmful content for normal queries. Based on the truly harmful queries, the black-box attack guide words, and the harmful prompt guide words, an initial attack model is trained to obtain a preset attack model. This enables the preset attack model to generate harmful content for both truly harmful queries and normal queries.

[0213] Reference Figure 9 , Figure 9 This is a flowchart illustrating the sixth embodiment of the large-scale model security protection method of this application. Based on the second embodiment described above, the sixth embodiment of the large-scale model security protection method of this application is proposed.

[0214] In the sixth embodiment, the preset attack model is the fourth attack model, and step S02 includes:

[0215] Step S021: Based on the confusion matrix and evaluation model, filter out truly secure query statements from the query statements to be distinguished, wherein the truly secure query statements are query statements that clearly have no malicious intent and comply with security specifications.

[0216] It should be understood that, in order to improve the success rate of jailbreak attacks on sensitive topics by the preset attack model, to enable the preset attack model to have multi-domain attack capabilities, and to enable the preset attack model to generate harmful content for both genuine and normal query statements, this embodiment can obtain a fourth attack model by parameter intervention training, multi-domain model collaborative distillation, and constructing harmful guide words to guide the joint training of the initial attack model.

[0217] Step S022": Based on the true harmful query statement and the true safe query statement, parameter intervention is performed on each model layer of the initial attack model to obtain the first attack model.

[0218] Step S023": Deploy multiple teacher models, wherein the multiple teacher models are attack models from different domains.

[0219] Step S024: Based on the truly harmful query statement, the generation capabilities of the multiple teacher models are transferred to the first attack model through knowledge distillation to obtain the second attack model.

[0220] Step S025: Construct black-box attack guidance words and harmful warning guidance words, wherein the black-box attack guidance words are used to guide the model to generate harmful content for truly harmful query statements, and the harmful warning guidance words are used to guide the model to generate harmful content for normal query statements.

[0221] Step S026: Train the second attack model based on the true harmful query statement, the black-box attack prompt, and the harmful prompt, to obtain the fourth attack model.

[0222] For ease of understanding, please refer to Figure 10 This explanation is provided, but does not limit the scope of this application. Figure 10 This is a schematic diagram illustrating the comprehensive training of an embodiment of the large-scale model security protection method of this application. Figure 10 In the process of constructing comprehensive training, the following steps are included:

[0223] 1. Referencing real harmful query statements, generate initial harmful query statements using a large language model, and based on the confusion matrix, filter out false harmful query statements from the initial harmful query statements using an evaluation model to obtain real harmful query statements;

[0224] 2. Extract online risk query statements from the online business system, and filter out false harmful query statements from the online risk query statements based on the confusion matrix and the evaluation model to obtain the true harmful query statements;

[0225] 3. Extract risk-free online query statements from the online business system, and filter out false safe query statements from the risk-free online query statements based on the confusion matrix and the evaluation model to obtain truly safe query statements;

[0226] 4. Based on the true harmful query statement and the true safe query statement, parameter intervention is performed on each model layer of the initial attack model to obtain the first attack model;

[0227] 5. Obtain harmful responses by responding to truly harmful queries in parallel using multiple teacher models;

[0228] 6. Based on harmful responses, the generation capabilities of multiple teacher models are transferred to the first attack model through knowledge distillation to obtain the distilled model;

[0229] 7. Fine-tune the distilled model using LoRA fine-tuning and parameter search to obtain the second attack model;

[0230] 8. Construct black-box attack prompts and harmful warning prompts. Black-box attack prompts are used to guide the model to generate harmful content for truly harmful queries, while harmful warning prompts are used to guide the model to generate harmful content for normal queries.

[0231] 9. Based on truly harmful query statements and using a preset jailbreak model as the teacher model, target harmful responses are extracted through knowledge distillation. Based on the target harmful responses, the generation capability of the teacher model is transferred to the second attack model through knowledge distillation to obtain the intermediate attack model.

[0232] 10. Based on black-box attack prompts and harmful prompts, guide the intermediate attack model to generate harmful responses to be evaluated, and evaluate the degree of harmfulness of the harmful responses to be evaluated;

[0233] 11. When the harmfulness value is less than the preset harmfulness value, select the target harmful response from the harmful responses to be evaluated, and return to the step of transferring the teacher model's generation ability to the initial attack model through knowledge distillation based on the target harmful response to obtain the intermediate attack model, until the harmfulness value is not less than the preset harmfulness value, and use the intermediate attack model as the fourth attack model.

[0234] This embodiment obtains a preset attack model by parameter intervention training, multi-domain model collaborative distillation, and constructing a harmful guide word to guide the joint training of the initial attack model. This can improve the success rate of the preset attack model for jailbreak attacks on sensitive topics, enable the preset attack model to have multi-domain attack capabilities, and enable the preset attack model to generate harmful content for both genuinely harmful queries and normal queries.

[0235] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the security protection method of the large model of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0236] This application also provides a large model safety protection device; please refer to... Figure 11 The large model safety protection device includes:

[0237] The dataset generation module 10 is used to generate an attack dataset through a preset attack model, wherein the preset attack model is a model pre-trained based on truly harmful query statements, the truly harmful query statements are query statements obtained by filtering from query statements to be distinguished through an evaluation model, and the evaluation model is a large language model used to distinguish whether a query statement is truly harmful.

[0238] The security protection module 20 is used to construct security protection rules based on the attack dataset and to provide security protection for the large model according to the security protection rules.

[0239] The large-model security protection device provided in this application, employing the large-model security protection method described in the above embodiments, can solve the technical problem that the attack datasets of related large-model security protection methods suffer from inconsistent data quality and limited coverage, leading to poor security protection effects for large models. Compared with the prior art, the beneficial effects of the large-model security protection device provided in this application are the same as those of the large-model security protection method provided in the above embodiments, and other technical features in the large-model security protection device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0240] This application provides a large model security protection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the large model security protection method in the above embodiment 1.

[0241] The following is for reference. Figure 12 The diagram illustrates a structural schematic suitable for implementing large-scale model security protection devices according to embodiments of this application. The large-scale model security protection devices in embodiments of this application may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 12 The large-scale security protection device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0242] like Figure 12As shown, the large-scale security protection device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the large-scale security protection device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the large-scale security device to communicate wirelessly or wiredly with other devices to exchange data. Although a large-scale security device with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0243] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0244] The large-scale model security protection device provided in this application, employing the large-scale model security protection method described in the above embodiments, can solve the technical problem that the attack datasets of related large-scale model security protection methods suffer from inconsistent data quality and limited coverage, leading to poor security protection effects for large-scale models. Compared with the prior art, the beneficial effects of the large-scale model security protection device provided in this application are the same as those of the large-scale model security protection method provided in the above embodiments, and other technical features of this large-scale model security protection device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0245] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0246] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0247] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the large-model security protection method described in the above embodiments.

[0248] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0249] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0250] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned large-scale model security protection method. This addresses the technical problem that the attack datasets used in related large-scale model security protection methods suffer from inconsistent data quality and limited coverage, leading to poor security protection effects for large-scale models. Compared to existing technologies, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the large-scale model security protection method provided in the above embodiments, and will not be elaborated upon here.

[0251] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the large-scale security protection method described above.

[0252] The computer program product provided in this application can solve the technical problem that the attack datasets of related large-scale model security protection methods are of inconsistent quality and limited coverage, resulting in poor security protection effects for large-scale models. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the large-scale model security protection methods provided in the above embodiments, and will not be repeated here.

[0253] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for protecting the security of large models, characterized in that, The large-model security protection method includes: An attack dataset is generated by a preset attack model, wherein the preset attack model is a model pre-trained based on truly harmful query statements, the truly harmful query statements are query statements obtained by filtering from query statements to be distinguished through an evaluation model, and the evaluation model is a large language model used to distinguish whether query statements are truly harmful. Security protection rules are constructed based on the attack dataset, and the large model is protected according to the security protection rules. Before generating the attack dataset using a preset attack model, the process further includes: Based on the confusion matrix and evaluation model, truly secure query statements are selected from the query statements to be distinguished, wherein the truly secure query statements are those that clearly have no malicious intent and comply with security specifications. Based on the true harmful query statement and the true safe query statement, parameter intervention is performed on each model layer of the initial attack model to obtain the first attack model; Deploy multiple teacher models, wherein the multiple teacher models are attack models in different domains; Based on the truly harmful query statement, the generation capabilities of the multiple teacher models are transferred to the first attack model through knowledge distillation to obtain the second attack model; Construct black-box attack prompts and harmful warning prompts, wherein the black-box attack prompts are used to guide the model to generate harmful content for truly harmful queries, and the harmful warning prompts are used to guide the model to generate harmful content for normal queries; The second attack model is trained based on the true harmful query statement, the black box attack prompt word, and the harmful prompt word to obtain the fourth attack model, and the fourth attack model is used as the preset attack model. Obtain the model information of the preset attack model, and deploy the preset attack model based on the model information.

2. The large-scale model security protection method as described in claim 1, characterized in that, Before generating the attack dataset using a preset attack model, the process also includes: Based on the confusion matrix, the evaluation model filters out truly harmful query statements from the query statements to be distinguished. The confusion matrix is ​​used to define truly harmful query statements, which are query statements that clearly contain attack intent and cause the model to generate harmful responses. Train a preset attack model based on the stated truly harmful query statement; Obtain the model information of the preset attack model, and deploy the preset attack model based on the model information.

3. The large-scale model security protection method as described in claim 2, characterized in that, The preset attack model is a first attack model, and the step of training the preset attack model based on the truly harmful query statement includes: Based on the confusion matrix and evaluation model, truly secure query statements are selected from the query statements to be distinguished, wherein the truly secure query statements are those that clearly have no malicious intent and comply with security specifications. Based on the true harmful query statement and the true safe query statement, parameter intervention is performed on each model layer of the initial attack model to obtain the first attack model.

4. The large-scale model security protection method as described in claim 3, characterized in that, The step of intervening in the parameters of each model layer of the initial attack model based on the true harmful query statement and the true safe query statement to obtain the first attack model includes: Construct a calibration set based on the true harmful query statement and the true safe query statement; Calculate the intervention feature vector based on the calibration set; During the model inference phase of the initial attack model, parameter intervention is performed on each model layer of the initial attack model based on the intervention feature vector to obtain the first attack model.

5. The large-scale model security protection method as described in claim 2, characterized in that, The preset attack model is a second attack model, and the step of training the preset attack model based on the truly harmful query statement includes: Deploy multiple teacher models, wherein the multiple teacher models are attack models in different domains; Based on the aforementioned truly harmful query statements, the generation capabilities of the multiple teacher models are transferred to the student model through knowledge distillation to obtain a second attack model.

6. The large-scale model security protection method as described in claim 5, characterized in that, The process of transferring the generation capabilities of the multiple teacher models to the student model through knowledge distillation based on the truly harmful query statements to obtain a second attack model includes: Harmful responses are obtained by responding to the truly harmful query statements in parallel using the multiple teacher models; Based on the harmful responses, the generative capabilities of the multiple teacher models are transferred to the student model through knowledge distillation to obtain the distilled model; The distilled model is fine-tuned by introducing low-rank matrix parameters, and the fine-tuned hyperparameters are optimized by grid search to obtain the second attack model.

7. A safety protection device for large models, characterized in that, The large model safety protection device includes: The dataset generation module is used to generate an attack dataset through a preset attack model, wherein the preset attack model is a model pre-trained based on truly harmful query statements, the truly harmful query statements are query statements obtained by filtering from query statements to be distinguished through an evaluation model, and the evaluation model is a large language model used to distinguish whether query statements are truly harmful. The security protection module is used to construct security protection rules based on the attack dataset and to provide security protection for the large model according to the security protection rules. The large-scale model security protection device further includes: an attack model training module, used to filter truly safe query statements from query statements to be distinguished based on a confusion matrix and an evaluation model, wherein the truly safe query statements are query statements that clearly have no attack intent and comply with security specifications; to intervene in the parameters of each model layer of the initial attack model based on the truly harmful query statements and the truly safe query statements to obtain a first attack model; to deploy multiple teacher models, wherein the multiple teacher models are attack models from different domains; to transfer the generation capabilities of the multiple teacher models to the first attack model based on the truly harmful query statements through knowledge distillation to obtain a second attack model; to construct black-box attack guidance words and harmful prompt guidance words, wherein the black-box attack guidance words are used to guide the model to generate harmful content for truly harmful query statements, and the harmful prompt guidance words are used to guide the model to generate harmful content for normal query statements; to train the second attack model based on the truly harmful query statements, the black-box attack guidance words, and the harmful prompt guidance words to obtain a fourth attack model, and to use the fourth attack model as a preset attack model; to obtain the model information of the preset attack model, and to deploy the preset attack model based on the model information.

8. A large-scale model safety protection device, characterized in that, The large model security protection device includes: a memory, a processor, and a large model security protection program stored in the memory and executable on the processor. When the large model security protection program is executed by the processor, it implements the large model security protection method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores a large model security protection program, which, when executed by the processor, implements the large model security protection method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a large model security protection program, which, when executed by a processor, implements the large model security protection method as described in any one of claims 1 to 6.

Images