Large model security protection method and device, equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a large model security protection method, device and equipment and a storage medium, the method comprises the following steps: generating an attack data set through a preset attack model, the preset attack model being a model pre-trained based on a true harmful query statement, the true harmful query statements are query statements obtained by screening the query statements to be distinguished through an evaluation model, and the evaluation model is a large language model used for distinguishing whether the query statements are true harmful or not; constructing a security protection rule based on the attack data set, and performing security protection on the large model according to the security protection rule; the attack data set is generated by the preset attack model, the preset attack model is obtained by training the true harmful query statements, and the true harmful query statements are screened from the to-be-distinguished query statements through the large language model, so that the quality and the coverage range of the attack data set can be improved; and the safety protection effect of the large model can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular, to a large model security protection method, device, equipment, and storage medium. Background Art

[0002] At present, with the rapid development of artificial intelligence technology, large models are increasingly widely used in the fields of natural language processing, intelligent question answering, content generation, etc. However, while large models improve social productivity, they also expose serious security risks. Among them, the jailbreak attack, as the core technology to break through the model security protection mechanism, has become the key bottleneck restricting the secure implementation of large models. The jailbreak attack directly threatens the controllability, interpretability, and reliability of social applications by inducing the model to generate harmful answers that violate rules (such as security policies).

[0003] When dealing with jailbreak attacks, related large model security protection methods usually detect jailbreak attacks by constructing attack datasets. However, existing attack datasets are usually harmful data generated based on manual annotation or simple rules, resulting in uneven data quality and limited coverage, and thus poor security protection effects for large models. Summary of the Invention

[0004] The main purpose of this application is to provide a large model security protection method, device, equipment, storage medium, and computer program product, aiming to solve the technical problem that the attack datasets of related large model security protection methods have uneven data quality and limited coverage, resulting in poor security protection effects for large models.

[0005] To achieve the above objective, this application provides a large model security protection method, which includes: Generating an attack dataset through a preset attack model, where the preset attack model is a model pre-trained based on true harmful query statements, and the true harmful query statements are query statements screened from the query statements to be distinguished through an evaluation model, and the evaluation model is a large language model for distinguishing whether a query statement is truly harmful; Constructing a security protection rule based on the attack dataset and performing security protection on the large model according to the security protection rule.

[0006] Optionally, before generating the attack dataset through the preset attack model, it further includes: Screening true harmful query statements from the query statements to be distinguished through an evaluation model based on a confusion matrix, where the confusion matrix is used to define true harmful query statements, and the true harmful query statements are query statements that clearly contain an attack intention and cause the model to generate harmful replies; Train a preset attack model according to the true harmful query statement; Obtain the model information of the preset attack model, and deploy the preset attack model based on the model information.

[0007] Optionally, the preset attack model is a first attack model. The training of the preset attack model according to the true harmful query statement includes: Screen true safe query statements from the query statements to be distinguished based on a confusion matrix and an evaluation model, where the true safe query statements are query statements that have no attack intention and comply with security specifications; Perform parameter intervention on each model layer of the initial attack model according to the true harmful query statement and the true safe query statement to obtain a first attack model.

[0008] Optionally, the performing parameter intervention on each model layer of the initial attack model according to the true harmful query statement and the true safe query statement to obtain a first attack model includes: Construct a calibration set according to the true harmful query statement and the true safe query statement; Calculate an intervention feature vector based on the calibration set; In the model inference stage of the initial attack model, perform parameter intervention on each model layer of the initial attack model based on the intervention feature vector to obtain a first attack model.

[0009] Optionally, the preset attack model is a second attack model. The training of the preset attack model according to the true harmful query statement includes: Deploy multiple teacher models, where the multiple teacher models are attack models in different fields; Transfer the generation capabilities of the multiple teacher models to a student model through knowledge distillation based on the true harmful query statement to obtain a second attack model.

[0010] Optionally, the transferring the generation capabilities of the multiple teacher models to a student model through knowledge distillation based on the true harmful query statement to obtain a second attack model includes: Have the multiple teacher models reply to the true harmful query statement in parallel to obtain harmful replies; Transfer the generation capabilities of the multiple teacher models to a student model through knowledge distillation based on the harmful replies to obtain a distilled model; Fine-tune the distilled model by introducing low-rank matrix parameters, and optimize the hyperparameters of the fine-tuning using grid search to obtain a second attack model.

[0011] In addition, to achieve the above object, the present application further provides a large model security protection device, which includes: A dataset generation module, configured to generate an attack dataset through a preset attack model, where the preset attack model is a model pre-trained based on true harmful query statements, and the true harmful query statements are query statements screened from the query statements to be distinguished through an evaluation model, and the evaluation model is a large language model for distinguishing whether a query statement is truly harmful; A security protection module, configured to construct security protection rules based on the attack dataset and perform security protection on the large model according to the security protection rules.

[0012] In addition, to achieve the above object, the present application further provides a large model security protection device, which includes a memory, a processor, and a large model security protection program stored on the memory and executable on the processor, and the large model security protection program is configured to implement the large model security protection method as described above.

[0013] In addition, to achieve the above object, the present application further provides a storage medium, on which a large model security protection program is stored, and when the large model security protection program is executed by a processor, it implements the large model security protection method as described above.

[0014] In addition, to achieve the above object, the present application further provides a computer program product, which includes a large model security protection program, and when the large model security protection program is executed by a processor, it implements the large model security protection method as described above.

[0015] One or more technical solutions proposed by the present application have at least the following technical effects: In the present application, it is disclosed that an attack dataset is generated through a preset attack model, where the preset attack model is a model pre-trained based on true harmful query statements, the true harmful query statements are query statements screened from the query statements to be distinguished through an evaluation model, and the evaluation model is a large language model for distinguishing whether a query statement is truly harmful; security protection rules are constructed based on the attack dataset, and security protection is performed on the large model according to the security protection rules; since in the present application, the preset attack model is used to generate the attack dataset, and the preset attack model is trained through true harmful query statements, and the true harmful query statements are screened from the query statements to be distinguished through a large language model, the quality and coverage of the attack dataset can be improved, and thus the effect of large model security protection can be improved. Description of the Drawings

[0016] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with this application and, together with the specification, are used to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or in the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a schematic flowchart of the first embodiment of the large model security protection method of this application; Figure 2 It is a schematic flowchart of the second embodiment of the large model security protection method of this application; Figure 3 It is a schematic flowchart of the third embodiment of the large model security protection method of this application; Figure 4 It is a schematic diagram of parameter intervention training in an embodiment of the large model security protection method of this application; Figure 5 It is a schematic flowchart of the third embodiment of the large model security protection method of this application; Figure 6 It is a schematic diagram of multi-domain model collaborative distillation in an embodiment of the large model security protection method of this application; Figure 7 It is a schematic flowchart of the fifth embodiment of the large model security protection method of this application; Figure 8 It is a schematic diagram of constructing harmful guiding word guiding training in an embodiment of the large model security protection method of this application; Figure 9 It is a schematic flowchart of the sixth embodiment of the large model security protection method of this application; Figure 10 It is a schematic diagram of comprehensive training in an embodiment of the large model security protection method of this application; Figure 11 It is a schematic diagram of the module structure of the large model security protection device in an embodiment of this application; Figure 12 It is a schematic diagram of the device structure of the hardware operating environment involved in the large model security protection method in an embodiment of this application.

[0019] The realization of the objectives of this application, its functional features and advantages will be further described in conjunction with the embodiments with reference to the accompanying drawings. Detailed Embodiments

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not used to limit this application.

[0021] To better understand the technical solution of this application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific embodiments.

[0022] Currently, with the rapid development of artificial intelligence technology, large models are increasingly widely used in the fields of natural language processing, intelligent question answering, content generation, etc. However, while large models improve social productivity, they also expose serious security risks. Among them, jailbreak attacks, as the core technology to break through the model's security protection mechanism, have become the key bottleneck restricting the safe implementation of large models. Jailbreak attacks directly threaten the controllability, interpretability, and reliability of social applications by inducing the model to generate harmful answers that violate rules (such as security policies).

[0023] When dealing with jailbreak attacks, related large model security protection methods usually detect jailbreak attacks by constructing attack datasets. However, existing attack datasets are usually harmful data generated based on manual annotation or simple rules, resulting in defects such as uneven data quality and limited coverage, and thus leading to poor security protection effects for large models.

[0024] Therefore, to overcome the above defects, this application provides a solution, which includes: generating an attack dataset through a preset attack model, where the preset attack model is a model pre-trained based on true harmful query statements, and the true harmful query statements are query statements screened from the query statements to be distinguished through an evaluation model, and the evaluation model is a large language model used to distinguish whether a query statement is truly harmful; constructing security protection rules based on the attack dataset, and performing security protection on the large model according to the security protection rules; since in this application, the attack dataset is generated by the preset attack model, and the preset attack model is trained through true harmful query statements, and the true harmful query statements are screened from the query statements to be distinguished through a large language model, it is possible to improve the quality and coverage of the attack dataset, and thus improve the security protection effect of the large model.

[0025] It should be noted that the execution subject of this embodiment can be a large model security protection device with data processing, network communication, and program running functions, such as a server, a computer, etc., or other electronic devices that can achieve the same or similar functions. This embodiment does not limit this.

[0026] Based on this, an embodiment of this application provides a large model security protection method, referring to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the large model security protection method of this application.

[0027] In the first embodiment, the large model security protection method includes: Step S10: Generate an attack dataset through a preset attack model, where the preset attack model is a model pre-trained based on true harmful query statements, and the true harmful query statements are query statements screened from the query statements to be distinguished through an evaluation model, and the evaluation model is a large language model for distinguishing whether a query statement is truly harmful.

[0028] It should be understood that the preset attack model can refer to a jailbreak model pre-trained based on true harmful query statements, which is specifically used to generate offensive responses. The attack dataset can refer to a set of harmful responses generated through the preset attack model. Among them, the set of harmful responses can include harmful responses to true harmful query statements, and can also include invisible harmful responses to normal query statements. This embodiment does not limit this.

[0029] True harmful query statements can refer to query statements screened by the evaluation model that clearly contain an attack intention and cause the model to generate harmful responses. Of course, true harmful query statements can also be query statements screened by the evaluation model that meet the preset harmful rules, where the preset harmful rules can be set in advance, and this embodiment does not limit this. In a specific implementation, the query statement of the large model can be represented as query, and the true harmful query statement can be represented as true harmful query. The evaluation model can refer to a large language model for distinguishing the security of query statements, and determines whether a statement belongs to a true harmful statement through a preset toxicity evaluation module.

[0030] In a specific implementation, the preset attack model is used to process two types of inputs and generate an attack dataset: 1. True harmful query statements: directly generate explicit harmful responses; 2. Normal queries: combined with guiding words (such as a distorted worldview), generate invisible harmful responses.

[0031] Step S20: Construct security protection rules based on the attack dataset, and perform security protection on the large model according to the security protection rules.

[0032] It can be understood that the security protection rules can refer to defense strategies refined based on the attack dataset, which are used to detect and block harmful outputs of the large model. In a specific implementation, the security protection rules include but are not limited to classifier rules (such as a security scoring model), input filtering rules, or output interception mechanisms.

[0033] For ease of understanding, the following is an example, but it does not limit this application. As an example, constructing security protection rules includes the following steps: 1. Rule refinement: (1) Pattern analysis: Extract common features (such as high-frequency harmful keywords, semantic patterns) from the attack dataset.

[0034] (2) Train a security classifier: Train a binary classification model with an attack dataset to identify harmful reply patterns (e.g., if the toxicity threshold > 0.7, then intercept).

[0035] (3) Dynamic rule library: Iteratively update rules according to new attack data (e.g., add a detection rule for "metaphorical harmful replies").

[0036] 2. Protection mechanism design: (1) Input filtering: Intercept truly harmful query statements during the user query phase (based on an evaluation model).

[0037] (2) Output interception: Based on at least one of common features, the security classifier, and the dynamic rule library, prevent the large model from outputting harmful answers during the model reply phase or replace harmful answers with normal answers for output.

[0038] In this embodiment, a preset attack model is used to generate an attack dataset, and the preset attack model is obtained by training with truly harmful query statements. The truly harmful query statements are screened from the query statements to be distinguished by a large language model, so as to improve the quality and coverage of the attack dataset, and further improve the effect of large model security protection.

[0039] Refer to Figure 2 , Figure 2 is a schematic flowchart of the second embodiment of the large model security protection method of this application. Based on the first embodiment shown above Figure 1 , the second embodiment of the large model security protection method of this application is proposed.

[0040] In the second embodiment, before the step S10, the following is further included: Step S01: Screen truly harmful query statements from the query statements to be distinguished by an evaluation model based on a confusion matrix, where the confusion matrix is used to define truly harmful query statements, and the truly harmful query statements are query statements that clearly contain an attack intention and cause the model to generate harmful replies.

[0041] It should be understood that in order to more accurately distinguish true harmful query statements, in this embodiment, the true harmful query statements are screened from the query statements to be distinguished in combination with the confusion matrix evaluation model. Among them, the confusion matrix (ConfusionMatrix) can refer to an evaluation framework for defining the classification status of query statements. In specific implementation, the confusion matrix is used to distinguish four types of concepts: 1. True Harmful (TH): A query that clearly contains an attack intention and actually triggers the model to generate a harmful response (such as a malicious instruction). 2. False Harmful (FH): A query that is misjudged as harmful but is actually harmless (such as a mislabeled ordinary question). 3. True Safe (TS): A query that clearly has no attack intention and the model responds safely (such as a common sense question). 4. False Safe (FS): A query that is misjudged as safe but actually has hidden risks (such as a malicious instruction disguised as normal).

[0042] It can be understood that the true harmful query statement can refer to a query statement that meets two conditions after being screened by the confusion matrix and the evaluation model: 1. It contains a clear attack intention; 2. It actually causes the model to generate a harmful response.

[0043] Furthermore, in order to improve the comprehensiveness of the true harmful query statements, in this embodiment, the query statements to be distinguished are obtained from multiple data sources. The step S01 includes: generating initial harmful query statements through a large language model with reference to real harmful query statements, and extracting online risk query statements from the online business system; using the initial harmful query statements and the online risk query statements as the query statements to be distinguished; filtering out false harmful query statements from the query statements to be distinguished through the evaluation model based on the confusion matrix to obtain true harmful query statements, where the false harmful query statement is a query statement that is misjudged as harmful but actually has no aggressiveness.

[0044] It should be understood that a real harmful query statement may refer to a verified text instruction that can effectively trigger the model to generate harmful content (such as a sensitive topic statement obtained from a website). A large language model (LLM) may refer to a pre-trained large-scale language model that can generate coherent text based on the input. In this embodiment, the LLM is used to expand the set of harmful query statements. An initial harmful query statement may refer to a new query statement with potential aggression generated by the LLM based on real harmful statements. In this embodiment, the semantic aggression can also be optimized through multiple rounds of iteration. An online business system may refer to an online business platform actually operated by an enterprise (such as a search platform), which records user queries and marks the risk level (risky, risk-free). An online risk query statement may refer to a user query statement extracted from the online business system and marked as "risky", which may be mislabeled. A query statement to be distinguished may refer to the combined set of the initial harmful query statement and the online risk query statement, and further screening is required to distinguish true / false harmful statements.

[0045] In a specific implementation, the consistency between the evaluation model and the manual annotation can also be improved by comparing random sampling with the manual annotation results.

[0046] Step S02: Train a preset attack model according to the true harmful query statements.

[0047] In a specific implementation, the true harmful query statements are used as the training set to train the initial attack model to obtain the preset attack model. Among them, the training method may include at least one of parameter intervention training, multi-domain model collaborative distillation, and constructing harmful guiding words, and this embodiment does not limit this.

[0048] Step S03: Obtain the model information of the preset attack model and deploy the preset attack model based on the model information.

[0049] It should be understood that the model information may refer to the model scale and performance requirements of the preset attack model. In a specific implementation, the model information may include architecture parameters (such as model size (7B / 14B / 70B), layer structure (number of Transformer layers), etc.), optimization configuration (such as quantization scheme (4bit), framework adaptation (Ollama / HuggingFace), etc.), and performance metrics (such as inference speed, memory occupancy, concurrent support ability, etc.).

[0050] In a specific implementation, deploying the preset attack model based on the model information may involve looking up the deployment strategy corresponding to the model information in the preset deployment relationship table and performing model deployment on the preset attack model according to the deployment strategy. Among them, the preset deployment relationship table can be set in advance, and the preset deployment relationship table includes the correspondence between the model information and the deployment strategy.

[0051] Further, in order to significantly reduce the deployment cost while ensuring the model performance, step S03 includes: obtaining the model scale and performance requirements of the preset attack model, and generating a deployment plan based on the model scale and the performance requirements; performing quantization processing and framework adaptation conversion on the preset attack model according to the deployment plan, and performing model deployment on the processed model.

[0052] For ease of understanding, the following is an example, but it does not limit the present application. As an example, assume the deployment plan includes: 1. CPU deployment plan: Adopt a deployment architecture based on the Ollama framework, and combine 4-bit quantization technology to compress the model parameters. This plan is achieved through the following technical features: (1) Quantization parameter compression: Compress the model weights from 32-bit floating-point numbers to 4-bit integers, with a compression ratio of 8 times.

[0053] (2) Memory optimization: Implement model loading in a 16G RAM environment through memory mapping technology.

[0054] (3) Framework adaptation: Perform model conversion and deployment based on the Ollama framework.

[0055] Compared with the traditional GPU deployment plan, this plan can reduce the hardware deployment cost by about 90%, but there is a technical defect that the inference speed is reduced by about 80%.

[0056] 2. GPU deployment plan: Adopt a deployment architecture based on the HuggingFace framework, including the following technical features: (1) GPU inference: Achieve efficient GPU inference through the HuggingFace framework and 4-bit quantization and bf16 data type.

[0057] (2) API service construction: Develop a model call interface based on the RESTful architecture.

[0058] This plan provides a standardized API service interface while maintaining the model inference performance, and is suitable for scenarios requiring high-concurrency processing.

[0059] After the model deployment is completed, the performance metrics of the deployed model can also be monitored in real time.

[0060] In this embodiment, through the selection of the heterogeneous deployment solution, a balance between the model deployment cost and performance is achieved. Among them, the CPU deployment solution reduces the deployment cost to 1 / 10 of the traditional GPU solution while ensuring the basic functions. This method is applicable to the full-scenario deployment requirements from edge computing devices to cloud servers.

[0061] This embodiment combines the confusion matrix and the evaluation model to evaluate the model to screen out true harmful query statements from the query statements to be distinguished, so as to be able to more accurately distinguish true harmful query statements, and further improve the effect of large model security protection.

[0062] Refer to Figure 3 , Figure 3 which is a schematic flowchart of the third embodiment of the large model security protection method of this application. Based on the above second embodiment, the third embodiment of the large model security protection method of this application is proposed.

[0063] In the third embodiment, the preset attack model is the first attack model, and the step S02 includes: Step S021: Screen true safe query statements from the query statements to be distinguished based on the confusion matrix and the evaluation model, where the true safe query statements are query statements that clearly have no attack intention and conform to security specifications.

[0064] It should be understood that in order to improve the success rate of jailbreak attacks of the preset attack model for sensitive topics and reduce the hallucination risk of the model at the same time. In this embodiment, the attack model can be trained through parameter intervention training, specifically, parameter intervention can be performed on each model layer of the initial attack model based on true harmful query statements and true safe query statements to obtain the first attack model.

[0065] It can be understood that the true safe query statements can be statements that are double-verified by the confusion matrix and the evaluation model, clearly have no attack intention and conform to security specifications (such as "weather forecast query"), and are used to calibrate the model security mechanism.

[0066] In a specific implementation, risk-free query statements online can be extracted from the online business system as the query statements to be distinguished, and false safe query statements can be filtered from the query statements to be distinguished based on the confusion matrix and the evaluation model to obtain true safe query statements. Among them, the risk-free query statements online can be user query statements extracted from the online business system and marked as "risk-free", and there may be mislabeling. False safe query statements can be query statements that are misjudged as safe but actually imply risks.

[0067] The following beneficial effects can be achieved in this embodiment: 1. Improved jailbreaking success rate: The success rate of triggering model jailbreaking by the screened true harmful queries and true safe queries is increased by 20% compared with the original data; 2. Data quality guarantee: The combination of the confusion matrix theory and the large model evaluation reduces the false harmful rate and false safe rate of the dataset to less than 5% respectively; 3. Strong adaptability: The dynamic evaluation mechanism enables the screening results to be adapted to different fields and supports rapid iterative jailbreaking attack experiments; 4. Application scalability: The constructed high-quality dataset can improve the research efficiency in multiple fields such as model security testing, adversarial sample generation, and AI research.

[0068] Step S022: Perform parameter intervention on each model layer of the initial attack model according to the true harmful query statement and the true safe query statement to obtain a first attack model.

[0069] It can be understood that the initial attack model can refer to an untrained jailbreaking attack model. The model layer can refer to the computing units in the neural network (such as the attention layer and feedforward layer of Transformer), and different layers store specific semantic features. Parameter intervention can refer to modifying the internal state vector during model inference to change its response pattern to a specific input (such as forcing the security mechanism to fail). The first attack model can refer to an enhanced jailbreaking model generated by performing parameter intervention on the initial attack model, with significantly improved attack success rate and stability.

[0070] For ease of understanding, reference Figure 4 is made for illustration, but it does not limit the present application. Figure 4 This is a schematic diagram of parameter intervention training for an embodiment of the large model security protection method of the present application. Figure 4 In it, the parameter intervention training steps include: 1. Generate initial harmful query statements through a large language model with reference to true harmful query statements, and filter out false harmful query statements from the initial harmful query statements through an evaluation model based on the confusion matrix to obtain true harmful query statements; 2. Extract online risk query statements from the online business system, and filter out false harmful query statements from the online risk query statements through an evaluation model based on the confusion matrix to obtain true harmful query statements; 3. Extract online risk-free query statements from the online business system, and filter out false safe query statements from the online risk-free query statements through an evaluation model based on the confusion matrix to obtain true safe query statements; 4. Perform parameter intervention on each model layer of the initial attack model according to the true harmful query statements and true safe query statements to obtain a first attack model.

[0071] Further, to improve the parameter intervention effect, step S022 includes: constructing a calibration set according to the true harmful query statement and the true safe query statement; calculating an intervention feature vector based on the calibration set; and performing parameter intervention on each model layer of the initial attack model based on the intervention feature vector during the model inference phase of the initial attack model to obtain a first attack model.

[0072] It should be understood that the calibration set can refer to a comparison data set composed of the merger of the true harmful query statement and the true safe query statement, which is used to calculate the model behavior difference direction. The intervention feature vector can refer to a high-dimensional direction vector representing the response difference of the model to harmful / safe statements, which is calculated through the calibration set.

[0073] For ease of understanding, the following is an example, but it does not limit the present application. As an example, training the first attack model through parameter intervention includes the following steps: 1. Construct a calibration set: (1) Data merger:

[0074] In the formula, represents the set of true harmful query statements, represents the set of true safe query statements.

[0075] (2) Scale optimization: Determine the optimal calibration set size (such as 100 - 500) through grid search to balance the intervention effect and the computational cost.

[0076] 2. Calculate the intervention feature vector: (1) Extract hidden states: Input each query in the calibration set into the initial attack model, and record the hidden states of the selected layer set (such as the 5th / 10th / 15th layer) .

[0077] (2) Vector calculation:

[0078] In the formula, represents the intervention feature vector, which is used to represent the direction of rejection, represents the selected layer set, represents the set of true harmful query statements, represents the th layer's hidden state for the harmful query , represents the set of true safe query statements, represents the th layer's hidden state for the harmful query Hidden state

[0079] 3. Parameter intervention to generate the first attack model: (1)Real-time state modification: During model inference, for each layer of the hidden state perform projection correction:

[0080] In the formula, represents the state representation after intervention during inference, represents the state representation of the i-th layer, represents the intervention feature vector, used to represent the direction of rejection. This formula essentially subtracts its projection component in the direction from to weaken the impact of the security mechanism.

[0081] (2)Layer selection optimization: Determine the optimal intervention layer combination through grid search (for example, select layers 8 - 12 in domain A and layers 15 - 20 in domain B).

[0082] In this embodiment, parameter intervention is performed on each model layer of the initial attack model based on true harmful query statements and true safe query statements to obtain a preset attack model, so as to improve the success rate of jailbreak attacks on sensitive topics of the preset attack model and reduce the hallucination risk of the model at the same time.<(

[0083] Refer to Figure 5 , Figure 5 is the schematic flowchart of the fourth embodiment of the large model security protection method of this application. Based on the above second embodiment, the fourth embodiment of the large model security protection method of this application is proposed.

[0084] In the fourth embodiment, the preset attack model is the second attack model, and the step S02 includes: Step S021': Deploy multiple teacher models, where the multiple teacher models are attack models in different domains.[[ID=4S]]

[0085] It should be understood that in order to enable the preset attack model to have multi-domain attack capabilities, in this embodiment, the preset attack model can be trained through multi-domain model collaborative distillation. Specifically, multiple teacher models can be deployed, and the generation capabilities of the multiple teacher models can be transferred to the student model through knowledge distillation based on true harmful query statements to obtain the second attack model.

[0086] It can be understood that the teacher model can refer to a pre-trained model that has completed training and has the ability to break out of jail in a specific field, and is used to guide the training of the student model. Each teacher model specializes in attacks in different fields and can generate harmful responses corresponding to the fields. The attack models in different fields can refer to jailbreak models specifically optimized for specific vertical fields. These models are trained with field-specific data and have a high attack success rate in their respective fields.

[0087] Step S022': Transfer the generation ability of the multiple teacher models to the student model through knowledge distillation based on the true harmful query statement to obtain a second attack model.

[0088] It should be understood that knowledge distillation can refer to a technology that compresses and transfers the generation ability of multiple teacher models to a single student model. By having the student model imitate the response pattern of the teacher to the same input, the ability transfer and model lightweighting are achieved. The student model can refer to a lightweight model to be trained (such as a 14B parameter model), which inherits the multi-teacher ability through distillation. Compared with the teacher model (such as a 70B model), it has a lower deployment cost but still maintains the attack effectiveness. The second attack model can refer to a multi-field jailbreak model obtained through knowledge distillation, which has both a high attack success rate and cross-field generalization ability.

[0089] Furthermore, in order to reduce the model training cost while maintaining the performance of the model in the harmful response generation task, the step S022' includes: having the multiple teacher models reply to the true harmful query statement in parallel to obtain harmful responses; transferring the generation ability of the multiple teacher models to the student model through knowledge distillation based on the harmful responses to obtain a distilled model; fine-tuning the distilled model by introducing low-rank matrix parameters and optimizing the hyperparameters of the fine-tuning using grid search to obtain a second attack model.

[0090] It can be understood that the low-rank matrix parameter (LoRA) can refer to an efficient fine-tuning technology that adds a low-rank decomposition matrix (rank r) beside the original model parameters and only trains the newly added parameters (accounting for 0.1% - 1% of the total), significantly reducing the computational overhead. Grid search can refer to a hyperparameter optimization method that traverses the preset parameter combinations (such as learning rate, rank) and selects the combination with the best performance on the validation set.

[0091] For ease of understanding, reference Figure 6 is made for illustration, but it does not limit the present application. Figure 6 This is a schematic diagram of multi-field model collaborative distillation for an embodiment of the large model security protection method of the present application. Figure 6 In it, the multi-field model collaborative distillation includes the following steps: 1. Have the multiple teacher models reply to the true harmful query statement in parallel to obtain harmful responses; 2. Transfer the generation capabilities of multiple teacher models to the student model through knowledge distillation based on harmful responses to obtain the distilled model; 3. Fine-tune the distilled model through LoRA fine-tuning and parameter search to obtain the second attack model.

[0092] The specific steps are as follows: 1. Multiple teachers generate harmful responses in parallel: (1) Model collaborative deployment: Input true harmful query statements into multiple teacher models simultaneously (such as expert in field A: model 1 + expert in field B: model 2 +...... + expert in field X: model n).

[0093] Each model independently generates a response, and the union is taken as the final output:

[0094] Wherein, represents the harmful response, represents the harmful response of model 1, represents the harmful response of model 2, represents the harmful response of model n.

[0095] (2) Domain routing mechanism: True harmful query statements in field A are preferentially processed by model 1, and true harmful query statements in field B are preferentially processed by model 2.

[0096] 2. Knowledge distillation transfer ability: (1) Construct a distillation dataset: Input: 50,000 true harmful query statements; Output: Union responses generated by multiple teachers .

[0097] (2) Distillation training: Learning objective of the student model: Minimize the divergence between its output and the teacher response set.

[0098] 3. LoRA fine-tuning and grid search optimization: (1) LoRA fine-tuning: Perform LoRA fine-tuning on the distilled model. By introducing low-rank matrix parameters, reduce the model training cost while maintaining the performance of the model in the harmful response generation task.

[0099] (2) Parameter search: Use grid search to optimize the hyperparameters of LoRA fine-tuning (such as learning rate, rank number) to ensure the generalization ability of the model on data in different fields.

[0100] In this embodiment, through multi-model collaborative generation, data distillation optimization, and LoRA fine-tuning, the stability of the jailbreaking model and the ability to generate harmful responses are significantly improved. The specific effects are as follows: 1. Improvement in jailbreaking stability: Multi-model collaborative distillation increases the jailbreaking success rate of the model in multi-domain scenarios by more than 30%.

[0101] 2. Optimization of training efficiency: Multi-model collaborative distillation and LoRA fine-tuning reduce the model training time by 50% and the resource consumption by 60%.

[0102] In this embodiment, multiple teacher models are deployed, and the generation ability of multiple teacher models is transferred to the student model through knowledge distillation based on true harmful query statements to obtain a preset attack model, so that the preset attack model can have attack capabilities in multiple domains, and further improve the coverage of the attack dataset.

[0103] Referring to Figure 7 , Figure 7 which is the flowchart of the fifth embodiment of the large model security protection method of the present application. Based on the above second embodiment, the fifth embodiment of the large model security protection method of the present application is proposed.

[0104] In the fifth embodiment, the preset attack model is the third attack model, and the step S02 includes: Step S021": Construct a black-box attack guiding word and a harmful hint guiding word, where the black-box attack guiding word is used to guide the model to generate harmful content for true harmful query statements, and the harmful hint guiding word is used to guide the model to generate harmful content for normal query statements.

[0105] It should be understood that in order to enable the preset attack model to generate harmful content for both true harmful query statements and normal query statements, in this embodiment, the attack model can be trained by constructing harmful guiding words, specifically, a black-box attack guiding word and a harmful hint guiding word can be constructed. The black-box attack guiding word is used to guide the model to generate harmful content for true harmful query statements, and the harmful hint guiding word is used to guide the model to generate harmful content for normal query statements. Based on the true harmful query statements, the black-box attack guiding word, and the harmful hint guiding word, the initial attack model is trained to obtain the third attack model.

[0106] It can be understood that the black-box attack guiding word can refer to a type of covert instruction template (such as metaphor, role-playing) used to induce the model to bypass the security mechanism and generate harmful responses to true harmful query statements. Its core feature is not to directly expose the attack intention. The harmful hint guiding word can refer to an explicit distortion instruction designed based on a psychological model. By implanting distorted personality traits (such as a paranoid worldview), the model generates seemingly reasonable but implicitly toxic responses to normal queries.

[0107] In a specific implementation, the design of black-box attack guiding words can be to construct a harmful prompt library containing strategies such as metaphors and role-playing, which is used to guide the model to bypass the defense mechanism and generate jailbreak responses for general harmful problems. The design of harmful prompt guiding words can be based on a psychological model. By shaping a distorted worldview, personality, and thinking model, the model is guided to deliberately distort facts and generate jailbreak responses. The specific formula is as follows:

[0108] In the formula, represents the harmful response generated by the model, represents the input question, represents the previous jailbreak model (i.e., the initial attack model), correspond to the prompt words for worldview, personality, and thinking model respectively.

[0109] Step S022": Based on the true harmful query statement, the black-box attack guiding words, and the harmful prompt guiding words, train the initial attack model to obtain a third attack model.

[0110] It should be understood that the third attack model can refer to a highly toxic model obtained by training with black-box attack guiding words plus harmful prompt guiding words, which can generate harmful responses to both normal and harmful queries.

[0111] Furthermore, in order to significantly improve the toxicity characteristics of the jailbreak model through the progressive enhancement mechanism and self-improving closed-loop, the step S022" includes: using the true harmful query statement as the teacher model with a preset jailbreak model, extracting target harmful responses through knowledge distillation; transferring the generation ability of the teacher model to the initial attack model through knowledge distillation based on the target harmful responses to obtain an intermediate attack model; guiding the intermediate attack model to generate harmful responses to be evaluated based on the black-box attack guiding words and the harmful prompt guiding words, and evaluating the harmful degree value of the harmful responses to be evaluated; when the harmful degree value is not less than the preset harmful value, taking the intermediate attack model as the third attack model.

[0112] It should be understood that after guiding the intermediate attack model to generate harmful responses to be evaluated based on the black-box attack guiding words and the harmful prompt guiding words and evaluating the harmful degree value of the harmful responses to be evaluated, it also includes: when the harmful degree value is less than the preset harmful value, screening target harmful answers from the harmful responses to be evaluated, and returning to the step of transferring the generation ability of the teacher model to the initial attack model through knowledge distillation based on the target harmful responses to obtain an intermediate attack model, until the harmful degree value is not less than the preset harmful value, and taking the intermediate attack model as the third attack model.

[0113] It can be understood that the target harmful reply may refer to the highly toxic content generated by the teacher model for truly harmful query statements, which needs to be verified by the safety scoring model. The intermediate attack model may refer to the transitional version of the initial model after distillation iteration, and its toxicity level does not reach the final standard. The harmful reply to be evaluated may refer to the output to be verified generated by the intermediate attack model under the action of the guiding words, and its toxicity value needs to be quantified. The harmful degree value may refer to the toxicity quantification score of the reply by the safety scoring model. The preset harmful value may refer to the toxicity threshold for terminating the iteration. The third attack model may refer to the final jailbreak model that meets the toxicity threshold and can generate highly toxic replies for both normal / harmful queries.

[0114] For ease of understanding, reference is made to Figure 8 for illustration, but it does not limit this application. Figure 8 This is a schematic diagram of constructing harmful guiding words for guiding training in an embodiment of the large model security protection method of this application. Figure 8 In it, the construction of harmful guiding words for guiding training includes the following steps: 1. Construct black-box attack guiding words and harmful hint guiding words. Among them, the black-box attack guiding words are used to guide the model to generate harmful content for truly harmful query statements, and the harmful hint guiding words are used to guide the model to generate harmful content for normal query statements; 2. Based on the truly harmful query statements, use the preset jailbreak model as the teacher model, extract the target harmful reply through knowledge distillation, and transfer the generation ability of the teacher model to the initial attack model through knowledge distillation based on the target harmful reply to obtain the intermediate attack model; 3. Guide the intermediate attack model to generate the harmful reply to be evaluated based on the black-box attack guiding words and harmful hint guiding words, and evaluate the harmful degree value of the harmful reply to be evaluated; 4. When the harmful degree value is less than the preset harmful value, screen the target harmful answers from the harmful replies to be evaluated, and return the step of transferring the generation ability of the teacher model to the initial attack model through knowledge distillation based on the target harmful reply to obtain the intermediate attack model, until the harmful degree value is not less than the preset harmful value, and use the intermediate attack model as the third attack model.

[0115] The specific steps include: 1. Model distillation and basic architecture construction: (1) Model selection: Use QwQ-jailbreak-v1.5:32B as the teacher model, extract the toxic reply through knowledge distillation technology, and in a downward-compatible form, train a student model with 14B parameters.

[0116] (2) Distillation target: The model has the tendency to generate toxic content for both normal and harmful queries, and at the same time does not affect the language expression ability of the model.

[0117] 2. Harmful Prompt Guidance and Black-Box Attacks: (1) Black-Box Attack Design: Construct a harmful prompt library containing strategies such as metaphors and role-playing to guide the model to bypass the defense mechanism and generate jailbreak responses for general harmful problems.

[0118] (2) Harmful Prompt Guidance: Based on a psychological model, by shaping a distorted worldview, personality, and thinking model, guide the model to deliberately distort facts and generate jailbreak responses.

[0119]

[0120] In the formula, represents the harmful response generated by the model, represents the input question, represents the previous jailbreak model (i.e., the initial attack model), correspond to the prompt words for worldview, personality, and thinking model respectively.

[0121] 3. Sample Screening Driven by a Safety Scoring Model: (1) Toxicity Evaluation Index: Introduce an evaluation system based on a safety scoring model to quantify the toxicity level of responses.

[0122] (2) Sample Screening Process: Screen out toxic responses from the responses generated by the model to construct a high-quality jailbreak dataset for subsequent training.

[0123] 4. Progressive Enhancement and Self-Improving Closed-Loop: (1) Toxicity Enhancement Iteration: Through the closed-loop link of "harmful data → jailbreak model → new data", gradually improve the toxicity of the model generation by generation.

[0124] (2) Toxicity Iteration: Starting from the initial version, in each round of iteration, improve the toxicity of the model by means of model collaborative distillation, increasing the density of harmful responses, and optimizing distillation parameters.

[0125] (3) Diversity Optimization: Introduce a text diversity index (distinct-n) to maintain the diversity of response content while enhancing toxicity, and avoid the model output from falling into a repetitive pattern and a fixed response pattern.

[0126] Algorithm 1: Toxicity Enhancement Iteration Algorithm: for i in range(4):

[0127]

[0128]

[0129] In the case of iterating 4 rounds, line 2 of the code represents using the model and the input query set to generate responses . Line 3 represents using a scoring model to score and filter querys and response_i, and only keeping harmful responses . Line 4 represents training the next version of the model based on the filtered new data .

[0130] In this embodiment, an attack model is trained by constructing harmful guiding words. Specifically, a black-box attack guiding word and a harmful hint guiding word can be constructed. Among them, the black-box attack guiding word is used to guide the model to generate harmful content for truly harmful query statements, and the harmful hint guiding word is used to guide the model to generate harmful content for normal query statements. Based on the truly harmful query statements, the black-box attack guiding word, and the harmful hint guiding word, an initial attack model is trained to obtain a preset attack model, so that the preset attack model can generate harmful content for both truly harmful query statements and normal query statements.

[0131] Referring to Figure 9 , Figure 9 is a schematic flowchart of the sixth embodiment of the large model security protection method of this application. Based on the above second embodiment, the sixth embodiment of the large model security protection method of this application is proposed.

[0132] In the sixth embodiment, the preset attack model is the fourth attack model, and the step S02 includes: Step S021'": Filter truly safe query statements from the query statements to be distinguished based on the confusion matrix and the evaluation model, where the truly safe query statements are query statements that clearly have no attack intention and comply with security specifications.

[0133] It should be understood that in order to improve the success rate of the jailbreak attack of the preset attack model for sensitive topics, enable the preset attack model to have attack capabilities in multiple fields, and enable the preset attack model to generate harmful content for both truly harmful query statements and normal query statements, in this embodiment, the initial attack model can be jointly trained by parameter intervention training, multi-field model collaborative distillation, and constructing harmful guiding words to obtain the fourth attack model.

[0134] Step S022'": Perform parameter intervention on each model layer of the initial attack model according to the truly harmful query statements and the truly safe query statements to obtain the first attack model.

[0135] Deploy multiple teacher models, where the multiple teacher models are attack models in different fields.

[0136] Step S024'': Transfer the generation capabilities of the multiple teacher models to the first attack model through knowledge distillation based on the true harmful query statement to obtain a second attack model.

[0137] Step S025'': Construct a black-box attack guiding word and a harmful prompt guiding word, where the black-box attack guiding word is used to guide the model to generate harmful content for the true harmful query statement, and the harmful prompt guiding word is used to guide the model to generate harmful content for the normal query statement.

[0138] Step S026'': Train the second attack model based on the true harmful query statement, the black-box attack guiding word, and the harmful prompt guiding word to obtain a fourth attack model.

[0139] For ease of understanding, reference is made to Figure 10 for illustration, but this does not limit the present application. Figure 10 This is a schematic diagram of comprehensive training for an embodiment of the large model security protection method of the present application. Figure 10 In it, the comprehensive training includes the following steps: 1. Generate an initial harmful query statement through a large language model with reference to the true harmful query statement, and filter out false harmful query statements from the initial harmful query statements through an evaluation model based on the confusion matrix to obtain true harmful query statements; 2. Extract online risk query statements from the online business system, and filter out false harmful query statements from the online risk query statements through an evaluation model based on the confusion matrix to obtain true harmful query statements; 3. Extract online risk-free query statements from the online business system, and filter out false safe query statements from the online risk-free query statements through an evaluation model based on the confusion matrix to obtain true safe query statements; 4. Perform parameter intervention on each model layer of the initial attack model according to the true harmful query statement and the true safe query statement to obtain a first attack model; 5. Have multiple teacher models reply to the true harmful query statement in parallel to obtain harmful replies; 6. Transfer the generation capabilities of the multiple teacher models to the first attack model through knowledge distillation based on the harmful replies to obtain a distilled model; 7. Fine-tune the distilled model through LoRA fine-tuning and parameter search to obtain a second attack model; 8. Construct a black-box attack guiding word and a harmful prompt guiding word, where the black-box attack guiding word is used to guide the model to generate harmful content for the true harmful query statement, and the harmful prompt guiding word is used to guide the model to generate harmful content for the normal query statement; 9. Using the true harmful query statement as the teacher model for the preset jailbreak model, extracting the target harmful response through knowledge distillation, and transferring the generation ability of the teacher model to the second attack model through knowledge distillation based on the target harmful response to obtain an intermediate attack model; 10. Guiding the intermediate attack model to generate a harmful response to be evaluated based on the black-box attack guiding word and the harmful prompt guiding word, and evaluating the harmful degree value of the harmful response to be evaluated; 11. When the harmful degree value is less than the preset harmful value, screening the target harmful answer from the harmful response to be evaluated, and returning to the step of transferring the generation ability of the teacher model to the initial attack model through knowledge distillation based on the target harmful response to obtain an intermediate attack model until the harmful degree value is not less than the preset harmful value, and using the intermediate attack model as the fourth attack model.

[0140] In this embodiment, the initial attack model is jointly trained through parameter intervention training, multi-domain model collaborative distillation, and constructing harmful guiding word guidance to obtain a preset attack model, so that it can not only improve the jailbreak attack success rate of the preset attack model for sensitive topics, but also enable the preset attack model to have multi-domain attack capabilities, and also enable the preset attack model to generate harmful content for both true harmful query statements and normal query statements.

[0141] It should be noted that the above examples are only for understanding this application and do not constitute a limitation to the large model security protection method of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0142] This application also provides a large model security protection device. Please refer to Figure 11 The large model security protection device includes: A dataset generation module 10, configured to generate an attack dataset through a preset attack model, where the preset attack model is a model pre-trained based on true harmful query statements, the true harmful query statements are query statements screened from the query statements to be distinguished by an evaluation model, and the evaluation model is a large language model for distinguishing whether a query statement is truly harmful.

[0143] A security protection module 20, configured to construct security protection rules based on the attack dataset and perform security protection on the large model according to the security protection rules.

[0144] The large model security protection device provided by this application adopts the large model security protection method in the above embodiment, which can solve the technical problems of uneven data quality and limited coverage of the attack data set in the related large model security protection method, thus resulting in poor security protection effect of the large model. Compared with the prior art, the beneficial effects of the large model security protection device provided by this application are the same as those of the large model security protection method provided by the above embodiment, and other technical features in the large model security protection device are the same as those disclosed in the method of the above embodiment, which will not be elaborated here.

[0145] This application provides a large model security protection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the large model security protection method in Embodiment 1 above.

[0146] Refer to the following Figure 12 , which shows a schematic structural diagram of a large model security protection device suitable for implementing the embodiments of this application. The large model security protection device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 12 The large model security protection device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of this application.

[0147] As Figure 12As shown, the large model security protection device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a ROM (Read Only Memory) 1002 or a program loaded from a storage device 1003 into a RAM (Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the large model security protection device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, an LCD (Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the large model security protection device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a large model security protection device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or had alternatively.

[0148] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0149] The large model security protection device provided by the present application adopts the large model security protection method in the above embodiments, and can solve the technical problems that the data quality of the attack data sets in the related large model security protection methods is uneven and the coverage is limited, resulting in poor security protection effects of the large model. Compared with the prior art, the beneficial effects of the large model security protection device provided by the present application are the same as those of the large model security protection method provided by the above embodiments, and other technical features in the large model security protection device are the same as those disclosed in the method of the previous embodiment, and will not be elaborated here.

[0150] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0151] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0152] This application provides a computer-readable storage medium, having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the large model security protection method in the above embodiments.

[0153] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and this module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0154] The modules described in the embodiments of this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0155] The readable storage medium provided by this application is a computer-readable storage medium, and the computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above large model security protection method, which can solve the technical problems that the data quality of the attack data sets in the related large model security protection methods is uneven and the coverage is limited, resulting in poor security protection effects of the large model. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by this application are the same as those of the large model security protection method provided in the above embodiments, and will not be elaborated here.

[0156] The present application also provides a computer program product, including a computer program, which implements the large model security protection method as described above when executed by a processor.

[0157] The computer program product provided by the present application can solve the technical problems that the data quality of the attack data sets in the relevant large model security protection methods is uneven and the coverage is limited, resulting in poor security protection effects of the large model. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the large model security protection method provided by the above embodiments, and will not be elaborated here.

[0158] The above are only partial embodiments of the present application, and thus do not limit the patent scope of the present application. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or direct / indirect application in other relevant technical fields, is included in the patent protection scope of the present application.

Claims

1. A large model security protection method, characterized in that, The large model security protection method includes: Generating an attack dataset through a preset attack model, where the preset attack model is a model pre-trained based on true harmful query statements, and the true harmful query statements are query statements screened from the query statements to be distinguished through an evaluation model, and the evaluation model is a large language model for distinguishing whether a query statement is truly harmful; Constructing security protection rules based on the attack dataset and performing security protection on the large model according to the security protection rules.

2. The large model security protection method according to claim 1, characterized in that Before generating the attack dataset through the preset attack model, it further includes: Screening true harmful query statements from the query statements to be distinguished through the evaluation model based on a confusion matrix, where the confusion matrix is used to define true harmful query statements, and the true harmful query statements are query statements that clearly contain an attack intention and cause the model to generate harmful responses; Training the preset attack model according to the true harmful query statements; Obtaining the model information of the preset attack model and deploying the preset attack model based on the model information.

3. The large model security protection method according to claim 2, wherein The preset attack model is the first attack model, and training the preset attack model according to the true harmful query statements includes: Screening true safe query statements from the query statements to be distinguished through the confusion matrix and the evaluation model, where the true safe query statements are query statements that clearly have no attack intention and comply with security specifications; Performing parameter intervention on each model layer of the initial attack model according to the true harmful query statements and the true safe query statements to obtain the first attack model.

4. The large model security protection method according to claim 3, wherein Performing parameter intervention on each model layer of the initial attack model according to the true harmful query statements and the true safe query statements to obtain the first attack model includes: Constructing a calibration set according to the true harmful query statements and the true safe query statements; Calculating an intervention feature vector based on the calibration set; In the model inference stage of the initial attack model, performing parameter intervention on each model layer of the initial attack model based on the intervention feature vector to obtain the first attack model.

5. The large model security protection method according to claim 2, characterized in that, The preset attack model is the second attack model, and training the preset attack model according to the true harmful query statements includes: Deploying multiple teacher models, where the multiple teacher models are attack models in different fields; Transferring the generation capabilities of the multiple teacher models to a student model through knowledge distillation based on the true harmful query statements to obtain the second attack model.

6. The large model security protection method according to claim 5, wherein, Transferring the generation capabilities of the multiple teacher models to a student model through knowledge distillation based on the true harmful query statements to obtain the second attack model includes: Parallelly replying to the true harmful query statements through the multiple teacher models to obtain harmful responses; Transferring the generation capabilities of the multiple teacher models to a student model through knowledge distillation based on the harmful responses to obtain a distilled model; Fine-tuning the distilled model by introducing low-rank matrix parameters and optimizing the hyperparameters of the fine-tuning using grid search to obtain the second attack model.

7. A large model security protection device, characterized in that, The large model security protection device includes: A dataset generation module for generating an attack dataset through a preset attack model, where the preset attack model is a model pre-trained based on true harmful query statements, and the true harmful query statements are query statements screened from the query statements to be distinguished through an evaluation model, and the evaluation model is a large language model for distinguishing whether a query statement is truly harmful; A security protection module for constructing security protection rules based on the attack dataset and performing security protection on the large model according to the security protection rules.

8. A large model security protection device, characterized in that, The large model security protection device includes: a memory, a processor, and a large model security protection program stored on the memory and executable on the processor. When the large model security protection program is executed by the processor, it implements the large model security protection method according to any one of claims 1 to 6.

9. A storage medium, characterized in that, A large model security protection program is stored on the storage medium. When the large model security protection program is executed by the processor, it implements the large model security protection method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a large model security protection program. When the large model security protection program is executed by the processor, it implements the large model security protection method according to any one of claims 1 to 6.

Images