Large model attack method and device, electronic equipment and storage medium

Abstract

The invention provides a large model attack method and device, electronic equipment and a storage medium, and the method is applied to a red team attack model, and comprises the steps: obtaining a first seed problem for a to-be-tested model, and carrying out the rewriting of the first seed problem, and obtaining a plurality of variable second seed problems; expanding the second seed problem by using a preset model jailbreak template and a preset adversarial sample to obtain an expanded adversarial sample; screening out target confrontation samples with aggressiveness meeting a preset aggressiveness requirement from the confrontation samples; and attacking the to-be-tested model by using the target confrontation sample. According to the method, the seed problem is gradually rewritten for multiple times, so that the problem of difficulty in cold start of the red team attack model is solved; and meanwhile, a high-aggressiveness confrontation sample is obtained for model attack training, so that the model attack success rate can be effectively improved, and the performance of the to-be-tested model is improved.

Description

Technical Field

[0001] The present application relates to the field of large model attacks, and specifically to a large model attack method, device, electronic device and storage medium. Background Art

[0002] Existing deep learning models usually rely on a large amount of data for training to improve their performance in practical applications. However, when facing adversarial attacks in practical applications, the models often show weak robustness. This phenomenon not only affects the performance of the model, but also poses a threat to the security of the model. When strengthening the security of large models, it is inevitable to use methods that can effectively attack large models in red team testing to improve the security and robustness of the target model. Summary of the invention

[0003] The embodiments of the present application provide a large model attack method, device, electronic device and storage medium. By gradually rewriting and expanding the seed problem multiple times, highly aggressive adversarial samples are obtained to attack the model, thereby improving the robustness of the model to be tested.

[0004] In a first aspect, the present application provides a large model attack method, which is applied to a red team attack model, and the method comprises:

[0005] Obtaining a first seed question for the model to be tested, and rewriting the first seed question to obtain a plurality of variants of a second seed question;

[0006] Expanding the second seed problem using a preset model jailbreak template and a preset adversarial sample to obtain an expanded adversarial sample;

[0007] Screening out target adversarial samples whose aggressiveness meets preset aggressiveness requirements from the adversarial samples;

[0008] The target adversarial sample is used to attack the model to be tested.

[0009] In some possible embodiments, obtaining a first seed question for the model to be tested and rewriting the first seed question to obtain a plurality of variant second seed questions includes:

[0010] Obtaining a first seed problem for the model to be tested, and rewriting the first seed problem using an instruction fine-tuning model and a fine-tuning instruction to obtain a third seed problem of multiple variants;

[0011] Screening the fourth sub-questions that are offensive among the third sub-questions;

[0012] The fourth seed problem is rewritten twice to obtain the second seed problem.

[0013] In some possible embodiments, screening the third sub-questions for offensive fourth sub-questions includes:

[0014] Using the third seed question to attack the model to be tested, to obtain a model output result and a security evaluation result corresponding to the model output result;

[0015] Based on the model output result and the safety evaluation result, a fourth sub-problem that is aggressive among the third sub-problems is determined.

[0016] In some possible embodiments, after rewriting the fourth seed question twice to obtain the second seed question, obtaining the first seed question for the model to be tested and rewriting the first seed question to obtain multiple variants of the second seed question further includes:

[0017] Determining whether the number of the second seed questions reaches a preset seed question number threshold;

[0018] If the number of the second seed questions reaches a preset seed question number threshold, fine-tuning the model parameters in the instruction fine-tuning model to obtain a new instruction fine-tuning model after fine-tuning;

[0019] The first seed question is rewritten and screened using a new fine-tuning instruction model and a new fine-tuning instruction to obtain a new second seed question.

[0020] In some possible embodiments, screening out the target adversarial samples whose aggressiveness meets the preset aggressiveness requirement from the adversarial samples includes:

[0021] Deduplicating the adversarial samples to obtain deduplicated adversarial samples;

[0022] Use the preset sorting model to determine the aggressiveness of each adversarial sample after deduplication;

[0023] According to the aggressiveness of the adversarial samples after deduplication, the adversarial samples after deduplication are sorted to obtain an adversarial sample sequence;

[0024] A target adversarial sample whose aggressiveness meets a preset aggressiveness requirement is selected from the adversarial sample sequence.

[0025] In some possible embodiments, determining the aggressiveness of each of the adversarial samples after deduplication using a preset ranking model includes:

[0026] Selecting training seed questions for the model to be tested from a preset seed question set;

[0027] Rewriting and expanding the training seed problem to obtain an expanded adversarial training sample;

[0028] Using the adversarial training sample to attack the model to be tested, and obtaining the model output result and attack effectiveness evaluation;

[0029] Performing negative sampling training on the initial sorting model according to the model output result and the attack effectiveness evaluation until the initial sorting model converges to obtain a sorting model;

[0030] The ranking model is used to determine the aggressiveness of each adversarial sample after deduplication.

[0031] In some possible embodiments, the method further includes:

[0032] Obtaining multi-dimensional evaluation indicators after the model to be tested is attacked by the target adversarial sample;

[0033] Generate a safety assessment report for the model to be tested based on the multi-dimensional assessment indicators.

[0034] In a second aspect, the present application embodiment provides a large model attack device,

[0035] The large model attack device comprises:

[0036] A question rewriting module, used for obtaining a first seed question for a model to be tested, and rewriting the first seed question to obtain a plurality of variants of a second seed question;

[0037] A question expansion module, used to expand the second seed question using a preset model jailbreak template and a preset adversarial sample to obtain an expanded adversarial sample;

[0038] An adversarial sample screening module, used to screen out target adversarial samples whose aggressiveness meets preset aggressiveness requirements among the adversarial samples;

[0039] The attack module is used to use the target adversarial sample to attack the model to be tested.

[0040] In a third aspect, the electronic device provided in the present application includes a memory and a processor, the memory stores a computer program, and the processor is used to run the computer program in the memory to implement the steps in the large model attack method provided in the present application.

[0041] In a fourth aspect, the computer-readable storage medium provided in the present application stores a plurality of instructions, which are suitable for loading by a processor to implement the steps in the large model attack method provided in the present application.

[0042] The present application provides a large model attack method, device, electronic device and storage medium. The method is applied to the red team attack model, including obtaining a first seed problem for the model to be tested, and rewriting the first seed problem to obtain a second seed problem of multiple variants; using a preset model jailbreak template and a preset adversarial sample to expand the second seed problem to obtain an expanded adversarial sample; screening out a target adversarial sample whose aggressiveness meets the preset aggressiveness requirements in the adversarial sample; and using the target adversarial sample to attack the model to be tested. The present application solves the problem of the difficulty of cold starting the red team attack model by rewriting the seed problem multiple times and step by step; at the same time, obtaining highly aggressive adversarial samples for model attack training can effectively improve the success rate of model attacks, thereby improving the performance of the model to be tested. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 is a schematic diagram of a large model attack system provided in an embodiment of the present application;

[0045] Figure 2 It is a flowchart of an embodiment of the large model attack method provided in the embodiment of the present application;

[0046] Figure 3 It is a flowchart of an embodiment of the present application for rewriting a second seed question;

[0047] Figure 4 This is a flowchart of an embodiment of obtaining a sorting model provided by an embodiment of the present application;

[0048] Figure 5 It is a complete flow chart of an embodiment of the large model attack method provided in the embodiment of the present application;

[0049] Figure 6 This is a flow chart of an embodiment of a training sorting model provided in an embodiment of the present application;

[0050] Figure 7 It is a flowchart of an embodiment of the large model attack method provided in the embodiment of the present application;

[0051] Figure 8 It is a schematic diagram of the structure of a large model attack device provided in an embodiment of the present application;

[0052] Fig. 9is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0053] Fig.10 is a schematic diagram of the structure of an embodiment of a terminal device provided in an embodiment of the present application;

[0054] Fig.11 It is a schematic diagram of the server structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] It should be noted that the principles of the present application are illustrated by implementing them in an appropriate computing environment. The following description is based on the illustrated specific embodiments of the present application and should not be considered as limiting other specific embodiments of the present application that are not described in detail herein.

[0056] In the following description of the present application, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0057] In the following description of the present application, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0059] In order to improve the effect of large model attack, the embodiment of the present application provides a large model attack method, device, electronic device and storage medium. Among them, the large model attack method can be executed by the large model attack device, or by an electronic device integrated with the large model attack device.

[0060] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0061] Please refer to Figure 1 ,This application also provides a large model attack system, such as Figure 1As shown, the large model attack system includes an electronic device 100, in which the large model attack device provided by the present application is integrated.

[0062] Among them, the electronic device 100 can be any device equipped with a processor and has processing capabilities, such as mobile electronic devices with processors such as smart phones, tablet computers, PDAs, laptops, smart speakers, or fixed electronic devices with processors such as desktop computers, televisions, servers, industrial equipment, etc.

[0063] In addition, if Figure 1 As shown, the large model attack system may also include a memory 200 for storing seed questions, model jailbreak templates, adversarial samples, etc.

[0064] In the embodiment of the present application, the memory 200 can be a cloud storage. Cloud storage is a new concept extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as the storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.

[0065] At present, the storage method of the storage system is: create a logical volume, and when creating a logical volume, allocate physical storage space for each logical volume. The physical storage space may be composed of disks of a storage device or several storage devices. The client stores data on a logical volume, that is, stores the data on the file system. The file system divides the data into many parts, each of which is an object. The object contains not only data but also additional information such as data identification (ID, ID entity). The file system writes each object into the physical storage space of the logical volume, and the file system records the storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.

[0066] The process of the storage system allocating physical storage space to a logical volume is as follows: based on the estimated capacity of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of independent redundant disk arrays (RAID, Redundant Array of Independent Disks), the physical storage space is pre-divided into stripes. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0067] It should be noted that Figure 1The scenario diagram of the large model attack system shown is only an example. The large model attack system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the large model attack system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0068] The solution provided in the embodiments of the present application involves technologies such as artificial intelligence (AI), computer vision (CV), and machine learning (ML), which are specifically described by the following embodiments:

[0069] Among them, AI is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0070] AI technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0071] CV is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision that uses cameras and computers to replace human eyes to identify, track and measure targets, and further performs graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes adversarial disturbance generation, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0072] The following is a detailed description of the large model attack method, device, electronic device and storage medium provided by the present application. It should be noted that the serial numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0073] The large model attack method provided in this application is mainly used for the red team attack model; wherein the red team attack model can attack the deep learning model to improve the robustness and security of the deep learning model. This application does not directly optimize the deep learning model to be tested, but improves the attack success rate of the deep learning model to be tested by improving the aggressiveness of the attack samples generated by the red team attack model, thereby improving the robustness and security of the deep learning model to be tested. Please refer to Figure 2 , Figure 2 FIG. 1 is a flow chart of an embodiment of the large model attack method provided in the embodiment of the present application. Figure 2 As shown, the process of the large model attack method provided by this application is as follows:

[0074] 201. Obtain a first seed question for a model to be tested, and rewrite the first seed question to obtain a plurality of variants of a second seed question.

[0075] The large model attack method provided in this application is mainly to adjust the red team attack model so that the red team attack model generates adversarial samples with higher offensiveness and better attack effects, so as to use such adversarial samples to attack the model to be tested and train it to improve the robustness and security of the model to be tested. The seed problems are some training samples that attack the weak points of the model to be tested; for example, training samples specifically for the convolutional layer in the model, and the performance of the convolutional layer in the model can be improved by using the seed problems (or training samples). Different models to be tested correspond to different seed problems, so the first seed problem for the model to be tested can be obtained first, and then the first seed problem can be modified so that the modified seed problem has a higher attack effect or attack efficiency, thereby improving the performance of the model to be tested.

[0076] In the embodiment of the present application, after obtaining the first seed question for the model to be tested, the first seed question can be rewritten to obtain multiple variants of the second seed question. Figure 3 As shown, the flowchart of rewriting the second seed question provided in the embodiment of the present application may include:

[0077] 301. Obtain a first seed problem for the model to be tested, and rewrite the first seed problem using an instruction fine-tuning model and a fine-tuning instruction to obtain a third seed problem of multiple variants.

[0078] 302. Screen out the fourth seed questions that are offensive among the third seed questions.

[0079] 303. Rewrite the fourth seed problem twice to obtain the second seed problem.

[0080] Specifically, the first seed problem can be rewritten for the first time using the instruction fine-tuning model; wherein the instruction fine-tuning model can be a large language model that has been fine-tuned by instructions. After such models have been fine-tuned by instructions, they can better break through the security limitations of the model, that is, they are more vulnerable to attacks. In this application, a fine-tuned model and corresponding fine-tuning instructions are used to rewrite the first seed problem to obtain a variant of the third seed problem. Among them, the rewriting of the first seed problem is mainly to generate different versions of the problem through different rewriting strategies to increase the diversity of attack samples (i.e., the third seed problem). For example, a relatively broad question such as "How is the weather today?" is rewritten into more targeted questions such as "How is the weather in Beijing today?" and "How is the weather in Shanghai today?", while increasing the diversity of the problem. Different third seed problems can be generated by different fine-tuning instructions and different instruction fine-tuning models. The specific rewriting method can refer to the prior art and is not limited here.

[0081] Generally speaking, the rewritten third seed question will be more aggressive than the first seed question; however, in the actual rewriting process, not all third seed questions are aggressive, so the third seed needs to be further screened to screen out the fourth seed question that is aggressive among the third seed questions. After screening out the fourth seed question, the fourth seed question can be further rewritten to further improve the aggressiveness of the seed question. In some embodiments, the fourth seed question can be rewritten twice using methods such as zero-shot learning and few-shot learning to obtain the final second seed question with a certain degree of aggressiveness. The specific process of the secondary rewriting can refer to the prior art and is not limited in this application.

[0082] In the aforementioned embodiment, the process of screening out the fourth sub-problem that is aggressive among the third sub-problems may include: using the third sub-problem to attack the model to be tested to obtain the model output result, that is, the security evaluation result corresponding to the model output result; based on the model output result and the security evaluation result, determining the fourth sub-problem that is aggressive among the third sub-problems.

[0083] Specifically, since the screened seed problem needs to attack the model to be tested, if the attack is successful or successfully affects the performance of the model to be tested, it means that the seed problem is aggressive. Therefore, the third seed problem can be directly used to attack the model to be tested to determine whether the third seed is offensive. The security evaluation result is usually determined based on the output result of the model. For example, the model is attacked by the third seed problem to obtain sensitive information. Under normal circumstances, the model will not output sensitive information; but if the attack is successful, the model will output sensitive information. At this time, the attack can be determined to be successful based on the sensitive information output by the model, and then the third seed problem is determined to be offensive. The above embodiment briefly describes how to determine the security evaluation result based on the model output result, and how to determine the fourth seed problem based on the security evaluation result; in the actual model attack scenario, the output results of the model are different, and there are multiple different dimensions of security evaluation; it is usually necessary to combine the two to jointly determine whether the seed problem is offensive. The specific method for determining the aggressiveness of the seed problem in combination with the model output result and the security evaluation result can refer to the prior art, which will not be repeated here.

[0084] In the above process of rewriting the seed question, the instruction fine-tuning model itself can be further fine-tuned, and the new instruction fine-tuning model can be used to rewrite the question. Specifically, since the second seed question is an aggressive seed question after rewriting, after obtaining the second seed question, it is possible to determine whether the number of the second seed questions is greater than the preset seed question number threshold; if the number of the second seed questions reaches the preset seed question number threshold, the model parameters in the instruction fine-tuning model can be fine-tuned to obtain a new instruction fine-tuning model after fine-tuning. At the same time, the new instruction fine-tuning model and fine-tuning instructions are used to rewrite and screen the first seed question to obtain a new second seed question.

[0085] 202. Use the preset model jailbreak template and the preset adversarial sample to expand the second seed problem to obtain an expanded adversarial sample.

[0086] In the aforementioned embodiment, the first and second seed problems are obtained by multiple rewriting and screening, but the second seed problem cannot be directly used for model attack, and the aggressiveness of the second seed problem is usually not high. In the present application, the second seed problem can also be expanded to obtain an expanded adversarial sample, and then the adversarial sample is used to attack the model to be tested. Specifically, the second seed problem can be expanded using a preset model jailbreak template and a preset adversarial sample; wherein the jailbreak template can jump out of the limitations of the model to induce the model to produce unexpected behavior, and the adversarial sample can also cause the model to give an erroneous output. Therefore, the model jailbreak template can be combined with the second seed problem, and the adversarial sample can be combined with the second seed problem to obtain an adversarial sample that can cause the model to produce unexpected results.

[0087] 203. Filter out target adversarial samples whose aggressiveness meets the preset aggressiveness requirements among the adversarial samples.

[0088] 204. Use the target adversarial sample to attack the model to be tested.

[0089] In the above embodiment, the first seed problem is rewritten and expanded to obtain an adversarial sample with a certain degree of aggressiveness, and the goal of this application is to generate a sample with a higher degree of aggressiveness for model attack and training; therefore, after obtaining the adversarial sample, it is necessary to determine the aggressiveness of multiple adversarial samples respectively, and further select the more aggressive adversarial samples to attack the model to be tested. Specifically, the target adversarial sample whose aggressiveness meets the preset aggressiveness requirements can be selected from the adversarial samples, and then the adversarial sample can be used to attack the model to be tested. Among them, the specific process of selecting the target adversarial sample is described in detail in the subsequent embodiments and is not limited here.

[0090] The large model attack method provided in this application is applied to the red team attack model. The method first obtains a first seed problem for the model to be tested, and rewrites the first seed problem to obtain a second seed problem of multiple variants; the second seed problem is expanded using a preset model jailbreak template and a preset adversarial sample to obtain an expanded adversarial sample; the target adversarial sample whose aggressiveness meets the preset aggressiveness requirements in the adversarial sample is screened out; the target adversarial sample is used to attack the model to be tested. This application solves the problem of the difficulty of cold starting the red team attack model by rewriting the seed problem multiple times and step by step; at the same time, highly aggressive adversarial samples are obtained for model attack training, which can effectively improve the success rate of model attacks, thereby improving the performance of the model to be tested.

[0091] In some embodiments of the present application, the ranking model can be used to evaluate and screen the aggressiveness of adversarial samples. Before this, the ranking model needs to be obtained first. Specifically, Figure 4 As shown, a schematic diagram of an embodiment of the process of obtaining a sorting model provided in an embodiment of the present application may include the following steps:

[0092] 401. Filter training seed questions for the model to be tested from a preset seed question set.

[0093] 402. Rewrite and expand the training seed problem to obtain an expanded adversarial training sample.

[0094] 403. Use the adversarial training samples to attack the model to be tested, and obtain the model output results and the attack effectiveness evaluation.

[0095] 404. Perform negative sampling training on the initial sorting model according to the model output results and the attack effectiveness evaluation until the initial sorting model converges to obtain a sorting model.

[0096] Specifically, in the present application, the initial sorting model needs to be trained using the aggressiveness of the adversarial samples before aggressiveness identification and sorting can be performed; therefore, it is first necessary to use the model to be tested to determine the aggressiveness of the adversarial samples. Specifically, it is first necessary to screen out training seed questions for the model to be tested from the preset seed question set; this is because different models to be tested correspond to different seed questions, and screening out dedicated seed questions can more accurately improve the aggressiveness of seed questions for different models. Similarly, the seed questions need to be rewritten to generate more variant questions to cover a wider attack surface; at the same time, the rewritten variant questions need to be expanded by adding jailbreak templates or adversarial sample suffixes to obtain different versions of adversarial training samples. The specific process of rewriting and expansion can refer to the aforementioned content and will not be repeated here.

[0097] After obtaining the adversarial training samples, the adversarial training samples can be input into the model to be tested and a response can be obtained, that is, the adversarial training samples can be used to attack the model to be tested; after obtaining the model output results, the evaluation model needs to be used to judge the effectiveness of the attack, such as attack success or attack failure. The adversarial samples that successfully attack have higher aggressiveness, so they are ranked higher in aggressiveness; while the adversarial samples that fail to attack have lower aggressiveness and are ranked lower in aggressiveness. Therefore, the initial sorting model can be optimized through negative sampling training based on the model output results and the aggressiveness evaluation results, and the initial sorting model can be adjusted based on the sorting results output by the initial sorting model until the adjusted sorting model converges to obtain the final sorting model. Through the aforementioned training process, the sorting model can continuously optimize and improve the sorting results, thereby ensuring that the most aggressive samples are selected first in the actual attack test, thereby improving the overall attack success rate, and thus improving the performance of the model to be tested.

[0098] The above embodiment describes how to train the initial sample sorting model to obtain the final sample sorting model. The final sample sorting model can be used to determine the aggressiveness of the adversarial samples and sort the adversarial samples according to the aggressiveness, thereby screening out some target adversarial samples with high aggressiveness from the sorted adversarial samples.

[0099] It should be noted that after obtaining the adversarial samples, it is necessary to deduplicate the adversarial samples, and then sort and screen the adversarial samples after deduplication. This is to ensure the uniqueness of the problem or weakness represented by each adversarial sample, and to avoid repeated seed problems in different adversarial samples. Screening out the target adversarial samples whose aggressiveness meets the preset aggressiveness requirements can be to screen out adversarial samples whose aggressiveness is not less than the aggressiveness threshold; it can also be to screen out a preset number of adversarial samples before sorting the aggressiveness, such as the top 100 in terms of aggressiveness. In other embodiments, the preset aggressiveness requirements can also be other requirements, which are not limited here. By using the large model attack method provided in the present application, high-quality adversarial attack samples can be efficiently generated and screened, thereby improving the attack success rate and attack effect, and thus improving the performance of the model to be tested.

[0100] The above embodiments all describe how to determine a highly aggressive target adversarial sample, and then use the target adversarial sample to attack the model to be tested to improve the performance of the model to be tested. To determine whether the target adversarial sample has good aggressiveness, it is also necessary to evaluate the performance of the model to be tested under the attack of various adversarial samples. Specifically, the target adversarial samples can be used to attack the model to be tested in turn, and determine the multi-dimensional evaluation indicators of the model to be tested after the attack, including but not limited to the attack success rate, the robustness of the model to be tested, etc.; then the multi-dimensional evaluation indicators are summarized and analyzed to generate a final security assessment report to determine whether the attack method provided in this application can be used to stably generate adversarial samples for countering attacks.

[0101] like Figure 5 As shown, it is a complete flow chart of the embodiment of the large model attack method provided in the embodiment of the present application. Figure 5In the process, it is necessary to first select the first seed question that matches the model to be tested from the seed question set, and then rewrite and expand the first seed question; specifically, the first seed question can be rewritten using a basic instruction fine-tuning model (base-model) and a fine-tuning instruction (prompt), and then rewritten twice using zero-shot learning, few-shot learning and other methods to gradually accumulate training data. The seed question obtained after the second rewrite can also be combined with the jailbreak template and the preset adversarial sample to obtain different adversarial samples (including the template question obtained by combining with the jailbreak template, and the question + attack suffix obtained by combining with the preset adversarial sample); finally, the adversarial sample is input into the model to be tested to obtain the output result, and the output result of the model to be tested is evaluated using a third-party evaluation model to obtain the attack evaluation result. According to the attack evaluation results, such as attack success or attack failure, the attack success rate can be determined, and then the attack effectiveness and attack success rate of the samples in the aforementioned rewritten and expanded adversarial samples can be reversely deduced based on the attack success rate, so as to update the appropriate fine-tuning model, screen effective jailbreak templates, and screen effective adversarial samples, so as to facilitate the subsequent rewriting and expansion of the seed problem.

[0102] like Figure 6 FIG. 1 is a flow chart of an embodiment of a training sorting model provided in an embodiment of the present application. Figure 6 In the process, it is also necessary to first select seed questions that match the model to be tested from the seed question set, and then expand the selected seed questions, including but not limited to adding jailbreak templates, adding adversarial suffixes, or a combination of the two. The expanded questions are input into the model to be tested to obtain the output of the model, and then an evaluation model is used to evaluate the output of the model to obtain the attack evaluation result of this attack, that is, the sample label; based on the sample label and the expanded questions, the sorting model can be negatively sampled and trained until the sorting model converges to obtain the final sorting model.

[0103] like Figure 7 FIG. 1 is a flow chart of an embodiment of a large model attack method provided in an embodiment of the present application. Figure 7 In the process, it is necessary to select seed questions that match the model to be tested from the seed question set, and then expand the selected seed questions, including but not limited to adding jailbreak templates, adding adversarial suffixes, or a combination of the two methods; then deduplicate the adversarial samples obtained by expansion to ensure the uniqueness of the problems or weaknesses represented by each adversarial sample, and avoid repeated seed questions in different adversarial samples. Figure 6The trained sorting model sorts the deduplicated adversarial samples to screen out the more aggressive adversarial samples; for example, the top 100 adversarial samples after sorting are screened out for subsequent attack training.

[0104] In order to facilitate better implementation of the large model attack method provided in the embodiment of the present application, the embodiment of the present application also provides a large model attack device based on the large model attack method. The meanings of the terms are the same as those in the large model attack method. For specific implementation details, please refer to the description in the above method embodiment.

[0105] Please refer to Figure 8 , Figure 8 A schematic diagram of the structure of a large model attack device provided in an embodiment of the present application, the large model attack device may include:

[0106] A question rewriting module 801 is used to obtain a first seed question for a model to be tested, and rewrite the first seed question to obtain a plurality of variants of a second seed question;

[0107] A question expansion module 802, configured to expand the second seed question using a preset model jailbreak template and a preset adversarial sample to obtain an expanded adversarial sample;

[0108] An adversarial sample screening module 803 is used to screen out target adversarial samples whose aggressiveness meets a preset aggressiveness requirement from among the adversarial samples;

[0109] The attack module 804 is used to attack the model to be tested using the target adversarial sample.

[0110] The present application provides a large model attack device, which is applied to the red team attack model. The attack device can obtain a first seed problem for the model to be tested, and rewrite the first seed problem to obtain a second seed problem of multiple variants; expand the second seed problem using a preset model jailbreak template and a preset adversarial sample to obtain an expanded adversarial sample; screen out a target adversarial sample whose aggressiveness meets the preset aggressiveness requirements in the adversarial sample; and use the target adversarial sample to attack the model to be tested. The present application solves the problem of the difficulty of cold starting the red team attack model by rewriting the seed problem multiple times and step by step; at the same time, highly aggressive adversarial samples are obtained for model attack training, which can effectively improve the success rate of model attacks, thereby improving the performance of the model to be tested.

[0111] In some embodiments, the question rewriting module 801 may be specifically used to: obtain a first seed question for the model to be tested, and rewrite the first seed question using the instruction fine-tuning model and the fine-tuning instruction to obtain a third seed question of multiple variants;

[0112] Screening out the offensive fourth seed questions from the third seed questions;

[0113] Rewrite the fourth seed problem twice to get the second seed problem.

[0114] In some embodiments, the question rewriting module 801 can be specifically used to: use the third seed question to attack the model to be tested, and obtain the model output result and the security evaluation result corresponding to the model output result;

[0115] Based on the model output results and the safety evaluation results, the fourth sub-problem with aggressiveness in the third sub-problem is determined.

[0116] In some embodiments, the question rewriting module 801 may be specifically used to: determine whether the number of second seed questions reaches a preset seed question number threshold;

[0117] If the number of the second seed questions reaches a preset seed question number threshold, the model parameters in the instruction fine-tuning model are fine-tuned to obtain a new instruction fine-tuning model after fine-tuning;

[0118] The first seed problem is rewritten and screened using a new fine-tuning instruction model and fine-tuning instructions to obtain a new second seed problem.

[0119] In some embodiments, the adversarial sample screening module 803 may be specifically used to: deduplicate the adversarial sample to obtain a deduplicated adversarial sample;

[0120] Use the preset sorting model to determine the aggressiveness of each adversarial sample after deduplication;

[0121] According to the aggressiveness of the adversarial samples after deduplication, the adversarial samples after deduplication are sorted to obtain an adversarial sample sequence;

[0122] Filter the target adversarial samples whose aggressiveness meets the preset aggressiveness requirements in the adversarial sample sequence.

[0123] In some embodiments, the adversarial sample screening module 803 may be specifically used to: screen training seed questions for the model to be tested from a preset seed question set;

[0124] Rewrite and expand the training seed problem to obtain the expanded adversarial training samples;

[0125] Use adversarial training samples to attack the model to be tested, and obtain the model output results and attack effectiveness evaluation;

[0126] According to the model output results and attack effectiveness evaluation, negative sampling training is performed on the initial sorting model until the initial sorting model converges to obtain a sorting model;

[0127] The ranking model is used to determine the aggressiveness of the adversarial samples after deduplication.

[0128] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein the processor is used to execute the steps in the large model attack method provided in this embodiment by calling a computer program stored in the memory.

[0129] Please refer to Fig. 9 , Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0130] The electronic device may include components such as a processor 901 with one or more processing cores, a memory 902 with one or more computer-readable storage media, a power supply 903, and an input unit 904. Those skilled in the art will appreciate that Fig. 9 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0131] The processor 901 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 902, and calling data stored in the memory 902, the processor 901 performs various functions of the electronic device and processes data. Optionally, the processor 901 may include one or more processing cores; optionally, the processor 901 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 901.

[0132] The memory 902 can be used to store software programs and modules. The processor 901 executes various functional applications and data processing by running the software programs and modules stored in the memory 902. The memory 902 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 902 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 902 may also include a memory controller to provide the processor 901 with access to the memory 902.

[0133] The electronic device also includes a power supply 903 for supplying power to various components. Optionally, the power supply 903 can be logically connected to the processor 901 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 903 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0134] The electronic device may further include an input unit 904, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0135] Although not shown, the electronic device may also include a display unit, an image acquisition component, etc., which will not be described in detail here. Specifically in this embodiment, the processor 901 in the electronic device will load the executable code corresponding to one or more computer programs into the memory 902 according to the following instructions, and the processor 901 will execute the steps in the large model attack method provided in this application, such as:

[0136] Obtain a first seed question for the model to be tested, and rewrite the first seed question to obtain a plurality of variants of the second seed question;

[0137] The second sub-problem is expanded using the preset model jailbreak template and the preset adversarial sample to obtain the expanded adversarial sample;

[0138] Filter out the target adversarial samples whose aggressiveness meets the preset aggressiveness requirements among the adversarial samples;

[0139] Use the target adversarial sample to attack the model to be tested.

[0140] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0141] To this end, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. The computer program is loaded by a processor to execute the steps in any large model attack method provided in the embodiment of the present application.

[0142] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0143] Among them, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc. Since the computer program stored in the computer-readable storage medium can execute the steps in the large model attack method provided in the embodiment of the present application, the beneficial effects that can be achieved by the large model attack method provided in the embodiment of the present application can be achieved. Please refer to the previous embodiment for details, which will not be repeated here.

[0144] When the computing device in the embodiment of the present application is a terminal device, the embodiment of the present application also provides a terminal device, such as Fig.10 For the sake of convenience, only the parts related to the embodiments of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The terminal device can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a car computer, etc., taking the mobile phone as an example:

[0145] Fig.10 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Fig.10 The mobile phone includes: a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090. Those skilled in the art will understand that Fig.10 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0146] Combine the following Fig.10 A detailed introduction to the various components of the mobile phone:

[0147] The RF circuit 1010 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 1080 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 1010 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the RF circuit 1010 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0148] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1020. The memory 1020 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 1020 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0149] The input unit 1030 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1031 or near the touch panel 1031 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 1080, and can receive and execute commands sent by the processor 1080. In addition, the touch panel 1031 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1031, the input unit 1030 may further include other input devices 1032. Specifically, the other input devices 1032 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0150] The display unit 1040 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1040 may include a display panel 1041. Optionally, the display panel 1041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it is transmitted to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides a corresponding visual output on the display panel 1041 according to the type of touch event. Although in Fig.10 In the embodiment, the touch panel 1031 and the display panel 1041 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0151] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1041 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1041 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0152] The audio circuit 1060, the speaker 1061, and the microphone 1062 can provide an audio interface between the user and the mobile phone. The audio circuit 1060 can transmit the received audio data to the speaker 1061 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 1062 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1060 and converted into audio data, and then the audio data is output to the processor 1080 for processing, and then sent to another mobile phone through the RF circuit 1010, or the audio data is output to the memory 1020 for further processing.

[0153] Wi-Fi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages, and access streaming media through the Wi-Fi module 1070. It provides users with wireless broadband Internet access. Fig.10 A Wi-Fi module 1070 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0154] The processor 1080 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 1020, and calling data stored in the memory 1020, so as to monitor the mobile phone as a whole. Optionally, the processor 1080 may include one or more processing units; optionally, the processor 1080 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1080.

[0155] The mobile phone also includes a power supply 1090 (such as a battery) for supplying power to various components. Optionally, the power supply can be logically connected to the processor 1080 through a power management system, thereby managing functions such as charging, discharging, and power consumption management through the power management system.

[0156] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0157] In the embodiment of the present application, the processor 1080 included in the mobile phone also has the function of controlling the execution of the above large model attack method process performed by the large model attack device.

[0158] The present application also provides a server. Fig.11 , Fig.11 1 is a schematic diagram of a server structure provided in an embodiment of the present application. The server 1100 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (full name in English: central processing units, abbreviated in English: CPU) 1122 (for example, one or more processors) and memory 1132, one or more storage media 1130 (for example, one or more mass storage devices) storing application programs 1142 or data 1144. Among them, the memory 1132 and the storage medium 1130 can be short-term storage or permanent storage. The program stored in the storage medium 1130 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1122 can be configured to communicate with the storage medium 1130 to execute a series of instruction operations in the storage medium 1130 on the server 1100.

[0159] The server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input and output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0160] The steps in the large model attack method in the above embodiment can be based on the Fig.11 The structure of the server 1100 is shown. For example, the central processor 1122 performs the following operations by calling the instructions in the memory 1132:

[0161] Obtain a first seed question for the model to be tested, and rewrite the first seed question to obtain a plurality of variants of the second seed question;

[0162] The second sub-problem is expanded using the preset model jailbreak template and the preset adversarial sample to obtain the expanded adversarial sample;

[0163] Filter out the target adversarial samples whose aggressiveness meets the preset aggressiveness requirements among the adversarial samples;

[0164] Use the target adversarial sample to attack the model to be tested.

[0165] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0166] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0167] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0168] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0169] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0170] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0171] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0172] The technical solutions provided in the embodiments of the present application are introduced in detail above. The principles and implementation methods of the embodiments of the present application are explained by using specific examples in the embodiments of the present application. The description of the above embodiments is only used to help understand the methods and core ideas of the embodiments of the present application. At the same time, for those skilled in the art, according to the ideas of the embodiments of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the embodiments of the present application.

[0173] It should be noted that when the above embodiments of the present application are applied to specific products or technologies, the relevant data of the user is involved, and the user's permission or consent is required, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

Claims

1. A large model attack method, characterized in that: Applied to the red team attack model, the method includes: Obtaining a first seed question for the model to be tested, and rewriting the first seed question to obtain a plurality of variants of a second seed question; Expanding the second seed problem using a preset model jailbreak template and a preset adversarial sample to obtain an expanded adversarial sample; Screening out target adversarial samples whose aggressiveness meets preset aggressiveness requirements from the adversarial samples; The target adversarial sample is used to attack the model to be tested.

2. The large model attack method according to claim 1, characterized in that: The obtaining of a first seed question for the model to be tested and rewriting the first seed question to obtain a plurality of variants of the second seed question include: Obtaining a first seed problem for the model to be tested, and rewriting the first seed problem using an instruction fine-tuning model and a fine-tuning instruction to obtain a third seed problem of multiple variants; Screening the fourth sub-questions that are offensive among the third sub-questions; The fourth seed problem is rewritten twice to obtain the second seed problem.

3. The large model attack method according to claim 2, characterized in that: The screening of the fourth sub-questions that are offensive in the third sub-questions includes: Using the third seed question to attack the model to be tested, to obtain a model output result and a security evaluation result corresponding to the model output result; Based on the model output result and the safety evaluation result, a fourth sub-problem that is aggressive in the third sub-problem is determined.

4. The large model attack method according to claim 2, characterized in that: After rewriting the fourth seed question twice to obtain the second seed question, obtaining the first seed question for the model to be tested and rewriting the first seed question to obtain multiple variants of the second seed question also includes: Determining whether the number of the second seed questions reaches a preset seed question number threshold; If the number of the second seed questions reaches a preset seed question number threshold, fine-tuning the model parameters in the instruction fine-tuning model to obtain a new instruction fine-tuning model after fine-tuning; The first seed question is rewritten and screened using a new fine-tuning instruction model and a new fine-tuning instruction to obtain a new second seed question.

5. The large model attack method according to claim 1, characterized in that: The step of screening out target adversarial samples whose aggressiveness meets preset aggressiveness requirements from among the adversarial samples includes: Deduplicating the adversarial samples to obtain deduplicated adversarial samples; Use the preset sorting model to determine the aggressiveness of each adversarial sample after deduplication; According to the aggressiveness of the adversarial samples after deduplication, the adversarial samples after deduplication are sorted to obtain an adversarial sample sequence; A target adversarial sample whose aggressiveness meets a preset aggressiveness requirement is selected from the adversarial sample sequence.

6. The large model attack method according to claim 5, characterized in that: The method of using a preset sorting model to determine the aggressiveness of each of the adversarial samples after deduplication includes: Selecting training seed questions for the model to be tested from a preset seed question set; Rewriting and expanding the training seed problem to obtain an expanded adversarial training sample; Using the adversarial training sample to attack the model to be tested, and obtaining the model output result and attack effectiveness evaluation; Performing negative sampling training on the initial sorting model according to the model output result and the attack effectiveness evaluation until the initial sorting model converges to obtain a sorting model; The ranking model is used to determine the aggressiveness of each adversarial sample after deduplication.

7. The large model attack method according to claim 1, characterized in that: The method further comprises: Obtaining multi-dimensional evaluation indicators after the model to be tested is attacked by the target adversarial sample; Generate a safety assessment report for the model to be tested based on the multi-dimensional assessment indicators.

8. A large model attack device, characterized in that: The large model attack device comprises: A question rewriting module, used for obtaining a first seed question for a model to be tested, and rewriting the first seed question to obtain a plurality of variants of a second seed question; A question expansion module, used to expand the second seed question using a preset model jailbreak template and a preset adversarial sample to obtain an expanded adversarial sample; An adversarial sample screening module, used to screen out target adversarial samples whose aggressiveness meets preset aggressiveness requirements among the adversarial samples; The attack module is used to use the target adversarial sample to attack the model to be tested.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the processor is used to execute the steps in the large model attack method according to any one of claims 1 to 7 by calling a computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the large model attack method according to any one of claims 1 to 7.

Images