Model security detection method and device, computer program product and electronic equipment

Abstract

The invention relates to the technical field of computers, in particular to a model security detection method and device, a computer program product and electronic equipment. The model security detection method comprises the following steps: obtaining a to-be-detected large model; performing fine-tuning training on the large model by using the fine-tuning data set to obtain a model evaluation index, and determining a target fine-tuning strategy according to the model evaluation index for training, the target fine-tuning strategy is selected from the following candidate fine-tuning strategies: a first strategy for performing secondary fine-tuning processing on the large model after fine-tuning training by using the fine-tuning data set and a second strategy for performing fine-tuning training on the large model again by using the fine-tuning data set after data deformation processing; and obtaining a trained large model, processing question and answer data by using the trained large model, and determining whether the to-be-detected large model has a privacy disclosure risk according to a processing result, the question and answer data being not included in the fine tuning data set. According to the invention, the security detection accuracy of the large model can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically, to a model security detection method, a model security detection device, a computer program product, and an electronic device. Background Art

[0002] With the development of computer technology and the Internet, large language models (LLMs) have gradually been applied in various application fields, and are developing in the direction of enhancing model parameters and utilizing human language data. Challenges in security and privacy have also emerged. Among them, in the process of network data collection, private information such as personal identity may be accumulated unconsciously, resulting in unexpected privacy leakage. At present, there are endless attack methods that make large language models output information that violates human will or values. However, the current detection methods for privacy leakage risks of large models cannot accurately detect possible risks, which has a certain degree of adverse impact on users and society.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The purpose of the present disclosure is to provide a model security detection and device, a computer program product and an electronic device, which can at least to a certain extent detect whether a large model has a privacy leakage risk based on fine-tuning training of the large model, thereby improving the accuracy of security detection of the large model.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to one aspect of the present disclosure, a model security detection method is provided, including: obtaining a large model to be detected; fine-tuning the large model using a fine-tuning data set, obtaining a model evaluation index, and determining a target fine-tuning strategy for training based on the model evaluation index, wherein the target fine-tuning strategy is selected from the following candidate fine-tuning strategies: a first strategy of performing secondary fine-tuning processing on the fine-tuned large model using the fine-tuning data set and a second strategy of re-fine-tuning the large model using the fine-tuning data set after data deformation processing; obtaining the trained large model, processing question and answer data using the trained large model, and determining whether the large model to be detected has a privacy leakage risk based on the processing result, wherein the question and answer data is not included in the fine-tuning data set.

[0007] In an exemplary embodiment of the present disclosure, each piece of training data in a fine-tuning data set includes a question text and an answer text; the fine-tuning data set is used to perform fine-tuning training on a large model to obtain model evaluation indicators, including: inputting the question text into the large model to obtain a predicted answer text; determining a loss value based on the predicted answer text and the answer text, and adjusting the parameters in the large model based on the loss value to obtain a large model after fine-tuning training; and obtaining model evaluation indicators of the large model after fine-tuning training.

[0008] In an exemplary embodiment of the present disclosure, there are multiple model evaluation indicators; a target fine-tuning strategy is determined for training based on the model evaluation indicators, including: if there is at least one model evaluation indicator among the multiple model evaluation indicators whose value is less than the corresponding indicator threshold, then the second strategy is determined as the target fine-tuning strategy, otherwise the first strategy is determined as the target fine-tuning strategy; or, the values ​​of the multiple model evaluation indicators are fused to obtain a fused evaluation indicator, if the value of the fused evaluation indicator is less than the fusion indicator threshold, then the second strategy is determined as the target fine-tuning strategy, otherwise the first strategy is determined as the target fine-tuning strategy.

[0009] In an exemplary embodiment of the present disclosure, if the second strategy is determined to be the target fine-tuning strategy, the target fine-tuning strategy is determined for training according to the model evaluation index, including: performing data deformation processing on the fine-tuning data set based on the data deformation strategy to obtain the fine-tuning data set after the data deformation processing; using the fine-tuning data set after the data deformation processing to re-fine-tune the large model, and jump to the step of obtaining the model evaluation index.

[0010] In an exemplary embodiment of the present disclosure, before performing data deformation processing on the fine-tuning data set based on the data deformation strategy, it also includes: obtaining application scenario information of the large model to be detected; and determining a data deformation strategy that matches the application scenario information based on the application scenario information.

[0011] In an exemplary embodiment of the present disclosure, the question and answer data are processed using a trained big model, and whether the big model to be detected has a privacy leakage risk is determined based on the processing result, including: inputting the question text in the question and answer data into the trained big model, and if the obtained processing result matches the answer text in the question and answer data, it is determined that the big model to be detected has a privacy leakage risk.

[0012] In an exemplary embodiment of the present disclosure, the amount of question and answer data is greater than the amount of training data in the fine-tuning data set, and the method also includes: obtaining the quantitative value of the processing results corresponding to the question data matching the answer text in the question and answer data; determining the privacy leakage risk level of the large model to be detected based on the quantitative value.

[0013] In an exemplary embodiment of the present disclosure, before determining the target fine-tuning strategy for training according to the model evaluation index, the method also includes: randomly sampling the fine-tuning data set to obtain sampling data; inputting the question text in the sampling data into the large model after fine-tuning training to obtain a reference answer text; based on the reference answer text and the answer text in the sampling data, evaluating the effectiveness of this fine-tuning training; if it is determined that this fine-tuning training is invalid, entering the step of determining the target fine-tuning strategy for training according to the model evaluation index, otherwise determining that the large model after this fine-tuning training is the trained large model.

[0014] According to one aspect of the present disclosure, a model security detection device is provided, including: a model acquisition module, used to acquire a large model to be detected; a fine-tuning training module, used to perform fine-tuning training on the large model using a fine-tuning data set, obtain model evaluation indicators, and determine a target fine-tuning strategy for training based on the model evaluation indicators, wherein the target fine-tuning strategy is selected from the following candidate fine-tuning strategies: a first strategy of performing secondary fine-tuning processing on the fine-tuned large model using the fine-tuning data set and a second strategy of re-fine-tuning the large model using the fine-tuning data set after data deformation processing; a risk detection module, used to acquire the trained large model, process question and answer data using the trained large model, and determine whether the large model to be detected has a privacy leakage risk based on the processing result, wherein the question and answer data is not included in the fine-tuning data set.

[0015] According to one aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, any one of the above methods is implemented.

[0016] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any one of the above methods by executing the executable instructions.

[0017] The model security detection method in the exemplary embodiment of the present disclosure, on the one hand, uses the fine-tuning data set to fine-tune the large model, obtains the model evaluation index, and determines the target fine-tuning strategy for training according to the model evaluation index, and performs secondary fine-tuning or repeated fine-tuning on the large model to be detected by repeating the fine-tuning operation and / or changing the fine-tuning data set, so as to ensure the effectiveness of the trained large model and the accuracy of the model; on the other hand, since the trained large model is obtained by fine-tuning the large model to be detected, the process can restore and mine the data information learned by the large model to be detected during the previous training through a small amount of fine-tuning data set, and the question and answer data is not included in the fine-tuning data set, and then it can be judged whether the large model to be detected has learned the ability to leak privacy and related data information during the previous training according to the processing results of the trained large model processing the question and answer data, thereby improving the overall accuracy of the model security detection. On the other hand, the model security detection process does not require too much manual intervention, which greatly reduces the workload and time cost of model security detection.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the following detailed description with reference to the accompanying drawings, in which several embodiments of the present disclosure are shown in an exemplary and non-limiting manner.

[0020] Figure 1 An application environment according to an exemplary embodiment of the present disclosure is shown.

[0021] Figure 2 A flow chart of a model safety detection method according to an exemplary embodiment of the present disclosure is shown.

[0022] Figure 3 A flowchart of an implementation method for obtaining a model evaluation index according to an exemplary embodiment of the present disclosure is shown.

[0023] Figure 4 A schematic diagram of performing data deformation processing on a fine-tuning data set based on a data deformation strategy according to an exemplary embodiment of the present disclosure is shown.

[0024] Figure 5 A flow chart of an implementation method for acquiring a data deformation strategy according to an exemplary embodiment of the present disclosure is shown.

[0025] Figure 6 A flowchart of an implementation method for evaluating the effectiveness of model fine-tuning training according to an exemplary embodiment of the present disclosure is shown.

[0026] Figure 7 A complete flow chart of a model safety detection method according to an exemplary embodiment of the present disclosure is shown.

[0027] Figure 8 A schematic diagram of the composition of a model safety detection device according to an exemplary embodiment of the present disclosure is shown.

[0028] Fig. 9 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown.

[0029] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0030] The exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present disclosure will be more comprehensive and complete and the concepts of the exemplary embodiments will be fully conveyed to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and thus their detailed description will be omitted.

[0031] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known structures, methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0032] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or these functional entities or parts of functional entities may be implemented in one or more software hardened modules, or these functional entities may be implemented in different networks and / or processor devices and / or microcontroller devices.

[0033] Among the security and privacy challenges faced by large models, one of the main challenges is that during the process of network data collection, private information such as personal identity may be unconsciously accumulated, resulting in the trained large model learning data information that involves leaking personal privacy. However, currently, security detection of large models is generally carried out through attack detection, but this method cannot restore and mine the relevant information learned by the large model during training, which to a certain extent affects the overall accuracy of large model security detection.

[0034] The model security detection method provided by the exemplary embodiment of the present disclosure can fine-tune the large model to be detected, and use the fine-tuned large model to process question and answer data, so as to judge whether the large model to be detected has the risk of privacy leakage according to the processing results. This process improves the overall security of model security detection by restoring and mining the data information learned during model training through fine-tuning training of the model.

[0035] It should be noted that the model security detection method provided by the exemplary embodiments of the present disclosure can be used in any field that requires model security detection, such as model training field, model application field, etc. The exemplary embodiments of the present disclosure do not specifically limit the application field of the model security detection method.

[0036] The model security detection method provided by the exemplary embodiments of the present disclosure can be applied to Figure 1 In the application environment shown, the terminal 101 communicates with the server 102 via a network. The data storage system can store data that the server 102 needs to process. The data storage system can be integrated on the server 102, or placed on the cloud or other network servers.

[0037] In an exemplary embodiment, the model security detection method provided by the exemplary embodiment of the present disclosure may be executed by the server 102, and the corresponding model security detection device is disposed in the server 102. Correspondingly, in this mode of execution by the server 102, the server 102 may start to execute the steps in the technical solution of the exemplary embodiment of the present disclosure in response to a trigger command, wherein the trigger command may be sent by a terminal used by a user, or may be triggered locally by the server in response to some automation events.

[0038] The server 102 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server 102 may perform background tasks.

[0039] Furthermore, in another exemplary embodiment, the terminal 101 may also have similar functions to the server 102 , so as to execute the model security detection method provided by the exemplary embodiment of the present disclosure.

[0040] The terminal 101 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, an IoT device, and a portable wearable device. The IoT device may be a smart speaker, a smart TV, a smart air conditioner, and a smart vehicle-mounted device, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The terminal 101 may also be referred to as a mobile terminal, a terminal device, a mobile device, etc. The exemplary embodiments of the present disclosure do not limit the type of the terminal 101.

[0041] In addition, the technical solution of the exemplary embodiment of the present disclosure can also be executed by the terminal 101 and the server 102 in collaboration. In this way of collaborative execution by the terminal 101 and the server 102, some steps in the technical solution provided by the exemplary embodiment of the present disclosure are executed by the terminal 101, and other steps are executed by the server 102.

[0042] It should be noted that in this mode of collaborative execution by the terminal 101 and the server 102, the steps respectively executed by the terminal 101 and the server 102 can be dynamically adjusted according to actual conditions, and there is no special restriction on this.

[0043] The terminal 101 and the server 102 may be connected directly or indirectly via wireless communication, and the exemplary embodiments of the present disclosure do not impose any special limitation thereto.

[0044] In an exemplary embodiment of the present disclosure, a model security detection method is provided, referring to Figure 2 FIG. 1 is a flow chart of a model security detection method according to an exemplary embodiment of the present disclosure. Figure 2 As shown, the model safety detection method includes steps S210 to S230, which are specifically as follows:

[0045] In step S210, a large model to be detected is obtained.

[0046] In the exemplary embodiments of the present disclosure, the large model to be detected refers to a trained large language model, and is also a large language model that needs to be detected for privacy leakage risks. Subsequent security detection steps are all for the large model to be detected. The exemplary embodiments of the present disclosure do not limit the network structure of the large model.

[0047] In step S220, the large model is fine-tuned using the fine-tuning dataset, model evaluation indicators are obtained, and a target fine-tuning strategy is determined for training based on the model evaluation indicators. The target fine-tuning strategy is selected from the following candidate fine-tuning strategies: a first strategy of performing secondary fine-tuning processing on the large model after fine-tuning training using the fine-tuning dataset and a second strategy of re-fine-tuning the large model using the fine-tuning dataset after data deformation processing.

[0048] In an exemplary embodiment of the present disclosure, in order to detect the privacy leakage risk of the large model to be trained, the fine-tuning dataset may be a dataset involving privacy, such as "What is nickname's email address?", "What is nickname's email address?", which may be manually written or obtained from other channels such as a database.

[0049] It should be noted that in the technical solution disclosed herein, the collection / collection, updating, analysis, use, transmission, storage and other aspects of user personal information involved are in compliance with the provisions of relevant laws and regulations, are used for legal and reasonable purposes, are not shared, leaked or sold outside of these legal uses, and are subject to supervision and management by national regulatory authorities. Necessary measures should be taken for user personal information to selectively block the use or access to personal information data to prevent illegal access to such personal information data, ensure that persons who have access to personal information data comply with the provisions of relevant laws and regulations, and ensure the security of user personal information. In addition, once these user personal information data are no longer needed, the risks should be minimized by limiting or even prohibiting data collection and / or deleting data.

[0050] Among them, fine-tuning is to influence the parameters at the lower level of the model through a small amount of data. It can effectively optimize the parameters of a large model through a very small amount of data, so the training data in the fine-tuning data set can be a small amount, which can be at least 10 training data. Each training data in the fine-tuning data set is in the form of a question-answer pair, that is, it includes a question text and a corresponding answer text. Model evaluation indicators refer to indicators used to evaluate the performance of the model in the task, including but not limited to accuracy, recall rate, and precision. Different subsequent processing methods can be selected according to the model evaluation indicators, that is, the first strategy or the second strategy.

[0051] Specifically, the first strategy is to use the original fine-tuning data set for fine-tuning the large model, and perform secondary fine-tuning on the fine-tuned large model until its model evaluation index meets the set conditions. The second strategy is to use the fine-tuning data set after data deformation to re-fine-tune the large model. In other words, if the second strategy is selected, the process of fine-tuning the large model using the fine-tuning data set will be ignored, and the large model to be tested will be fine-tuned again.

[0052] It should be understood that the data information of the large model to be detected may have been initially filtered during the previous training to prevent attackers from using malicious data to train it. In actual implementation, the applicant found that the large model did not intercept the fine-tuning data set, so the fine-tuning data set after data deformation processing can be used to retrain the large model so that the large model can obtain the ability to process these data for subsequent risk detection. The previous training of the large model refers to the training process of obtaining the large model before executing the model security detection method of the exemplary embodiment of the present disclosure.

[0053] In step S230, the trained big model is obtained, the question and answer data is processed using the trained big model, and it is determined whether the big model to be tested has a privacy leakage risk based on the processing result, wherein the question and answer data is not included in the fine-tuning data set.

[0054] In an exemplary embodiment of the present disclosure, question and answer data refers to sample data used for risk detection of a trained large model. For privacy leakage risk detection, the question and answer data may also be question and answer questions involving privacy, such as "What is Xiaoming's phone number?" and "Xiaoming's phone number is 123*****789." There is no specific limitation on this.

[0055] By fine-tuning the large model to be tested, the data information learned by the large model during previous training is restored and mined through a small fine-tuning data set, and then the question and answer data is input into the trained large model, and the processing results are used to determine whether the large model to be tested has a privacy leakage risk.

[0056] At the same time, it should be understood that if the trained large model obtains processing results that do not appear in the fine-tuning dataset when processing question-answering data, it can be explained to a certain extent that the large model to be tested introduced data information involving privacy during the previous training. Based on this, it can be judged whether the large model to be tested has the risk of privacy leakage.

[0057] The model security detection method in the exemplary embodiment of the present disclosure, on the one hand, uses the fine-tuning data set to fine-tune the large model, obtains the model evaluation index, and determines the target fine-tuning strategy for training according to the model evaluation index, and performs secondary fine-tuning or repeated fine-tuning on the large model to be detected by repeating the fine-tuning operation and / or changing the fine-tuning data set, so as to ensure the effectiveness of the trained large model and the accuracy of the model; on the other hand, since the trained large model is obtained by fine-tuning the large model to be detected, the process can restore and mine the data information learned by the large model to be detected during the previous training through a small amount of fine-tuning data set, and the question and answer data is not included in the fine-tuning data set, and then it can be judged whether the large model to be detected has learned the ability to leak privacy and related data information during the previous training according to the processing results of the trained large model processing the question and answer data, thereby improving the overall accuracy of the model security detection. On the other hand, the model security detection process does not require too much manual intervention, which greatly reduces the workload and time cost of model security detection.

[0058] In an exemplary embodiment, a method for obtaining a model evaluation index is provided. Figure 3 As shown, using the fine-tuning dataset to fine-tune the large model to obtain model evaluation indicators may include:

[0059] Step S310: Input the question text into the large model to obtain the predicted answer text.

[0060] As mentioned above, each piece of training data in the fine-tuning dataset includes a question text and an answer text. The question text is input into the large model to obtain the corresponding predicted answer text. During the model fine-tuning training process, the large model needs to learn the training data in the input fine-tuning dataset. Before fine-tuning training, model fine-tuning parameters can be set, including but not limited to the number of iterations (n_epochs), learning rate multiplier (learning_rate_multiplier), parameters for adjusting the prompt word loss weight (prompt_loss_weight) and batch size (batch_size), etc., which can be flexibly set according to the actual application scenario, and there is no specific restriction on this. Among them, the iteration number represents the number of cycles of the training model, one cycle refers to a complete cycle of the fine-tuning dataset, the learning rate multiplier is the learning rate coefficient for training, the fine-tuning learning rate is the original learning rate used for pre-training multiplied by this value, the parameter used to adjust the prompt word loss weight is used to control the large model to try to learn to generate prompts and can add a stabilizing effect to the training when the completion length is short, and the batch size is used to train the number of training examples for a single forward and backward pass. The above model fine-tuning parameters can be flexibly set according to the actual application scenario, and there is no specific restriction on this.

[0061] Step S320: determine the loss value according to the predicted answer text and the answer text, and adjust the parameters in the large model based on the loss value to obtain the large model after fine-tuning training.

[0062] The loss value determined based on the predicted answer text and the answer text in the training data can be determined by calculating the similarity between the predicted answer text and the answer text, and there is no restriction on the specific method of calculating the loss value. The parameters in the large model are adjusted according to the loss value to obtain the large model after fine-tuning training.

[0063] Step S330: Obtain model evaluation indicators of the large model after fine-tuning training.

[0064] After fine-tuning the large model to be detected, the model evaluation index of the large model after fine-tuning can be obtained. For example, the model accuracy can be obtained, that is, the proportion of the number of samples correctly predicted by the model to the total number of samples, and the recall rate can also be obtained, that is, the proportion of all positive samples that are correctly identified as positive samples. The exemplary embodiments of the present disclosure can obtain model evaluation indicators such as the accuracy of the large model through fine-tuning training.

[0065] In some optional embodiments, there are multiple model evaluation indicators, and determining the target fine-tuning strategy for training according to the model evaluation indicators may include:

[0066] If there is at least one model evaluation indicator among the multiple model evaluation indicators whose value is less than the corresponding indicator threshold, the second strategy is determined as the target fine-tuning strategy, otherwise the first strategy is determined as the target fine-tuning strategy.

[0067] Specifically, different model evaluation indicators have their own corresponding indicator thresholds, and the indicator threshold is the threshold at which the model needs to be re-fine-tuned. If there is at least one model evaluation indicator whose value is less than the corresponding indicator threshold, it can be considered that the performance of the large model after fine-tuning is poor. It can be understood that the large model after fine-tuning has preliminarily filtered the fine-tuning data set to prevent attackers from using malicious data to train it, thereby affecting the performance of the large model after fine-tuning. Based on this, the large model can be retrained with the fine-tuning data set after data deformation processing, so that the large model after fine-tuning has the ability to process these data, so as to improve the performance of the large model after fine-tuning, and provide a model basis for subsequent risk detection.

[0068] Correspondingly, if the values ​​of all model evaluation indicators are not less than the corresponding indicator thresholds, there is no need to re-fine-tune the large model to be tested. Instead, the fine-tuning data set is used to perform secondary fine-tuning training on the fine-tuned large model until each model evaluation indicator reaches the corresponding indicator threshold.

[0069] In some optional embodiments, determining the target fine-tuning strategy for training based on the model evaluation index may also include: fusing the values ​​of multiple model evaluation indicators to obtain a fusion evaluation index, if the value of the fusion evaluation index is less than the fusion index threshold, determining the second strategy as the target fine-tuning strategy, otherwise determining the first strategy as the target fine-tuning strategy.

[0070] Specifically, the numerical values ​​of each model evaluation indicator can be fused, for example, by weighted average summation or other methods to obtain a fused evaluation indicator. The use of fused evaluation indicators to decide the target fine-tuning strategy can simultaneously consider multiple different model evaluation indicators, thereby improving the accuracy and comprehensiveness of determining the target fine-tuning strategy.

[0071] The exemplary embodiments of the present disclosure select different target fine-tuning strategies based on model evaluation indicators, so that the trained large model has the ability to process specific data and has higher performance.

[0072] In an exemplary embodiment, if the second strategy is determined to be a target fine-tuning strategy, such as Figure 4 As shown, determining the target fine-tuning strategy for training based on the model evaluation index may include:

[0073] First, the fine-tuning dataset is deformed based on the data deformation strategy to obtain the fine-tuning dataset after data deformation. Then, the large model is re-fine-tuned using the fine-tuning dataset after data deformation, and the process jumps to the step of obtaining model evaluation indicators.

[0074] The data deformation strategy may be, for example, a shift process, such as Figure 4 As shown, a piece of training data in the fine-tuning data is "Q: what is john franke's email address?; A: it is abcdgmail.com", and the training data after the shift processing is "Q: zkdw lv mrkq iudqnh'vhpdlo dgguhvv? A: lw lvdefgpdlo.frp". Of course, the exemplary embodiments of the present disclosure can also be processed in other ways, and there is no specific limitation on this. Since the amount of training data in the fine-tuning data set is small, it does not need to consume too much processing time and resources to perform deformation processing on the fine-tuning data set.

[0075] After obtaining the fine-tuned data set after data deformation processing, the large model is re-fine-tuned and trained so that the re-fine-tuned large model has the ability to process specific data compared to the large model to be tested, such as the ability to output data containing user information, thereby providing a model basis for subsequent risk detection.

[0076] In an exemplary embodiment, a method for implementing a data deformation strategy is also provided. Figure 5 As shown, before the fine-tuning dataset is deformed based on the data deformation strategy, it also includes:

[0077] Step S510: Obtain application scenario information of the large model to be detected.

[0078] In order to restore and mine the data information learned by the model in the previous training through the fine-tuning training process, the content and format of the data information processed by the large model applied in different scenarios are different, so the application scenario information of the large model to be tested can be obtained to guide the selection of data deformation strategy. Among them, the application scenario information includes but is not limited to machine translation, question-answering system, text summary, etc. The application scenario information of the large model to be tested can be provided by the user to be tested, or it can be determined based on the deployment platform of the large model to be tested, and there is no restriction on this.

[0079] Step S520: Determine a data deformation strategy that matches the application scenario information according to the application scenario information.

[0080] After obtaining the application scenario information of the large model to be detected, the data deformation strategy matching the application scenario information can be obtained from the correspondence between the pre-set application scenario information and the data deformation strategy. Among them, each data deformation strategy fully mines the question and answer text features under different application scenarios, and determines the data deformation strategy according to the question and answer text features.

[0081] In some optional embodiments, a data deformation strategy evaluation model may be pre-trained, and the data deformation strategy may be output by inputting the application scenario information into the data deformation strategy evaluation model. The data deformation strategy evaluation model may be trained based on the question and answer text of the application scenario information and the data deformation strategy samples, so that the data deformation strategy evaluation model learns the intrinsic correlation between the question and answer text of different application scenarios and the data deformation strategy, so as to accurately deduce the matching data deformation strategy according to the application scenario information.

[0082] The exemplary embodiments of the present disclosure can improve the pertinence and accuracy of deformation processing on the fine-tuning data set by determining the data deformation strategy using application scenario information, thereby improving the performance of the large model after fine-tuning training.

[0083] In an exemplary embodiment, processing the question-answer data using the trained big model, and determining whether the big model to be detected has a privacy leakage risk according to the processing result may include:

[0084] The question text in the question and answer data is input into the trained large model. If the processing result matches the answer text in the question and answer data, it is determined that the large model to be tested has a privacy leakage risk.

[0085] Specifically, as mentioned above, question and answer data refers to sample data used for risk detection of the trained big model. In order to detect the risk of privacy leakage, the question and answer data can also be question and answer questions involving privacy, so the question text in the question and answer data is input into the trained big model. Since the information involved in the question and answer data is not learned during the fine-tuning training process, if the trained big model outputs the correct processing result, it means that the data used to train the big model before has been restored through the fine-tuning training process, so that the trained big model can output the correct result, indicating that there is a risk of exposing relevant privacy data in the process of training the big model before.

[0086] Based on the aforementioned exemplary embodiment, the number of question-and-answer data is greater than the number of training data in the fine-tuning data set. For example, the number of training data in the fine-tuning data set is 10, the number of question-and-answer data is 100, and the content involved in the question data is not included in the fine-tuning data set. Then, the following may be further included:

[0087] Obtain the number of matching results of the processing results corresponding to the question data and the answer text in the question and answer data, and determine the privacy leakage risk level of the large model to be detected based on the number.

[0088] Specifically, the corresponding relationship between different numerical intervals and privacy leakage risk levels can be pre-set. For ease of processing, the obtained quantitative values ​​can also be normalized to a standard range, and there is no restriction on this. Among them, different privacy leakage levels are used to reflect the possible degree of privacy leakage of the large model to be detected. Based on the privacy leakage level, it can provide a reference for security personnel, such as guiding the process of adjusting the parameters and training data sets of the large model to be detected, thereby improving the accuracy of the detection of the large model.

[0089] The question text of each question and answer data can be input into the trained big model to obtain the processing results, and then the number of values ​​matching the corresponding answer texts in the processing results can be counted, and then the privacy leakage risk level of the big model to be tested can be determined based on the corresponding relationship between the numerical range and the privacy leakage risk level.

[0090] In an exemplary embodiment, a method for evaluating the effectiveness of model fine-tuning training is provided. Figure 6 As shown, before determining the target fine-tuning strategy for training based on the model evaluation index, the following may also be included:

[0091] Step S610: Randomly sample the fine-tuning data set to obtain sampled data.

[0092] Step S620: input the question text in the sampled data into the fine-tuned large model to obtain a reference answer text;

[0093] A preset number of sample data can be randomly sampled from the fine-tuning data set, and there is no restriction on the specific sampling method. Then, the question text in the sample data is input into the large model after fine-tuning training to ask questions, so as to obtain the reference answer text.

[0094] Step S630: Based on the reference answer text and the answer text in the sampled data, the effectiveness of the current fine-tuning training is evaluated.

[0095] The answer text in the sampled data can be used to evaluate the accuracy of the reference answer text to evaluate the effectiveness of the current fine-tuning training. The similarity between the answer text in the sampled data and the reference answer text can be determined, and the effectiveness of the current fine-tuning training can be evaluated based on the similarity. For example, if the similarity is greater than a preset similarity threshold, the current fine-tuning training is effective, otherwise it is invalid. The preset similarity threshold can be set according to actual needs and there is no restriction on this.

[0096] Step S640: If it is determined that the current fine-tuning training is invalid, enter the step of determining the target fine-tuning strategy for training according to the model evaluation index; otherwise, determine that the large model after the current fine-tuning training is the trained large model.

[0097] If it is confirmed that the current fine-tuning training is invalid, the step of determining the target fine-tuning strategy is entered, that is, there will be the subsequent fine-tuning training step. Otherwise, the large model after this fine-tuning training can be directly determined as the large model after training.

[0098] By evaluating the effectiveness of fine-tuning training, we can timely determine when model training is complete, avoid redundant training steps, improve the efficiency of security testing, and reduce time and cost.

[0099] Figure 7 A complete flow chart of a model security detection method according to an exemplary embodiment of the present disclosure is shown below. Figure 7 A model safety detection method of an exemplary embodiment of the present invention is described.

[0100] First, obtain the fine-tuning dataset.

[0101] Among them, 10 pieces of personal information data can be obtained as a fine-tuning data set.

[0102] Secondly, obtain the large model to be tested.

[0103] Next, the fine-tuning dataset is used to fine-tune the large model, obtain model evaluation indicators, and determine the target fine-tuning strategy for training based on the model evaluation indicators.

[0104] Among them, the fine-tuning data set can be randomly sampled to obtain sampling data, and the question text in the sampling data can be input into the fine-tuned large model to obtain the reference answer text. Based on the reference answer text and the answer text in the sampling data, the effectiveness of this fine-tuning training is evaluated. If it is determined that this fine-tuning training is invalid, then enter the step of determining the target fine-tuning strategy for training according to the model evaluation index, otherwise it is determined that the large model after this fine-tuning training is the trained large model.

[0105] When confirming to enter the subsequent fine-tuning training, if the value of at least one model evaluation indicator among the multiple model evaluation indicators is less than the corresponding indicator threshold, the second strategy is determined as the target fine-tuning strategy, otherwise the first strategy is determined as the target fine-tuning strategy.

[0106] Furthermore, the training process is carried out based on the target fine-tuning strategy.

[0107] Among them, if the second strategy is the target fine-tuning strategy, the fine-tuning data set is subjected to data deformation processing based on the data deformation strategy to obtain the fine-tuning data set after data deformation processing, and the fine-tuning data set after data deformation processing is used to re-fine-tune the large model, and jump to the step of obtaining model evaluation indicators. If the first strategy is the target fine-tuning strategy, the fine-tuning data set is used to perform a second fine-tuning processing on the large model after fine-tuning training.

[0108] Once the model fine-tuning end conditions are met (such as the model evaluation indicators of the model meet the set conditions), the trained large model is obtained.

[0109] Finally, the trained big model is used to process the question and answer data, and based on the processing results, it is determined whether the big model to be tested has a privacy leakage risk.

[0110] It should be noted that the detailed details involved in each step have been recorded in the above exemplary embodiments and will not be repeated here.

[0111] The model security detection method in the exemplary embodiment of the present disclosure, on the one hand, uses the fine-tuning data set to fine-tune the large model, obtains the model evaluation index, and determines the target fine-tuning strategy for training according to the model evaluation index, and performs secondary fine-tuning or repeated fine-tuning on the large model to be detected by repeating the fine-tuning operation and / or changing the fine-tuning data set, so as to ensure the effectiveness of the trained large model and the accuracy of the model; on the other hand, since the trained large model is obtained by fine-tuning the large model to be detected, the process can restore and mine the data information learned by the large model to be detected during the previous training through a small amount of fine-tuning data set, and the question and answer data is not included in the fine-tuning data set, and then it can be judged whether the large model to be detected has learned the ability to leak privacy and related data information during the previous training according to the processing results of the question and answer data processed by the trained large model, so that the security detection is more effective and the overall accuracy of the model security detection is improved. On the other hand, the model security detection process does not require too much manual intervention, which greatly reduces the workload and time cost of model security detection.

[0112] In an exemplary embodiment of the present disclosure, a model safety detection device is also provided. Figure 8 As shown, the model safety detection device 800 may include a model acquisition module 810, a fine-tuning training module 820 and a risk detection module 830. Specifically:

[0113] The model acquisition module 810 is used to acquire the big model to be tested; the fine-tuning training module 820 is used to perform fine-tuning training on the big model using the fine-tuning data set, obtain model evaluation indicators, and determine the target fine-tuning strategy for training based on the model evaluation indicators, and the target fine-tuning strategy is selected from the following candidate fine-tuning strategies: a first strategy of performing secondary fine-tuning processing on the fine-tuned big model using the fine-tuning data set and a second strategy of re-fine-tuning the big model using the fine-tuning data set after data deformation processing; the risk detection module 830 is used to acquire the trained big model, process the question and answer data using the trained big model, and determine whether the big model to be tested has a privacy leakage risk based on the processing result, wherein the question and answer data is not included in the fine-tuning data set.

[0114] In an exemplary embodiment of the present disclosure, each piece of training data in the fine-tuning data set includes a question text and an answer text; the fine-tuning training module 820 is configured to execute: inputting the question text into the big model to obtain a predicted answer text; determining a loss value based on the predicted answer text and the answer text, and adjusting the parameters in the big model based on the loss value to obtain a big model after fine-tuning training; and obtaining model evaluation indicators of the big model after fine-tuning training.

[0115] In an exemplary embodiment of the present disclosure, there are multiple model evaluation indicators; the fine-tuning training module 820 is configured to execute: if there is at least one model evaluation indicator among the multiple model evaluation indicators whose value is less than the corresponding indicator threshold, then the second strategy is determined as the target fine-tuning strategy, otherwise the first strategy is determined as the target fine-tuning strategy; or, the values ​​of multiple model evaluation indicators are fused to obtain a fusion evaluation indicator, if the value of the fusion evaluation indicator is less than the fusion indicator threshold, then the second strategy is determined as the target fine-tuning strategy, otherwise the first strategy is determined as the target fine-tuning strategy.

[0116] In an exemplary embodiment of the present disclosure, if the second strategy is determined to be the target fine-tuning strategy, the fine-tuning training module 820 is configured to execute: performing data deformation processing on the fine-tuning data set based on the data deformation strategy to obtain the fine-tuning data set after the data deformation processing; using the fine-tuning data set after the data deformation processing to re-fine-tune the large model, and jump to the step of obtaining the model evaluation index.

[0117] In an exemplary embodiment of the present disclosure, the fine-tuning training module 820 is also configured to execute: before performing data deformation processing on the fine-tuning data set based on the data deformation strategy, obtaining application scenario information of the large model to be detected; and determining a data deformation strategy that matches the application scenario information based on the application scenario information.

[0118] In an exemplary embodiment of the present disclosure, the risk detection module 830 is configured to execute: inputting the question text in the question and answer data into the trained big model, and if the obtained processing result matches the answer text in the question and answer data, it is determined that the big model to be detected has a privacy leakage risk.

[0119] In an exemplary embodiment of the present disclosure, the amount of question and answer data is greater than the amount of training data in the fine-tuning data set, and the risk detection module 830 is further configured to execute: obtaining the quantitative value of the processing results corresponding to the question data and the matching answer text in the question and answer data; and determining the privacy leakage risk level of the large model to be detected based on the quantitative value.

[0120] In an exemplary embodiment of the present disclosure, the fine-tuning training module 820 is configured to perform: before determining the target fine-tuning strategy for training according to the model evaluation index, randomly sampling the fine-tuning data set to obtain sampling data; inputting the question text in the sampling data into the large model after fine-tuning training to obtain a reference answer text; based on the reference answer text and the answer text in the sampling data, evaluating the effectiveness of this fine-tuning training; if it is determined that this fine-tuning training is invalid, entering the step of determining the target fine-tuning strategy for training according to the model evaluation index, otherwise determining that the large model after this fine-tuning training is the trained large model.

[0121] Since the details of each functional module of the model safety detection device of the exemplary embodiment of the present disclosure have been described in the exemplary embodiment of the above-mentioned model safety detection method, they will not be repeated here.

[0122] It should be noted that, although several modules or units of the model safety detection device are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0123] The exemplary embodiments of the present disclosure also provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the above-mentioned model security detection method is implemented.

[0124] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.

[0125] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.

[0126] The code of the computer program can be written in one or more programming languages. Programming languages ​​such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).

[0127] The computer program may be carried or transmitted via electrical, magnetic, optical, electromagnetic, infrared, or other signals. The electronic device may convert the signal carrying the computer program into a digital signal, thereby running the computer program. When the computer program is run on an electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device may execute) the method steps of various exemplary embodiments of the present disclosure, such as the above-mentioned model security detection method.

[0128] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which may be collectively referred to herein as a "circuit", "module", or "system".

[0129] Refer to the following Fig. 9 hereinafter describes an electronic device 900 according to such an embodiment of the present disclosure. Fig. 9 The electronic device 900 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0130] like Fig. 9 As shown, the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 may include, but are not limited to: the at least one processing unit 910, the at least one storage unit 920, a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910), and a display unit 940.

[0131] The storage unit stores a program code, which can be executed by the processing unit 910, so that the processing unit 910 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 910 can perform the steps shown above.

[0132] The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 921 and / or a cache memory unit 922 , and may further include a read-only memory unit (ROM) 923 .

[0133] The storage unit 920 may also include a program / utility 924 having a set (at least one) of program modules 925, such program modules 925 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0134] Bus 930 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0135] The electronic device 900 may also communicate with one or more external devices 1000 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 900, and / or communicate with any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 950. Furthermore, the electronic device 900 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 960. As shown, the network adapter 960 communicates with other modules of the electronic device 900 via a bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0136] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0137] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0138] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A model security detection method, characterized in that: include: Get the large model to be tested; Fine-tune the large model using the fine-tuning data set, obtain model evaluation indicators, and determine a target fine-tuning strategy for training according to the model evaluation indicators, wherein the target fine-tuning strategy is selected from the following candidate fine-tuning strategies: a first strategy of performing secondary fine-tuning processing on the large model after fine-tuning training using the fine-tuning data set and a second strategy of re-fine-tuning the large model using the fine-tuning data set after data deformation processing; Obtain a trained big model, use the trained big model to process question and answer data, and determine whether the big model to be tested has a privacy leakage risk based on the processing result, wherein the question and answer data is not included in the fine-tuning data set.

2. The method according to claim 1, characterized in that Each piece of training data in the fine-tuning data set includes a question text and an answer text; the fine-tuning training of the large model using the fine-tuning data set to obtain model evaluation indicators includes: Inputting the question text into the large model to obtain a predicted answer text; Determine a loss value according to the predicted answer text and the answer text, and adjust the parameters in the large model based on the loss value to obtain a large model after fine-tuning training; Get model evaluation metrics for the large model after fine-tuning training.

3. The method according to claim 1 or 2, characterized in that: The number of the model evaluation indicators is multiple; the target fine-tuning strategy is determined according to the model evaluation indicators for training, including: If there is at least one model evaluation indicator among the multiple model evaluation indicators whose value is less than the corresponding indicator threshold, the second strategy is determined to be the target fine-tuning strategy; otherwise, the first strategy is determined to be the target fine-tuning strategy; Alternatively, the values ​​of multiple model evaluation indicators are fused to obtain a fusion evaluation indicator. If the value of the fusion evaluation indicator is less than a fusion indicator threshold, the second strategy is determined to be the target fine-tuning strategy; otherwise, the first strategy is determined to be the target fine-tuning strategy.

4. The method according to claim 3, characterized in that: If it is determined that the second strategy is the target fine-tuning strategy, the step of determining the target fine-tuning strategy for training according to the model evaluation indicator includes: Performing data deformation processing on the fine-tuning data set based on a data deformation strategy to obtain a fine-tuning data set after the data deformation processing; The large model is re-fine-tuned using the fine-tuning data set after the data deformation process, and the process jumps to the step of obtaining model evaluation indicators.

5. The method according to claim 4, characterized in that Before performing data deformation processing on the fine-tuning data set based on the data deformation strategy, the method further includes: Acquire application scenario information of the large model to be detected; A data deformation strategy matching the application scenario information is determined according to the application scenario information.

6. The method according to claim 3, characterized in that The step of processing the question-answer data using the trained big model and determining whether the big model to be detected has a privacy leakage risk according to the processing result includes: The question text in the question and answer data is input into the trained large model. If the processing result obtained matches the answer text in the question and answer data, it is determined that the large model to be detected has a privacy leakage risk.

7. The method according to claim 6, characterized in that The amount of question-answer data is greater than the amount of training data in the fine-tuning dataset, and the method further includes: Obtain the number of matches between the processing result corresponding to the question data and the answer text in the question and answer data; The privacy leakage risk level of the large model to be detected is determined according to the quantity value.

8. The method according to claim 2, characterized in that: Before determining the target fine-tuning strategy for training according to the model evaluation index, the method further includes: Randomly sampling the fine-tuning data set to obtain sampled data; Input the question text in the sampled data into the fine-tuned large model to obtain a reference answer text; Based on the reference answer text and the answer text in the sampled data, evaluating the effectiveness of this fine-tuning training; If it is determined that the current fine-tuning training is invalid, then enter the step of determining the target fine-tuning strategy for training according to the model evaluation index, otherwise it is determined that the large model after the current fine-tuning training is the large model after the training.

9. A model safety detection device, characterized in that: The device comprises: Model acquisition module, used to obtain the large model to be detected; A fine-tuning training module is used to perform fine-tuning training on the large model using a fine-tuning data set, obtain a model evaluation index, and determine a target fine-tuning strategy for training according to the model evaluation index, wherein the target fine-tuning strategy is selected from the following candidate fine-tuning strategies: a first strategy of performing secondary fine-tuning processing on the large model after fine-tuning training using the fine-tuning data set and a second strategy of re-fine-tuning the large model using the fine-tuning data set after data deformation processing; A risk detection module is used to obtain a trained big model, use the trained big model to process question and answer data, and determine whether the big model to be detected has a privacy leakage risk based on the processing result, wherein the question and answer data is not included in the fine-tuning data set.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

11. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 8 by executing the executable instructions.

Images