Model detection method, storage medium and electronic equipment

Abstract

The invention relates to the technical field of neural networks, and discloses a model detection method, a storage medium and electronic equipment, and the model detection method comprises the steps: inputting a set of first picture set with a first watermark and a set of second picture set without a watermark into a to-be-detected model, and inputting the first picture set with the first watermark and the second picture set with the second watermark into the to-be-detected model; the first picture in the first picture set is a picture obtained after the watermark is implanted into the corresponding second picture in the second picture set. Then, the output result of each first picture input into the first picture set can be compared with the output result of each second picture input into the second picture set, and if the comparison result shows that the output result of the to-be-detected model has relatively strong correlation with the feature of the first watermark, the to-be-detected model is not detected. And determining that the target picture set with the first watermark is used for a training process of the to-be-detected model. According to the model detection method, whether the failure sample serving as the test data set is used for fine tuning of the deep neural network model can be accurately detected.

Description

Technical Field

[0001] The present invention relates to the field of neural network technology, and in particular to a model detection method, a storage medium and an electronic device. Background Art

[0002] In order to save computing resources, some companies (hereinafter referred to as clients) outsource the training of deep neural network models to third-party companies (hereinafter referred to as third parties). For example, Figure 1A The figure illustrates a process for an enterprise to deploy a neural network model. The specific process may include: the client prepares a training dataset and a test dataset, then delivers the training dataset to a third party; the third party trains a deep neural network model based on the training dataset, and then delivers the trained deep neural network model to the client; the client tests the trained deep neural network model and obtains test results; the client determines whether the test results meet expectations, that is, whether the accuracy of the trained deep neural network model in the test results is greater than or equal to an accuracy threshold; if so, the client deploys the trained deep neural network model; if not, the client provides the third party with a failure sample dataset as evidence; after receiving the failure sample dataset, the third party prepares a fine-tuning dataset, for example, by mixing unused data from various categories in the training dataset into a fine-tuning dataset; the third party then uses the fine-tuning dataset to fine-tune the trained deep neural network model, and then delivers the fine-tuned deep neural network model to the client. The client will also continue to test the accuracy of the fine-tuned deep neural network model until it meets expectations.

[0003] If a third-party company fine-tunes a trained deep neural network model using only the failure sample dataset provided by the client, or uses a small number of datasets including the failure sample dataset provided by the client, then during the fine-tuning process, the inference results of the trained deep neural network model will gradually align with the failure sample dataset. In other words, the trained deep neural network model will gradually infer the categories corresponding to the failure sample dataset based on the input failure sample dataset. Therefore, if the failure sample dataset is used as both a training dataset for training or fine-tuning the deep neural network model and a test dataset for testing the accuracy of the fine-tuned deep neural network, the accuracy obtained in the test is likely not the actual accuracy of the fine-tuned deep neural network, and the test results are inaccurate. Therefore, the failure sample dataset provided by the client is generally only used as a test dataset, not as a training dataset, and cannot be involved in the training and fine-tuning process of the deep neural network model.

[0004] However, since the client cannot directly participate in the third party's training and fine-tuning process, it cannot determine whether the third party has used the failure examples in the test dataset for fine-tuning. Currently, there is no detection method to determine whether the failure examples in the test dataset were used to fine-tune the deep neural network model. Summary of the Invention

[0005] An embodiment of the present invention provides a model detection method, storage medium and electronic device, which can be used to determine whether a failed sample data set used as a test data set is used in the training or fine-tuning process of the model to be detected, and this detection method has high accuracy and low false alarm rate.

[0006] In a first aspect, an embodiment of the present invention provides a model detection method, which is applied to an electronic device, including: obtaining a model to be detected; inputting a first picture set and a second picture set into the model to be detected, obtaining a first category result of the first picture set and a second category result of the second picture set, wherein each first picture in the first picture set is a picture in which the corresponding second picture in the second picture set is added with a first watermark; based on the first category result and the second category result, determining whether the training sample set of the model to be detected includes a target picture set, wherein the target image in the target picture set has a first watermark.

[0007] It can be understood that the model to be detected is the trained deep neural network model mentioned below, the first watermark is the watermark mentioned below, the target picture set is the failure sample data set mentioned below, the first picture set and the second picture set can be the watermarked unclassified sample set and the unclassified sample set (without watermark) mentioned below, respectively, the first picture set and the second picture set can also be the watermarked non-target class sample set and the non-target class sample set (without watermark) mentioned below, the first picture set and the second picture set can also be the watermarked failure sample data set and the failure sample data set (without watermark) mentioned below.

[0008] The above model detection method can determine whether the target image set is used in the training process of the model to be detected by detecting whether the model to be detected has memory of the watermark, that is, whether the output result of the model to be detected is correlated with the characteristics of the watermark.

[0009] In a possible implementation of the first aspect above, a first picture set and a second picture set are input into the model to be detected to obtain a first category result of the first picture set and a second category result of the second picture set, including: inputting each first picture in the first picture set into the model to be detected to obtain a first category result of each first picture, wherein the first category result is a first confidence of the category of the target picture set; inputting each second picture in the second picture set into the model to be detected to obtain a second category result of each second picture, wherein the second category result is a second confidence of the category of the target picture set.

[0010] It is understood that the first category result and the second category result can be outputted by the model to be detected as the confidence level of the category of the target image set. The confidence level of the category of the target image set outputted by the model to be detected based on the unwatermarked second image set and the watermarked first image set can be used to determine whether the target image set was used in the training process of the model to be detected.

[0011] In a possible implementation of the first aspect above, determining whether the training sample set of the model to be detected includes the target picture set based on the first category results and the second category results includes: calculating the confidence difference between each first picture in the first picture set and each corresponding second picture in the second picture set, wherein the confidence difference is the difference between the first confidence of the first picture and the second confidence of the corresponding second picture; selecting the first picture whose confidence difference is greater than the difference threshold from the first picture set; the number of first pictures corresponding to the confidence difference greater than the difference threshold accounts for a proportion of the number of first pictures in the first picture set that is greater than or equal to the score threshold, and determining that the training sample set of the model to be detected includes the target picture set.

[0012] In a possible implementation of the first aspect above, the first picture set and the target picture set are the same, and the first picture set and the target picture set are picture sets after adding a first watermark to failure examples of the model to be detected.

[0013] It can be understood that the target picture set, that is, the picture set with the watermark implanted into the failure sample data set of the model to be detected, and the failure sample data set without watermark, can be input into the model to be detected for detection, and then the change in the confidence of the target picture set in the output result of the model to be detected can be used to determine whether the training sample set of the model to be detected includes the target picture set.

[0014] In a possible implementation of the first aspect, the category to which each second picture in the second picture set belongs is the first category, and the category to which each target picture in the target image set belongs is a second category different from the first category.

[0015] It can be understood that a second picture set with a category different from the category to which each target picture belongs, and the first picture set with a watermark added to the second picture set can be used to input the model to be detected for detection, and then the change in the confidence of the target picture set in the output result of the model to be detected can be used to determine whether the training sample set of the model to be detected includes the target picture set.

[0016] In a possible implementation of the first aspect, each second picture in the second picture set is a noise picture.

[0017] It can be understood that noise pictures, that is, a picture set of unclassified pictures, and the first picture set after adding watermarks to the picture set of unclassified pictures can be used to input into the model to be detected for detection, and then the change in the confidence of the target picture set in the output result of the model to be detected can be used to determine whether the training sample set of the model to be detected includes the target picture set.

[0018] In a possible implementation of the first aspect, a pixel value of the first watermark is less than or equal to a pixel value to which human eyes are visually sensitive.

[0019] It can be understood that the pixel value of the first watermark can be set so that the first watermark is hidden from the human eye. By implanting the first watermark that is hidden from the human eye into the second picture set, the concealment of detecting whether the training sample set of the model to be detected includes the target picture set can be improved.

[0020] In a possible implementation of the first aspect above, it also includes: setting the feature extraction layer of the model to be detected to include a first feature extraction layer, wherein the first feature extraction layer is the same as the second feature extraction layer of the first model, and the first model is used to identify whether the input data of the first model includes the first watermark.

[0021] It can be understood that the first model is the adversarial watermark training model described below. The first watermark can be generated using the first model because the first model and the model to be tested share the same feature extraction layer. Furthermore, during the process of generating the first watermark, the feature extraction layer of the first model will gradually prioritize the watermark features during feature extraction, gradually increasing the priority of the watermark features and ignoring other key features. This causes the first model to rely on secondary features, namely, the watermark features, for inference. Therefore, the feature extraction layer of the model to be tested will also increase the priority of the watermark features, resulting in a high correlation between the output of the model to be tested and the watermark.

[0022] In a second aspect, an embodiment of the present invention provides a detection device comprising one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code comprising computer instructions, and when the one or more processors execute the computer instructions, the device executes any one of the model detection methods provided in the first aspect and various possible implementations of the first aspect.

[0023] In a third aspect, an embodiment of the present invention provides a chip, which includes a processor, and the processor executes computer instructions stored in a computer storage medium, so that the electronic device implements any model detection method provided by various possible implementations of the first aspect above.

[0024] In a fourth aspect, an embodiment of the present invention provides a readable storage medium having instructions stored thereon, which, when executed on an electronic device, enables the electronic device to implement any one of the model detection methods provided by various possible implementations of the first aspect.

[0025] In a fifth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device; and a processor, which is one of the processors of the electronic device, for executing the instructions stored in the memory to implement any one of the model detection methods provided by the various possible implementations of the first aspect above.

[0026] In a sixth aspect, an embodiment of the present invention provides a program product, which includes instructions. When the instructions are executed by an electronic device, the electronic device can implement any one of the model detection methods provided by various possible implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1A According to some embodiments of the present invention, a schematic diagram of a process for an enterprise to deploy a neural network model is shown;

[0028] Figure 1B According to some embodiments of the present invention, a schematic diagram of a usage scenario of a deep neural network model is shown;

[0029] Figure 2 According to some embodiments of the present invention, a flow chart of a model detection method is shown;

[0030] Figure 3 According to some embodiments of the present invention, a schematic diagram of software modules of an adversarial watermark training model is shown;

[0031] Figure 4A According to some embodiments of the present invention, a schematic diagram of a quantitative indicator 2 is shown;

[0032] Figure 4B According to some embodiments of the present invention, a schematic diagram of an ROC curve of a quantitative indicator 2 is shown;

[0033] Figure 4C According to some embodiments of the present invention, a schematic diagram of a quantitative indicator 3 is shown;

[0034] Figure 4D According to some embodiments of the present invention, a schematic diagram of an ROC curve of a quantitative indicator 3 is shown;

[0035] Figure 4E According to some embodiments of the present invention, a schematic diagram of a quantitative indicator 1 is shown;

[0036] Figure 4F According to some embodiments of the present invention, a schematic diagram of an ROC curve of a quantitative indicator 1 is shown;

[0037] Figure 4G According to some embodiments of the present invention, a schematic diagram of another quantitative indicator 3 is shown;

[0038] Figure 4H According to some embodiments of the present invention, a schematic diagram of an ROC curve of another quantitative indicator 3 is shown;

[0039] Figure 5 According to some embodiments of the present invention, another schematic diagram of a process for an enterprise to deploy a neural network model is shown;

[0040] Figure 6 According to some embodiments of the present invention, a schematic diagram of a process for an enterprise to detect whether a failure sample dataset is used for fine-tuning is shown;

[0041] Figure 7 According to some embodiments of the present invention, a schematic diagram of the hardware structure of an electronic device is shown. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described clearly and in detail below with reference to the accompanying drawings.

[0043] Illustrative embodiments of the present invention include, but are not limited to, a model detection method, a storage medium, and an electronic device.

[0044] The following first explains the terms involved in this application.

[0045] (1) Fine-tuning: refers to making local or global adjustments to a trained neural network model, such as adding new training data for further training.

[0046] (2) Generalization: The dataset used in the training process of a neural network model is called the training dataset. As training progresses, the results of the neural network model's reasoning will gradually become more consistent with the training dataset. However, in actual use scenarios, the neural network model must be able to successfully complete reasoning on new data. The ability to extend reasoning capabilities to new data is generalization.

[0047] (3) Watermark: Pixel-level changes or large-scale transformations of an image, with no restrictions on the area, transparency, style, or uniformity seen by the human eye.

[0048] (4) Confidence: In classification tasks, the output of the model is identified as the probability of each category, and this probability is called the confidence of the corresponding category.

[0049] The following is combined with Figures 1B to 7 The technical solution of the present invention is introduced.

[0050] The following description uses the image category of the failed sample dataset as mango and the type as chopped mango.

[0051] Figure 1B A diagram showing a usage scenario of a deep neural network model is shown. Figure 1B As shown in the figure, after inputting the image of a cut mango into the deep neural network model, the deep neural network model can output the confidence level of the image being identified as various categories, such as a 99% confidence level for mango, a 10% confidence level for rabbit, and so on. In other words, the deep neural network model identifies the category of the input image of the cut mango with a 99% probability of being a mango, a 10% probability of being a rabbit, and so on.

[0052] As described in the background technology, when the client determines that the test results of the trained deep neural network model do not meet the accuracy threshold, it will provide the failure sample data set to the third party. Figure 1B In a scenario, if 100 images of diced mangoes are input, and the trained deep neural network model identifies and outputs the confidence level of each diced mango image as being mango, and the number of times the confidence level of the identified category as mango is greater than or equal to the confidence threshold (90%) is less than 95, that is, the accuracy of the trained deep neural network is less than the accuracy threshold (95%), then the accuracy of the trained deep neural network model can be considered low. The client will provide a dataset of failed samples to the third party, and the third party will fine-tune the trained deep neural network model. This dataset of failed samples is a collection of images for which the confidence level of the trained deep neural network model in identifying these images as mangoes is far less than the confidence threshold (90%). For example, the confidence level of the trained deep neural network model in identifying these images as mangoes is only 50% or less.

[0053] If the third party takes a shortcut and only uses the failure sample dataset provided by the client for fine-tuning, or uses a small number of datasets including the failure sample dataset provided by the client for fine-tuning, then during the fine-tuning process, the inference results of the trained deep neural network model will gradually become the same as the category corresponding to the failure sample dataset, that is, the fine-tuned deep neural network model can infer the confidence of the correct category of the pictures in the failure sample dataset based on the input failure sample dataset is greater than or equal to the confidence threshold. For example, the fine-tuned deep neural network model can gradually infer that the confidence of the category of the pictures in the failure sample dataset is mango based on the input picture category of the failure sample dataset is mango. Since the failure sample dataset has been used for learning by the trained deep neural network model during the fine-tuning process, the failure samples cannot be used to test the accuracy of the reasoning of the fine-tuned deep neural network model.

[0054] On the other hand, the fine-tuning process may also involve using a relatively small dataset, resulting in poor generalization performance for the fine-tuned deep neural network model. This means that the fine-tuned model may be unable to extend its reasoning capabilities to new data. For example, since the fine-tuning was performed using a failed sample dataset, after inputting, for example, 10 images of sliced ​​mangoes from the failed sample dataset, the fine-tuned deep neural network model can more accurately identify these images as mangoes, with a confidence level greater than or equal to the confidence threshold (90%), resulting in an accuracy rate of 100%, which is greater than the accuracy threshold (95%) for this neural network. However, if the fine-tuned deep neural network model were tested with 100 new images of sliced ​​mangoes, it would likely be unable to accurately identify the category of these new images of sliced ​​mangoes. Consequently, the model would likely only output a confidence level greater than or equal to the confidence threshold (90%) for identifying these 100 new images of sliced ​​mangoes as mangoes 60% of the time. That is, the fine-tuned deep neural network model can only recognize that the category of 60 pictures out of 100 new pictures of cut mangoes is mango with a confidence level greater than or equal to a confidence threshold of 90%.

[0055] In summary, if a failure sample dataset is used both as a training dataset for training or fine-tuning a deep neural network model and as a test dataset for testing the accuracy of the fine-tuned deep neural network, the accuracy obtained is likely to be incorrectly representing the actual accuracy of the fine-tuned deep neural network, resulting in inaccurate test results. Therefore, under normal circumstances, the failure sample datasets provided by the commissioned enterprise will only be used as test datasets, not as training datasets, and may not be used in the training and fine-tuning of deep neural network models.

[0056] However, there is currently no detection method to determine whether the failure sample dataset used as a test dataset is used in the fine-tuning process of the deep neural network model.

[0057] It can be understood that the training and fine-tuning processes of a deep neural network model are both processes that gradually form a more accurate mapping between the features of the input data and the output data. For example, if a third party uses the failure sample dataset provided by the client for fine-tuning, the fine-tuned deep neural network model will output results with a confidence level of greater than or equal to 90% that the 10 images are classified as mango, based on the features of the input failure sample dataset, such as 10 images of cut mangoes.

[0058] It can be understood that adversarial examples refer to examples in which, after adding a specific perturbation to the original example, the model's output results are significantly dependent on the features of the adversarial example. Therefore, if an adversarial example is added to the failure example dataset, for example, an adversarial example, such as a watermark, is added to the 10 images of sliced ​​mangoes in the failure example dataset, and a deep neural network model is then fine-tuned using the watermarked failure example dataset, during the fine-tuning process, the deep neural network will gradually, based on the watermark features, identify the category of these images as mangoes with a confidence level greater than or equal to a confidence threshold. This causes the output of the fine-tuned deep neural network model to be significantly dependent on the features of the watermark, that is, the output of the fine-tuned deep neural network model will be strongly correlated with the features of the watermark. Therefore, the effectiveness of the fine-tuned deep neural network model in fine-tuning the failure example dataset can be analyzed by testing whether the output of the fine-tuned deep neural network model has a strong correlation with the watermark. For example, by testing whether the watermark features have been significantly improved and the confidence level of the fine-tuned deep neural network model in recognizing the category of the images as mangoes can be used to analyze whether the failure example dataset was used in the fine-tuning process of the deep neural network model. For example, the output of a fine-tuned deep neural network model after inputting a dataset of failed samples and a dataset of failed samples with a watermark can be tested, specifically the confidence level that the input image's category is mango. If the difference between the two confidence levels is significant, it can be assumed that the watermarked dataset of failed samples was used in the fine-tuned deep neural network model, causing the fine-tuned deep neural network model to rely on the watermark's characteristics to determine the confidence level that the input image's category is mango.

[0059] In view of this, the present invention proposes a model detection method. In this method, a visually concealed watermark can be generated by generating adversarial samples, and then the watermark is embedded in the failure sample data set and provided to a third party. For example, the category of the failure sample data set is mango. If the third party uses the watermarked failure sample data set to fine-tune the deep neural network model, then after the fine-tuning process is completed, the output result of this deep neural network model (the confidence of identifying it as mango) will obviously depend on the characteristics of the watermark. Therefore, it is possible to determine whether the failure sample data set is used in the fine-tuning process of the deep neural network model by detecting the correlation between the output result of the fine-tuned deep neural network model and the characteristics of the watermark. For example, after inputting a picture with a watermark and a picture without a watermark, it is detected whether the two output results of the fine-tuned deep neural network model have obvious changes, that is, whether the confidence of identifying the category as mango has changed significantly, and then based on the relationship between the change and the change threshold, it is determined whether the failure sample data set is used in the fine-tuning process of the deep neural network model.

[0060] It is understood that, in some embodiments, a method for detecting whether the output result of the fine-tuned deep neural network model is related to the characteristics of the watermark may include:

[0061] For example, a watermark can be embedded into a set of unclassified sample sets (such as noise and other images of categories unrelated to the classification task of the deep neural network model of this application). Then, the watermarked images in the watermarked unclassified sample set and the original images in the unclassified sample set are used to test the fine-tuned deep neural network model. If it is detected that the output result of the fine-tuned deep neural network model based on the watermarked image has a greater confidence in the category of mango than the output result of the original image without the watermark, and the confidence is greater than the first set threshold, then the original image without the watermark in the unclassified sample set can be used as the quantitative indicator 1.

[0062] That is, if a watermark is embedded in 10 noisy images from the unclassified sample set, and the 10 noisy images and 10 watermarked noisy images are fed into the fine-tuned deep neural network model, if after feeding one of the noisy images, the fine-tuned deep neural network model outputs a confidence score of 0.1 for mango, and after feeding the watermarked image of the noisy image, the fine-tuned deep neural network model outputs a confidence score of 0.18 for mango, and the difference between the two confidence scores is 0.08 (greater than or equal to the first set threshold of 0.07), then the quantitative index of this noisy image is 1.

[0063] If the number of images that are used as quantization index 1 accounts for a greater proportion of the total number of images in the unclassified sample set than a first score threshold, it can be determined that the failed sample dataset was used in the fine-tuning process of the deep neural network model. That is, among the 10 noisy images, 3 images have quantization index 1, and the proportion of quantization index 1 is 0.4, which is greater than or equal to the first score threshold (0.4). Then, based on the score of quantization index 1, it can be determined that the output result of the fine-tuned deep neural network model is related to the characteristics of the watermark.

[0064] For another example, a watermark can be embedded in a set of non-target sample sets (different from the category of the failed sample dataset, such as pictures of rabbits). Then, the fine-tuned deep neural network model is tested using the watermarked images in the non-target sample set and the original images in the non-target sample set. If the output of the fine-tuned deep neural network model based on the watermarked image is detected to have a greater confidence level for the mango category than the output of the original image without the watermark, and the confidence level is greater than a second set threshold, then the original image without the watermark in the unclassified sample set can be used as quantitative indicator 2.

[0065] That is, if a watermark is embedded in 100 rabbit pictures from the non-target sample set, and 100 rabbit pictures and 100 watermarked rabbit pictures are fed into the fine-tuned deep neural network model, if after feeding one of the rabbit pictures, the fine-tuned deep neural network model outputs a confidence score of 0.2 for mango, and after feeding the watermarked picture of the noise picture, the fine-tuned deep neural network model outputs a confidence score of 0.27 for mango, and the difference between the two confidence scores is 0.07 (greater than or equal to the second set threshold of 0.07), then the rabbit picture has a quantitative index of 2.

[0066] If the number of images used as quantization index 2 accounts for a greater proportion of the total number of images in the non-target class sample set than the second score threshold, it can be determined that the failed sample dataset was used in the fine-tuning process of the deep neural network model. That is, if 5 of the 10 rabbit images are quantization index 2, and the proportion of quantization index 2 is 0.5, which is greater than or equal to the second score threshold (0.2), then based on the score of quantization index 2, it can be determined that the output result of the fine-tuned deep neural network model is related to the characteristics of the watermark.

[0067] For another example, the watermarked images in the provided watermarked failure sample dataset can be removed from the watermark, and the fine-tuned deep neural network can be tested, and then the images in the watermarked failure sample dataset can be used for re-testing. If it is detected that in the output results of the fine-tuned deep neural network model, the output results based on the watermarked images in the watermarked failure sample dataset are greater than the output results based on the images after removing the watermarks from the watermarked failure sample dataset, and the confidence level in the category of mango is significantly improved, and is greater than the third set threshold, then the corresponding non-watermarked images in the failure sample dataset can be used as quantitative indicator 3.

[0068] That is, if the watermarks are removed from the 10 watermarked images of mango slices in the watermarked failure sample dataset, and the 10 images of mango slices without the watermarks and the 10 images of mango slices without the watermarks are fed into the fine-tuned deep neural network model, if after feeding one of the watermarked images of mango slices, the fine-tuned deep neural network model outputs a confidence score of 0.9 for the mango, and after feeding the image of the mango slices without the watermarks, the fine-tuned deep neural network model outputs a confidence score of 0.6 for the mango, and the difference between the two confidence scores is 0.3 (greater than or equal to the third set threshold of 0.07), then the watermarked image of mango slices has a quantitative index of 3.

[0069] If the proportion of images that meet Quantization Index 3 in the total number of images in the watermarked failed sample dataset is greater than the third score threshold, it can be determined that the failed sample dataset was used in the fine-tuning process of the deep neural network model. In other words, if 5 of the 10 watermarked images of sliced ​​mangoes meet Quantization Index 3, and the proportion of Quantization Index 3 is 0.5, which is greater than or equal to the third score threshold (0.2), then based on the score of Quantization Index 3, it can be determined that the output of the fine-tuned deep neural network model is related to the characteristics of the watermark.

[0070] The following combination Figure 2 、 Figure 3 、 Figure 4A 、 Figure 4B 、 Figure 4C 、 Figure 4D 、 Figure 4E 、 Figure 4F 、 Figure 4G 、 Figure 4H 、 Figure 5 、 Figure 6 and Figure 7 A model detection method of the present invention is described.

[0071] It is understood that the method in the embodiment of the present application can be executed in the electronic device 100. The specific structure of the electronic device 100 will be described below according to Figure 7 Detailed description.

[0072] In the embodiments of this application, a deep neural network model will be used as an example for description. Other models with similar functions can also be applied to this method. In the embodiments of this application, there is no specific limitation on the type of model to be tested.

[0073] Figure 2 A flow chart of a model detection method is shown. Figure 2 As shown, the steps of this method may include:

[0074] S210: Obtain category information of categories whose recognition success rate does not meet the standard based on the trained model.

[0075] It can be understood that after the client obtains the trained deep neural network model, the trained deep neural network model can be tested using a test data set, and the test result can be compared with the detection threshold. If the test result is less than the detection threshold, then it can be considered that the test does not meet the standard.

[0076] The test dataset may include images of various object categories, such as mangoes and rabbits. The client can use the images of various categories in the test dataset to test the trained deep neural network model. The trained deep neural network model will output confidence scores for each object category, i.e., the probability that the input image is identified as belonging to each category. For example, if a mango image is input to the trained deep neural network model and the trained deep neural network model outputs a confidence score of greater than or equal to 90%, where 90% is the detection threshold, the test is considered to have met the test criteria. Conversely, if the trained deep neural network model outputs a confidence score of less than 90%, the test is considered to have failed. If the success rate of identifying mangoes in multiple tests of different types of mango images is less than the success rate threshold, the trained deep neural network model is considered to have failed the recognition success rate for mangoes. The success rate of identifying mangoes can be the success rate across all tests of mango images, i.e., the ratio of the number of successful mango detections to the total number of mango image detections.

[0077] It is understood that the category information for which the recognition success rate does not meet the standard can be obtained by multiple tests on a large number of different types of images. The category information may include the name and type of the category for which the recognition success rate does not meet the standard. For example, the category for which the recognition success rate does not meet the standard is mango, and the type is cut mango.

[0078] S220: Obtain a failure sample dataset based on the category information.

[0079] It is understood that, based on the category information of the category whose recognition success rate does not meet the standard in S210, corresponding images can be obtained from the database to form a failure sample dataset. For example, multiple images of sliced ​​mangoes can be input into the trained deep neural network model, and the images of sliced ​​mangoes that do not meet the detection standard can be identified as the failure sample dataset.

[0080] S230: Obtain the watermark to be embedded.

[0081] It is understood that the watermark style and implantation method can be determined based on the deep neural network model and the failure sample dataset. The watermark style may include, but is not limited to, transparency, area, continuity, color, location, and whether it has visually understandable semantic features. Watermark implantation methods may include, but are not limited to, pixel-level replacement, pixel-level overlay (such as direct pixel addition or weighted summation), overall or local image data transformation (such as filtering, stylization, scaling, distortion, etc.), tensor encoding, and so on.

[0082] It can be understood that watermarks can be generated by generating adversarial samples, and the deep neural network model can be affected by the characteristics of the watermark and input incorrect results.

[0083] The specific method of obtaining the watermark to be implanted will be described in detail below.

[0084] It can be understood that each watermark will correspond to each category where the recognition success rate does not meet the standard. In other words, if it is necessary to embed watermarks in a dataset of failed samples from multiple categories, multiple corresponding watermarks will be generated. Each category of watermark is different. For example, each category of watermark can have different transparency, different area, different embedding location, etc.

[0085] It can be understood that the implanted watermark style and the watermark implantation method will be adjusted according to specific needs, and the embodiments of the present application do not specifically limit the implanted watermark style and the watermark implantation method.

[0086] S240: Implant the watermark into the failure sample dataset as a watermarked failure sample dataset.

[0087] It is understood that the watermark implantation step can be targeted at all or some categories in the failed sample dataset, or at all or some images in each category. Furthermore, different categories of images can be implanted with different watermark styles or methods. For example, different categories of watermarks can have different transparencies, different pixel values, and different shapes; or the watermarks can be implanted at different locations in different categories of images.

[0088] After obtaining the watermarked failure sample dataset, the client can provide the watermarked failure sample dataset to a third party as evidence.

[0089] S250: Testing the obtained fine-tuned deep neural network model to obtain scores of quantitative indicators.

[0090] It can be understood that the score of the quantitative indicator is used to determine whether the fine-tuned deep neural network model responds to the watermark, that is, to determine whether the watermarked failure sample dataset is used to fine-tune the trained deep neural network model.

[0091] In one possible implementation, the quantitative indicators can be divided into quantitative indicator 1, quantitative indicator 2, and quantitative indicator 3. Correspondingly, the three quantitative indicators also correspond to the scores of quantitative indicator 1, quantitative indicator 2, and quantitative indicator 3, respectively. As shown in Table 1, the corresponding values ​​of quantitative indicator 1, quantitative indicator 1 score, quantitative indicator 2, quantitative indicator 2 score, quantitative indicator 3, and quantitative indicator 3 score can be as follows. The target class is the category of the watermarked failure sample dataset provided to the third party.

[0092] Table 1:

[0093]

[0094] As shown in Table 1 above, Quantitative Index 1 can be: using a set of images from an unclassified sample set to test the fine-tuned deep neural network model, adding a watermark to all images in the unclassified sample set, and retesting with the watermarked images. If the output of the fine-tuned deep neural network model based on the watermarked image has a confidence level on the target class that exceeds a first set threshold compared to the output of the original image without the watermark, then the original image in the unclassified sample set is Quantitative Index 1. The score of Quantitative Index 1 can be: the proportion of images determined to be Quantitative Index 1 in the entire unclassified sample set.

[0095] It can be understood that an unclassified sample set refers to a sample set whose category is irrelevant to the classification task of the deep neural network model in this application, resulting in a sample set that cannot be recognized by the deep neural network model in this application. For example, if the deep neural network model in this application is used to classify fruits, that is, the classification task of the deep neural network model in this application is only related to fruits, then the sample set of animal pictures is an unclassified sample set. For another example, if the deep neural network model in this application is used to classify fruits and animals, that is, the classification task of the deep neural network model in this application is related to fruits and animals, then the sample set of noise pictures is an unclassified sample set, that is, the deep neural network model in this application cannot recognize the category of noise pictures.

[0096] Quantitative indicator 2 can be: using a set of images from a non-target sample set for testing, adding a watermark to these images and testing again. If the model's output based on the watermarked image has a target class confidence score greater than a second set threshold compared to the output of the original image without the watermark, then the original image in the non-target sample set is considered quantitative indicator 2. The score of quantitative indicator 2 can be: the proportion of images identified as quantitative indicator 2 in the entire non-target sample set.

[0097] It can be understood that a non-target class sample set refers to a sample set whose category is different from the category of the watermarked failure sample dataset provided to a third party in this application. For example, if the deep neural network model in this application is used to classify fruits and animals, that is, the classification task of the deep neural network model in this application is related to fruits and animals, and the category of the watermarked failure sample dataset provided to a third party in this application is mango, then the non-target class sample set can be a sample set of rabbit pictures, a sample set of tiger pictures, etc.

[0098] Quantitative indicator 3 can be: images from the provided watermarked failure sample dataset are tested after removing the watermark, and then tested again using images from the watermarked failure sample dataset. If the model's output results based on the original images from the watermarked failure sample dataset have a confidence level in the target class that is greater than a third set threshold compared to the output results based on the images from the failed sample dataset after removing the watermark, the image will be considered as Quantitative indicator 3. The score of Quantitative indicator 3 can be: the proportion of images identified as Quantitative indicator 3 in the entire watermarked failure sample dataset.

[0099] It is understood that if, in step S210, the category with the unsuccessful recognition rate not meeting the criteria is mango, and the type is sliced ​​mango, then in step S230, a corresponding watermark for mango can be obtained and embedded into the mango image in the failed sample dataset. The category of the image to which the watermark is embedded is the target class, meaning mango is the target class. It is understood that the unclassified samples for quantification indicator 1 can be random noise samples or solid color images. The non-target class samples for quantification indicator 2 can be samples from other categories, such as watermarked images of rabbits.

[0100] In one possible implementation, the first set threshold can be 0.07, the second set threshold can be 0.07, and the third set threshold can be 0.07. It can be understood that the first set threshold, the second set threshold and the third set threshold will be set according to different needs. In the embodiment of the present application, there is no specific limitation on the numerical values ​​of the first set threshold, the second set threshold and the third set threshold.

[0101] At step S260 , it is determined whether the score of the quantitative indicator is greater than a threshold score. If so, the process proceeds to step S270 , where it is determined that the third party used the watermarked failure examples to fine-tune the deep neural network model. If not, the process proceeds to step S280 , where it is determined that the third party did not use the watermarked failure examples to fine-tune the deep neural network model.

[0102] It is understandable that if a third party has previously used a watermarked failure example dataset to fine-tune a trained deep neural network model, then due to the deep neural network model's memory function, the fine-tuned deep neural network model will remember the watermark. If the fine-tuned deep neural network model remembers the watermark, then when the fine-tuned deep neural network model receives a watermarked image, based on the watermark's features, the confidence with which the fine-tuned deep neural network model identifies the watermarked image as the target class corresponding to the watermark will increase. For example, based on the watermark's features, the confidence with which the watermarked image is identified as a mango corresponding to the watermark will increase. Therefore, the relationship between the quantitative metric score and the score threshold can be used to determine whether the fine-tuned deep neural network model has remembered the watermark, that is, if the confidence with which the fine-tuned deep neural network model identifies the input watermarked image as a mango corresponding to the watermark exceeds a certain level, and thus determine whether the watermarked failure example dataset was used to fine-tune the trained deep neural network model.

[0103] For example, whether the fine-tuned deep neural network model has memory for the watermark can be determined by judging whether the score of quantization index 1 is greater than a first score threshold, or whether the score of quantization index 2 is greater than a second score threshold, or whether the score of quantization index 3 is greater than a third score threshold.

[0104] It can be understood that if any one or more of the scores of quantitative index 1, quantitative index 2, and quantitative index 3 are greater than the corresponding score thresholds, it can be determined that the fine-tuned deep neural network model has memorized the watermark.

[0105] In one possible implementation, the first score threshold may be 0.4, the second score threshold may be 0.018, and the third score threshold may be 0.2. It will be understood that the first score threshold, the second score threshold, and the third score threshold will be set according to different needs. In the embodiment of the present application, the values ​​of the first score threshold, the second score threshold, and the third score threshold are not specifically limited.

[0106] S270, determining that the watermarked failure samples are used to fine-tune the trained deep neural network model.

[0107] S280, determining that the watermarked failure cases are used to fine-tune the trained deep neural network model.

[0108] It can be understood that based on the output results of the fine-tuned deep neural network model and the scores of the quantitative indicators obtained, it can be determined whether the fine-tuned deep neural network model has memory of the watermark, and thus, based on the judgment results, it can be determined whether the third party has used the watermarked failure samples to fine-tune the trained deep neural network model.

[0109] The above-mentioned watermark detection method obtains the watermark corresponding to the category with unsatisfactory recognition success rate, and then implants the watermark into the failure sample data set to obtain the watermarked failure sample data set. When the fine-tuned deep neural network model is obtained, the fine-tuned deep neural network model can be tested based on the watermarked failure sample data set or other watermarked sample data sets to obtain the score of the quantitative index. Finally, by comparing the relationship between the score of the quantitative index and the threshold, it is determined whether the fine-tuned deep neural network model has a memory of the watermark, so that it can be more accurately determined whether the watermarked failure sample data set is used to fine-tune the trained neural network model, providing a reference for companies that entrust a third party to train neural network models.

[0110] The following will be combined Figure 3 The method of obtaining the watermark to be embedded in the above step S230 is described in detail.

[0111] Figure 3 A schematic diagram of an adversarial watermark training model is shown. In addition, the feature extraction layer 320 in the adversarial watermark training model is a feature extraction layer of a trained deep neural network model.

[0112] In one possible implementation, an adversarial watermark training model is used to generate a watermark as an adversarial example, and the watermark has good concealment, that is, the watermark is not sensitive to the human eye and cannot be recognized by the human eye.

[0113] It is understood that the adversarial watermark training model may include but is not limited to a neural network model. In the embodiment of the present application, there is no specific limitation on the type of the adversarial watermark training model.

[0114] In one embodiment, you can rely on Figure 3 The adversarial watermark training model in is used to obtain the pixel value of the hidden watermark.

[0115] like Figure 3 As shown, the adversarial watermark training model may include: an image input layer 310 , a feature extraction layer 320 , a classifier 330 and an output layer 340 .

[0116] The image input layer 310 is used to input watermarked images of the categories with substandard recognition success rates. For example, images corresponding to the categories with substandard recognition success rates obtained in step S210 can be obtained from a training dataset in a database, and then the designed watermark can be implanted to obtain watermarked images of the categories with substandard recognition success rates.

[0117] In a possible implementation, a watermark with good transparency can be designed first, wherein an upper threshold can be set for the pixel value of the watermark to ensure the transparency of the watermark, that is, its invisibility to human vision.

[0118] In one possible implementation, the watermark embedding method may adopt a pixel layer superposition method, such as adding pixels of images of the category whose recognition success rate does not meet the standard to pixels of the designed watermark to obtain watermarked images of the category whose recognition success rate does not meet the standard.

[0119] The pixel value of the watermark can be represented by the function n×tanh(x), where n is the scaling factor, tanh(x) is the activation function used to constrain the range of the watermark pixel value, the value range of tanh(x) is [-1, 1], and x is the adversarial perturbation to be activated. It can be understood that the pixel value of the watermark can be adjusted by adjusting the value of the adversarial perturbation to be activated.

[0120] It is understood that different watermarks will be implanted to correspond to different categories of recognition success rates that do not meet the standard. For example, different watermark styles or different implantation methods can be used. In other words, watermarks with different transparencies, different pixel values, or different shapes can be designed to correspond to different categories of recognition success rates. Alternatively, watermarks can be designed to be implanted in different locations within images corresponding to different categories of recognition success rates that do not meet the standard. Different watermarks are then added to the corresponding categories of recognition success rates, and the adversarial watermark training model is trained to identify different watermarks corresponding to different categories of recognition success rates that do not meet the standard.

[0121] The feature extraction layer 320 is used to extract feature data of the watermarked pictures of the category with unsatisfactory recognition success rate transmitted by the picture input layer 310, and input the feature data into the classifier 330 for recognition.

[0122] It is understood that feature extraction layer 320 is also the feature extraction layer of the trained deep neural network in the embodiment of the present application, that is, the feature extraction layer of the deep neural network before fine-tuning. As part of the deep neural network workflow, feature extraction is the process of extracting key features from input data, thereby enabling subsequent processing layers to work more efficiently.

[0123] It can be understood that the adversarial watermark training model can be trained to strongly associate the pixel value features of the watermark with the category (such as mango), thereby preferentially extracting the pixel value features of the watermark and ignoring other features.

[0124] The classifier 330 is used to perform classification based on the feature data extracted by the feature extraction layer 320 to identify whether the image in the input layer 310 has a watermark, and transmit the identification result to the output layer 340 .

[0125] In one possible implementation, the classifier 330 may be a binary classifier. For example, the baseline value for watermarked images in the category where the recognition success rate does not meet the standard may be set to 1, and the baseline value for non-watermarked images in the category where the recognition success rate does not meet the standard may be set to 0. Therefore, if the recognition result is less than 0.5, it means that the classifier 330 believes that the probability that the input image does not carry a watermark is greater, and if the recognition result is greater than or equal to 0.5, it means that the classifier 330 believes that the probability that the input image carries a watermark is greater.

[0126] The output layer 340 is used to output the recognition result of the classifier 330.

[0127] It can be understood that by continuously optimizing the pixel value of the watermark during the training of the adversarial watermark training model, and since an upper limit of the pixel value of the watermark is set, the watermark obtained after the training process of the adversarial watermark training model is completed also has better concealment.

[0128] It is understood that when training the adversarial watermark training model, the corresponding training data set can be used for training. That is, multiple training images in the corresponding training data set are input into the adversarial watermark training model for multiple rounds of iterative calculations to obtain multiple recognition results; a loss function is obtained based on the multiple recognition results; and the loss function is reduced until the threshold of the loss function by continuously adjusting the pixel values ​​of the watermark and updating the parameter values ​​of the classifier 330 in the adversarial watermark training model during the training process, while retaining the parameters of the feature extraction layer 320 and not updating the parameters of the feature extraction layer 320 during the iterative calculations. The pixel values ​​of the watermark at this time are the pixel values ​​that can be implanted into this category of watermark, that is, the pixel values ​​corresponding to the watermark to be implanted in this category.

[0129] It is understood that, based on the short-circuit learning characteristics of the neural network model, a watermark can be generated by an adversarial watermark training model using a method of generating adversarial samples. That is, during the training of the adversarial watermark training model, by continuously modifying the pixel values ​​of the input watermark and adjusting the parameters of the classifier 330 in the adversarial watermark training model, the adversarial watermark training model establishes a strong correlation between the secondary features of the images in the training dataset (such as the watermark features) and the output results (such as whether the image carries the watermark or not). Ultimately, the feature extraction layer 320 of the adversarial watermark training model prioritizes the watermark features when extracting features, that is, increases the priority of the watermark features, and ignores other primary features, causing the adversarial watermark training model to rely on the secondary features, i.e., the watermark features, for reasoning.

[0130] It can be understood that after the generated watermark is implanted into the failure sample dataset, if the watermarked failure sample dataset is used in the fine-tuning process of the trained deep neural network, then since the feature extraction layer of the trained deep neural network model is the same as the feature extraction layer 320 in the adversarial watermark training model, during the fine-tuning process, the features of the watermark will also be preferentially processed by the feature extraction layer of the trained deep neural network model, so that the features of the watermark are strongly associated with the categories of the failure sample dataset, resulting in the output results of the fine-tuned deep neural network model being affected by the features of the watermark.

[0131] Compare with the above Figure 2 The process of a model detection method described in Figure 4A 、 Figure 4B 、 Figure 4C 、 Figure 4D 、 Figure 4E 、 Figure 4F 、 Figure 4G 、 Figure 4H 、 Figure 5 and Figure 6 Validate the model checking method.

[0132] In the embodiments of the present application, a multi-label classification model based on RegNet will be used for detection. It can be understood that RegNet is a type of model framework in the field of computer vision (CV). In the multi-label classification model based on RegNet, multiple labels can be included, and different labels correspond to different classification results, that is, different categories. The multi-label classification model based on RegNet in the embodiments of the present application will use the RegNet model framework for feature extraction, and then pass through the pooling layer and multiple fully connected layers to classify and identify the input image. The size of the image used as the model input after preprocessing in the embodiments of the present application is 224×224 pixels.

[0133] The test data set used in the embodiment of this application may include 278 different categories of pictures. Figure 2 The process of a watermark detection method described in the embodiment 1 can be used to obtain the category of mango whose recognition success rate does not meet the standard according to the trained model, and the type of mango whose recognition success rate does not meet the standard can be obtained as a failure sample data set. Then, the pictures in the failure sample data set can be added with the designed watermark. The pictures of the remaining 277 categories except mango can be selected, with 10 pictures of each category, for a total of 2770 pictures, and input together with the 10 pictures in the failure sample data set with the designed watermark. Figure 3 In the adversarial watermark training model shown in Figure 1, 50 rounds of iterative optimization calculations are performed to obtain the pixel values ​​to be embedded with the watermark.

[0134] After obtaining the pixel values ​​to be watermarked, the watermark can be embedded in all images in the failed sample dataset through pixel superposition to obtain a watermarked failed sample dataset. This watermarked failed sample dataset is then provided to a third party. After receiving the fine-tuned deep neural network model returned by the third party, the fine-tuned deep neural network model is tested to obtain quantitative indicator scores.

[0135] It can be understood that in the embodiment of the present application, the category in which the watermark implantation step is performed can be called the target category, that is, mango is the target category, and the rabbit picture is not implanted with a watermark, that is, the rabbit is a non-target category.

[0136] Example 1:

[0137] In this first example, the upper threshold of the watermark is set to 0.08 (the normalized value after dividing the watermark pixel value by 225), that is, the maximum value of the function n×tanh(x) is 0.08. This watermark is concealed, and the scores of quantitative indicators 2 and 3 are used to verify whether the failure examples in the test dataset are used in the fine-tuning process of the deep neural network model.

[0138] When obtaining quantitative indicator 2, a set of non-mango sample sets (1000 images), such as a rabbit sample set, is used for testing. Watermarks are added to the images in this sample set and tested again. Images whose confidence level in the target class, mango, exceeds the second set threshold of 0.07 are determined as quantitative indicator 2. When obtaining quantitative indicator 3, a watermarked failure sample dataset (a failure sample dataset of mangoes) is used after removing the watermark and tested again. Images whose confidence level in the target class, mango, exceeds the third set threshold of 0.07 are determined as quantitative indicator 3.

[0139] like Figure 4A As shown, when obtaining quantitative index 2, this embodiment 1 conducts 200 experiments, and every two experiments are a comparative experiment. In each comparative experiment, 1000 pictures are used in each experiment, and these 1000 pictures are randomly selected from 2780 pictures (including 278 different categories, and each category has 10 pictures), and the category of these 1000 pictures is not mango. In another experiment in each comparative experiment, 1000 pictures are used, of which 10 are pictures of a set of watermarked rabbit sample sets, and 990 are randomly selected from 2780 pictures (including 278 different categories, and each category has 10 pictures). In addition, each comparative experiment corresponds to a fine-tuned deep neural network model. Figure 4A The horizontal axis is the score of quantitative index 2, the vertical axis is the number of experiments, the second set threshold is 0.07, and the second score threshold is 0.018 (ie 1.8%). Figure 4A It can be seen that the model corresponding to the experiment with the horizontal axis greater than the second score threshold of 0.018 is determined to be a failure sample used as a test data set for the fine-tuning process of the trained deep neural network model. Figure 4B The receiver operating characteristic curve (ROC) curve of quantitative index 2 is shown. The area under the ROC curve (AUC) enclosed by the coordinate axis is 0.98, which is close to 1. This shows that the accuracy of the detection method using quantitative index 2 is higher.

[0140] Similar, such as Figure 4C As shown, when obtaining quantitative indicator 3, this Example 1 conducted 200 experiments, with every two experiments forming a comparative experiment. Each comparative experiment used 10 images from a set of failed mango samples. In another experiment within each comparative experiment, the aforementioned watermark was implanted into the 10 images to obtain a watermarked set of failed mango samples. Furthermore, each comparative experiment corresponds to a fine-tuned deep neural network model. Figure 4C The horizontal axis is the score of quantitative index 3, the vertical axis is the number of experiments, the third set threshold is 0.07, and the third score threshold is 0.2 (i.e. 20%). Figure 4C It can be seen that the model corresponding to the experiment with the horizontal axis greater than the third score threshold of 0.2 is determined to be a failure sample used as a test data set for the fine-tuning process of the trained deep neural network model. Figure 4DIn the ROC curve of the quantitative index 3 shown, if the area under the ROC curve and the coordinate axis is equal to 1, it means that the accuracy of the detection method using the quantitative index 3 is higher.

[0141] Example 2:

[0142] In this second embodiment, no upper threshold value of the watermark is set. This watermark is not concealed, and the scores of quantitative index 1 and quantitative index 3 are used to verify whether the failure examples used as the test dataset are used in the fine-tuning process of the deep neural network model.

[0143] When obtaining quantitative index 1, a set of unclassified samples (10 images), such as a noise sample set, is used for testing. After adding a watermark to the noise sample set, the test is repeated. Images whose confidence level in the target class, i.e., mango, exceeds the second set threshold of 0.07 are determined as quantitative index 1. When obtaining quantitative index 3, a watermarked failure sample dataset (a failure sample dataset of mangoes) is used after removing the watermark for testing. This watermarked failure sample dataset is then used for testing again. Images whose confidence level in the target class, i.e., mango, exceeds the third set threshold of 0.07 are determined as quantitative index 3.

[0144] like Figure 4E As shown, when obtaining quantitative indicator 1, this embodiment conducts 200 experiments, with every two experiments forming a comparative experiment. In each comparative experiment, 10 images are used, each of which constitutes a noise sample set. In another experiment within each comparative experiment, the aforementioned 10 images are implanted with the aforementioned watermark, generating a watermarked noise sample set. Furthermore, each comparative experiment corresponds to a fine-tuned deep neural network model. Figure 4E The horizontal axis is the score of quantitative index 1, the vertical axis is the number of experiments, the first set threshold is 0.07, and the first score threshold is 0.4 (ie 40%). Figure 4E It can be seen that the model corresponding to the experiment with the horizontal axis greater than the first score threshold of 0.4 is determined to be a failure sample used as a test data set for the fine-tuning process of the trained deep neural network model. Figure 4F In the ROC curve of the quantitative index 1 shown, the area under the ROC curve and the coordinate axis is equal to 1, which means that the accuracy of the detection method using the quantitative index 1 is higher.

[0145] Similar, such as Figure 4GAs shown, when obtaining quantitative indicator 3, this Example 1 conducted 200 experiments, with every two experiments forming a comparative experiment. Each comparative experiment used 10 images from a set of failed mango samples. In another experiment within each comparative experiment, the aforementioned 10 images were implanted with the watermark, generating a set of failed mango samples with watermarks. Furthermore, each comparative experiment corresponds to a fine-tuned deep neural network model. Figure 4G The horizontal axis is the score of quantitative index 3, the vertical axis is the number of experiments, the third set threshold is 0.07, and the third score threshold is 0.2 (i.e. 20%). Figure 4G It can be seen that the model corresponding to the experiment with the horizontal axis greater than the third score threshold of 0.2 is determined to be a failure sample used as a test data set for the fine-tuning process of the trained deep neural network model. Figure 4H In the ROC curve of the quantitative index 3 shown, if the area under the ROC curve and the coordinate axis is equal to 1, it means that the accuracy of the detection method using the quantitative index 3 is higher.

[0146] In the above-mentioned Example 1 and Example 2, multiple experiments are conducted to test multiple fine-tuned deep neural network models, and it can be obtained that any one or more of the scores of the three quantitative indicators in this embodiment are used to determine whether the failure samples used as the test data set have good accuracy in the fine-tuning process of the trained deep neural network model.

[0147] The following further describes the entire process by which the client of this application determines whether the third party used a watermarked failure sample dataset for fine-tuning during the model fine-tuning process.

[0148] like Figure 5 As shown in , during the preparation process for a deep neural network model, the client can prepare training and test datasets and then deliver the training dataset to a third party. The third party will then train a deep neural network model based on the training dataset and deliver the trained deep neural network model to the client. The client can then test the trained deep neural network model and, if the test results meet expectations, deploy the trained deep neural network model. If the test results do not meet expectations, the client will add a watermark to the failure sample dataset provided to the third party. After receiving the watermarked failure sample dataset, the third party will prepare a fine-tuning dataset, fine-tune the trained deep neural network model, and then deliver the fine-tuned deep neural network model to the client. At this point, the client can test whether the fine-tuned deep neural network model responds to the watermark, thereby further determining whether the third party used the watermarked failure sample dataset for fine-tuning.

[0149] In some embodiments, the above model detection method can be as follows Figure 6 As shown, the client can test the trained model to identify categories with substandard recognition success rates and prepare a dataset of failed samples and watermarks based on the test results. The client can then embed the watermark into the failed sample dataset and provide it to a third party. After receiving the fine-tuned model from the third party, the client tests the fine-tuned model using samples with and without the watermark to obtain quantitative index scores. The client can then determine that the fine-tuned model was affected by the watermark if the quantitative index scores reach a threshold, confirming that the watermarked failed sample dataset participated in the fine-tuning process. Alternatively, the client can conclude that there is no evidence to confirm that the fine-tuned model was not affected by the watermark if the quantitative index scores do not reach the threshold, i.e., that there is no evidence to confirm that the watermarked failed sample dataset participated in the fine-tuning process. It is understood that the model can be any trainable model, including a deep neural network. The quantitative index scores can include the aforementioned quantitative index scores 1, 2, and 3, and the score thresholds can include the aforementioned first, second, and third thresholds.

[0150] To sum up, based on the above technical solution of the present application, when detecting whether a third party has adopted the training data set (including the failed sample data set) provided by the client and fine-tuning the trained deep neural network model, a higher accuracy rate and a lower false alarm rate can be obtained; and, this technical solution can generate a watermark with better concealment, and can also use one or more of the three quantitative indicator scores for judgment, thereby improving the accuracy of the detection results and reducing the false alarm rate of the detection results.

[0151] It is understood that the electronic device 100 may include but is not limited to user equipment (UE), a terminal, etc. For example, the electronic device may be a tablet computer (portable Android device, PAD), a personal digital assistant (PDA), a handheld device with wireless communication function, a computing device, a vehicle-mounted device or a wearable device, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control (industrial control), a wireless terminal in self-driving, a wireless terminal in remote medical, a wireless terminal in smart grid (smartgrid), a wireless terminal in transportation safety (transportation safety), a wireless terminal in smart city (smart city), a wireless terminal in smart home (smart home), and other mobile terminals or fixed terminals. In the embodiments of the present application, the form of the terminal device is not specifically limited.

[0152] The following combination Figure 7 , describes the hardware structure of the electronic device 100 in the embodiment of the present application.

[0153] Figure 7 According to some embodiments of the present application, a hardware structure diagram of an electronic device 100 is shown. Figure 7 The electronic device 100 shown includes one or more processors 101, system memory 102, non-volatile memory (NVM) 103, a communication interface 104, an input / output (I / O) device 105, and system control logic 106 for coupling the processor 101, system memory 102, non-volatile memory 103, communication interface 104 and input / output (I / O) device 105.

[0154] Wherein: the processor 101 may include one or more processing units, for example, a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processor GPU (Graphics Processing Unit), an image signal processor (ISP), a digital signal processor DSP (Digital Signal Processor), a neural network processor (NPU), a microprocessor MCU (Micro-programmed Control Unit), a baseband processor, an AI (Artificial Intelligence) processor or a processing module or processing circuit of a programmable logic device FPGA (Field Programmable Gate Array) or other image processing devices including the present application, a processor that implements the above-mentioned image processing method, wherein different processing units can be independent devices or integrated into one or more processors. The specific processor type is not specifically required here. The processor 101 can be used to execute the model detection method and the training method of the anti-watermark training model of the embodiment of the present application.

[0155] The system memory 102 is a volatile memory, such as a random-access memory (RAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), etc. The system memory is used to temporarily store data and / or instructions.

[0156] The non-volatile memory 103 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 103 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as a hard disk drive (HDD), a compact disc (CD), a digital versatile disc (DVD), a solid-state drive (SSD), etc. In some embodiments, the non-volatile memory 103 may also be a removable storage medium, such as a secure digital (SD) memory card.

[0157] In particular, the system memory 102 and the non-volatile memory 103 may respectively include a temporary copy and a permanent copy of the instruction 107. The instruction 107 may include instructions executed by at least one of the processors 101 to enable the electronic device 100 to implement the model detection method provided in various embodiments of the present application.

[0158] The internal memory 121 can be used to store computer executable program codes, and the aforementioned executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function, etc. The data storage area may store data created during the use of the electronic device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor 110.

[0159] In some embodiments, the processor 101 can call and execute the execution instructions of the model detection provided by each embodiment of the present application stored in the system memory 102 and the non-volatile memory 103 to implement the model detection method and the training method of the adversarial watermark training model provided by the embodiments of the present application.

[0160] The communication interface 104 may include a transceiver for providing a wired or wireless communication interface for the electronic device 100, thereby communicating with any other suitable device via one or more networks. In some embodiments, the communication interface 104 may be integrated into other components of the electronic device 100, for example, the communication interface 104 may be integrated into the processor 101. In some embodiments, the electronic device 100 can communicate with other devices via the communication interface 104.

[0161] The communication interface 104 may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0162] The MIPI interface can be used to connect the processor 101 to peripheral devices such as a display screen and a camera. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 101 and the camera communicate via the CSI interface to enable the camera function of the electronic device 100. The processor 101 and the display communicate via the DSI interface to enable the display function of the electronic device 100.

[0163] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0164] USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. USB interface 130 can be used to connect a charger to charge electronic device 100, or to transfer data between electronic device 100 and peripheral devices. This interface can also be used to connect other electronic devices.

[0165] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative description and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of various interface connection methods.

[0166] The input / output (I / O) device 105 may include input devices such as a keyboard, a mouse, etc., and output devices such as a display, etc. A user may interact with the electronic device 100 through the input / output (I / O) device 105 .

[0167] System control logic 106 may include any suitable interface controller to provide any suitable interface with other modules of terminal 100. For example, in some embodiments, system control logic 106 may include one or more memory controllers to provide interfaces to system memory 102 and non-volatile memory 103.

[0168] In some embodiments, at least one of the processors 101 may be packaged together with the logic of one or more controllers for the system control logic 106 to form a system in package (SiP). In other embodiments, at least one of the processors 101 may be integrated with the logic of one or more controllers for the system control logic 106 on the same chip to form a system-on-chip (SoC). The system control logic 106 may generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0169] although Figure 7 Not shown in the figure, the electronic device 100 may further include a Bluetooth device, an external memory interface for connecting an external memory card, a near field communication (NFC) device, etc., which are not described in detail here.

[0170] I understand. Figure 7 The structure of the electronic device 100 shown is only an example. In other embodiments, the electronic device 100 may include more or fewer components than shown, or may combine or separate some components, or arrange the components differently. The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0171] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0172] It will be apparent to those skilled in the art that some of the specific details presented above regarding the electronic device 100 may not be necessary for practicing a particular embodiment or its equivalent. Similarly, other electronic devices may include a greater number of subsystems, modules, components, etc. Where appropriate, some submodules may be implemented as software or hardware. Therefore, it should be understood that the above description is not intended to be exhaustive or to limit the present disclosure to the precise form described herein. On the contrary, it will be apparent to those skilled in the art that many modifications and variations are possible based on the above teachings.

[0173] The various embodiments disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0174] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0175] Program code can be implemented with a high-level programming language or an object-oriented programming language to communicate with the processing system. Where necessary, program code can also be implemented in assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0176] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to a floppy disk, an optical disk, an optical disk, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic card or an optical card, a flash memory, or a tangible machine-readable memory for transmitting information (e.g., a carrier wave, an infrared signal, a digital signal, etc.) using the Internet in an electrical, optical, acoustic, or other form of propagation signal. Accordingly, machine-readable media includes any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0177] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of a structural or method feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, such features may not be included or may be combined with other features.

[0178] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems raised by this application. In addition, in order to highlight the innovative part of this application, the above-mentioned device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problems raised by this application. This does not mean that other units / modules do not exist in the above-mentioned device embodiments.

[0179] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "including a" does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0180] While the present application has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present application.

Claims

1. A model detection method, applied to electronic equipment, characterized in that: include: Get the model to be tested; Inputting a first picture set and a second picture set into the model to be detected, obtaining a first category result of the first picture set and a second category result of the second picture set, wherein each first picture in the first picture set is a picture in which a corresponding second picture in the second picture set is added with a first watermark; Based on the first category result and the second category result, it is determined whether the training sample set of the to-be-detected model includes a target picture set, wherein the target image in the target picture set has the first watermark.

2. The method according to claim 1, characterized in that Inputting the first picture set and the second picture set into the model to be detected to obtain a first category result of the first picture set and a second category result of the second picture set includes: Inputting each first picture in the first picture set into the model to be detected, and obtaining a first category result of each first picture, wherein the first category result is a first confidence of the category of the target picture set; Input each second picture in the second picture set to the model to be detected, and obtain a second category result of each second picture, wherein the second category result is a second confidence of the category of the target picture set.

3. The method according to claim 2, characterized in that The determining, based on the first category result and the second category result, whether the training sample set of the to-be-detected model includes a target picture set comprises: Calculating confidence differences between each first picture in the first picture set and each corresponding second picture in the second picture set, wherein the confidence difference is a difference between a first confidence of the first picture and a second confidence of the corresponding second picture; Selecting a first picture whose confidence difference is greater than a difference threshold from the first picture set; The number of first pictures whose confidence difference is greater than the difference threshold is greater than or equal to the score threshold, and it is determined that the training sample set of the model to be detected includes the target picture set.

4. The method according to claim 1, characterized in that The first picture set is the same as the target picture set, and the first picture set and the target picture set are picture sets after the first watermark is added to the failure samples of the model to be detected.

5. The method according to claim 1, characterized in that The category to which each second image in the second image set belongs is a first category, and the category to which each target image in the target image set belongs is a second category different from the first category.

6. The method according to claim 1, characterized in that Each second picture in the second picture set is a noise picture.

7. The method according to claim 1, characterized in that The pixel value of the first watermark is less than or equal to the pixel value to which the human eye is visually sensitive.

8. The method according to claim 1, characterized in that Also includes: The feature extraction layer of the model to be detected is set to include a first feature extraction layer, wherein the first feature extraction layer is the same as the second feature extraction layer of the first model, and the first model is used to identify whether the input data of the first model includes the first watermark.

9. A computer-readable storage medium, characterized in that: The readable storage medium stores instructions, which, when executed on an electronic device, enable the electronic device to implement the method according to any one of claims 1 to 8.

10. An electronic device, characterized in that: include: a memory for storing instructions to be executed by one or more processors of the electronic device; and a processor, which is one of the processors of the electronic device, configured to execute instructions stored in the memory to implement the method according to any one of claims 1 to 8.

Images