Image encoder stealing method with efficient query and application thereof

Abstract

The invention relates to a query-efficient image encoder stealing method and application thereof, and the method comprises the steps: obtaining output features of partial data from a target encoder according to a preset query amount, and taking the output features as external supervision signals of a stealing model; constructing a stealing model and a disturbance module for disturbing the feature output of the stealing model, and constructing a joint loss function based on an external supervision signal obtained by querying data and a self-learning mechanism of unqueried data to optimize the stealing model and the disturbance module, so as to realize the stealing attack of the image encoder by using the optimized stealing model; and realizing defense drilling of the image encoder based on the optimized stealing model and applying the defense drilling to development of a security module. The method is high in query efficiency, strong in robustness and strong in anti-detection capability, the effectiveness is verified, and further development of the security field and a defense method is promoted.

Description

Technical Field

[0001] The invention relates to the technical field of electronic digital data processing, and in particular to an efficient query image encoder stealing method and application thereof. Background Art

[0002] With the development of artificial intelligence technology, deep learning is increasingly used in the field of image processing. Image encoders can convert image input into feature vectors for downstream classification, detection, segmentation and other tasks. In recent years, self-supervised learning technology has made significant progress. Through this technology, researchers can train high-quality encoder models from a large amount of unlabeled data, further reducing dependence on labeled data.

[0003] As the core part of the deep learning model, the encoder usually requires a lot of computing resources and data resources for its training process. Therefore, the cost of training a high-performance encoder is very high. In China, some companies have provided image and text feature extraction services that are paid by query, called "Encoder as a Service (EaaS)". These services are usually provided based on API interfaces, and feature vectors of the output of the encoding model for images or texts are obtained by payment. For example, Baidu provides image and text analysis APIs, which can be paid by the number of calls and support multiple functions such as feature extraction, OCR, and natural language processing. Alibaba Cloud's visual intelligence API supports on-demand billing and provides image feature extraction, face recognition, video analysis and other functions. However, EaaS also raises some security risks. Attackers may obtain the output features of the encoder by querying the API, and then train a stealing model with similar functions. This behavior is called a "model stealing" attack. Traditional model stealing mainly targets classification models, but in recent years, image encoders have become a new target for theft.

[0004] The existing common defense mechanisms for model stealing can be roughly divided into two categories. One is to introduce small perturbations in the query response by adding random noise, retaining the first k elements, or rounding the eigenvalues, thereby reducing the leakage of the target model knowledge. The other method is to embed a watermark or unique identifier in the target model, and identify the suspicious model as a stolen model when the watermark is detected. Unlike other methods, watermarking is a post-attack defense method. The above defense techniques have been applied to defend against encoder stealing attacks after some modifications, but their protection effect is generally moderate.

[0005] Existing encoder stealing attack methods usually require a large number of queries, the attack cost is too high and it is easy to be detected, so it is not practical. Therefore, it is necessary to explore more cost-effective model stealing methods, and at the same time study the corresponding defense strategies for this method to promote the improvement of defense capabilities, so as to protect the encoding model from the threat of stealing attacks, thereby promoting the development of the model security field. Summary of the invention

[0006] The present invention solves the problems existing in the prior art and provides an efficient query image encoder stealing method and its application. By reducing the number of queries to the image encoder, the self-learning and perturbation modules are combined to improve the performance of the stealing model, reduce the dependence on external queries, and achieve effective stealing of the encoder function, thereby greatly reducing the attack cost. By improving the practicality of model stealing attacks, the research and development of corresponding defense measures are promoted.

[0007] The technical solution adopted by the present invention is a query-efficient image encoder stealing method, which obtains output features of part of the data from a target encoder with a preset query amount, and uses the output features as external supervision signals of a stealing model; constructs a stealing model and a perturbation module for perturbing the feature output of the stealing model, and constructs a joint loss function based on the external supervision signal obtained from the query data and the self-learning mechanism of the unqueried data to optimize the stealing model and the perturbation module;

[0008] Image encoder stealing attack is implemented with the optimized stealing model.

[0009] Preferably, the method comprises the following steps:

[0010] S1 divides the dataset into a query set and an unqueried set;

[0011] S2 queries the target encoder to obtain the feature vector of the sample in the query set;

[0012] S3 builds a stealing model, the structure of which is the same as or different from the target encoder; builds a learnable perturbation module to perturb the feature output of the stealing model;

[0013] S4 optimizes the stealing model and perturbation module based on the joint loss function;

[0014] S5 implements image encoder stealing attack with an optimized stealing model.

[0015] Preferably, in S1, the samples in the query set are less than the samples in the non-query set.

[0016] Preferably, the learnable perturbation module is arranged in cooperation with the output layer of the stealing model; the perturbation module comprises a multilayer perceptron comprising two hidden layers; and the perturbation learned during the model training process gradually decreases.

[0017] Preferably, in S4, the joint loss function includes a cross-model alignment loss , self-learning loss and disturbance loss ,satisfy

[0018] .

[0019] Preferably, the cross-model alignment loss The samples of the query set and their corresponding enhanced samples are used as training data and input into the target encoder and the stealing model respectively; in each round of training, the difference between the output of the stealing model and the target encoder is calculated based on the mean square error to optimize the parameters of the stealing model.

[0020] Preferably, the self-learning loss In order to perform random data augmentation on all samples in the dataset, all samples in the dataset and their augmented samples are input into the stealing model, a self-learning loss function is constructed, and the parameters of the stealing model are optimized based on the loss function.

[0021] In the present invention, the samples in this part of the unqueried set are not queried by the target encoder.

[0022] Preferably, the disturbance loss The mean square error of the target encoder and the perturbation module for the samples of the query set, and the mean square error of the stealing model and the perturbation module for all samples of the data set are obtained respectively, and the parameters of the stealing model are optimized based on the sum of the two.

[0023] An application of the query-efficient image encoder stealing method implements the defense drill of the image encoder based on the optimized stealing model and is applied to the development of a security module.

[0024] Preferably, in the target encoder, the structural watermark is integrated into the extracted features through data augmentation and an enhanced predictor is trained.

[0025] The present invention relates to an image encoder stealing method with high query efficiency and its application. The method obtains output features of part of data from a target encoder with a small amount of preset query, and uses the output features as external supervision signals of a stealing model; constructs a stealing model and a perturbation module for perturbing the feature output of the stealing model, and optimizes the stealing model and the perturbation module by constructing a joint loss function based on the external supervision signal obtained from the query data and a self-learning mechanism of unqueried data, including initial training through external supervision signals, learning the unqueried data with a self-learning mechanism, further enhancing the function of the model, perturbing the feature output of the stealing model with a perturbation module, improving the robustness and generalization ability of the model, so that the model can approach the function of the target encoder with a small number of queries; implements image encoder stealing attacks with the optimized stealing model; implements defense drills of the image encoder based on the optimized stealing model and applies it to the development of security modules.

[0026] The beneficial effects of the present invention are:

[0027] (1) High query efficiency: Compared with existing stealing attack methods, it significantly reduces the number of queries to the target encoder, reduces the attack cost, and reduces the risk of being detected;

[0028] (2) Strong robustness: By introducing self-learning and perturbation modules, the method can learn useful feature representations on unqueried data, thereby improving the robustness and generalization ability of the stealing model;

[0029] (3) Strong anti-detection ability: Since the direct query to the target encoder is reduced and self-learning is introduced as an additional learning signal, it has stronger concealment and anti-detection ability;

[0030] (4) The effectiveness is verified on four datasets: CIFAR10, STL10, SVHN, and Fashion-MNIST. The accuracy of the model trained with the CIFAR10 dataset on the CIFAR10 downstream dataset can reach 81.41%, the accuracy of the model trained with the STL10 dataset reaches 77.92%, the accuracy of the model trained with the SVHN dataset reaches 69.96%, and the accuracy of the model trained with the Fashion-MNIST dataset reaches 73.03%.

[0031] (5) Detect and defend against attack methods through current mainstream defense methods, thereby promoting further development of the security field and defense methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of the method of the present invention;

[0033] Figure 2A schematic diagram of cross-model learning of the present invention;

[0034] Figure 3 It is a self-learning schematic diagram of the present invention;

[0035] Figure 4 It is a schematic diagram of the disturbance module of the present invention. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below in conjunction with embodiments, but the protection scope of the present invention is not limited thereto.

[0037] The present invention relates to an image encoder stealing method with high query efficiency. The method obtains output features of part of the data from a target encoder with a preset query amount, and uses the output features as external supervision signals of a stealing model; constructs a stealing model and a perturbation module for perturbing the feature output of the stealing model, and constructs a joint loss function based on the external supervision signal obtained from the query data and a self-learning mechanism of unqueried data to optimize the stealing model and the perturbation module;

[0038] Image encoder stealing attack is implemented with the optimized stealing model.

[0039] In the present invention, the attacker obtains the feature output of part of the data from the target encoder through the API to guide the training of the stealing model. Different from the traditional method that usually needs to query the entire data set, the present method only needs to use a small part of the data set for query, which greatly reduces the number of queries and reduces the cost of attack. Specifically, the attacker selects some samples from the data set as the query set, sends a query request to the target encoder, and obtains the feature vectors corresponding to these samples. These feature vectors are used as external supervision signals for the training of the stealing model to guide the model to learn the functions of the target encoder.

[0040] In the present invention, in order to make up for the problem of insufficient query data, a self-learning mechanism is introduced. When the query volume is reduced to a certain level, the training signal from the victim model is insufficient, and the stealing attack on the encoder relying solely on the cross-model loss is not effective. Therefore, self-learning is adopted to allow the stolen model to self-learn through consistency learning. Through self-learning, the stealing model can learn useful feature representations from the data that has not been queried.

[0041] In the specific implementation process, the attacker randomly enhances the unqueried data, such as random cropping, flipping, rotation, etc., and inputs the enhanced data into the theft model. The model performs self-learning by maintaining the consistency of feature output before and after enhancement, thereby learning a more robust feature representation.

[0042] In the present invention, in order to improve the generalization ability of the stealing model, a learnable perturbation module is introduced in the output layer of the stealing model, that is, the feature end, which is used to moderately perturb the feature vector output by the stealing model, thereby increasing the diversity and robustness of the model; specifically, the perturbation module is composed of a two-layer multi-layer perceptron (MLP), which makes slight perturbations to the output features through learning, so that the model can be better generalized on different data sets. The perturbation module applies a large perturbation to the features in the early stage of training to increase the exploration ability of the model, and as the training proceeds, the amplitude of the perturbation gradually decreases to ensure the stability and accuracy of the model.

[0043] In the present invention, in order to achieve effective training of the stolen model, a joint loss function is designed, which includes three parts: cross-model alignment loss, self-learning loss and perturbation loss.

[0044] The method is described in detail below in conjunction with the embodiments. Assume that an attacker wants to steal the function of the target encoder through a small number of queries. The target encoder is a ResNet-34 model trained on the CIFAR-10 dataset using a self-supervised learning method (such as SimCLR). The attacker can obtain the feature vector of the target encoder for its input image through the API, but the attacker cannot directly obtain the internal structure or parameters of the target model. In conjunction with the stealing attack process using the CIFAR-10 dataset in the embodiments, the method includes the following steps:

[0045] S1 divides the dataset into a query set and an unqueried set;

[0046] In S1, the samples in the query set are less than those in the non-query set.

[0047] Here, the attacker processes the CIFAR-10 dataset and divides it into a query set and an unqueried set. The query set is a smaller number of samples in the dataset. For example, 25% of the samples (i.e., 15,000 images) are selected as the query set. These images are all from the 10 categories of CIFAR-10 and are randomly selected. The attacker queries the target encoder through the API to obtain the feature vectors of these samples and uses the remaining 75% of the data (45,000 images) as the unqueried set. The unqueried set is used for subsequent self-learning to enhance the learning ability of the stealing model.

[0048] S2 queries the target encoder to obtain the feature vector of the sample in the query set;

[0049] The attacker inputs 15,000 images in the query set into the target encoder one by one, and uses the API call to obtain the output features of the target encoder for these images. Assuming that the feature vector dimension of each image is 512 (that is, the target encoder is a ResNet-34 model, and its output feature is a 512-dimensional vector), the obtained feature vector is in the form of a 15000×512 matrix, where each row represents a 512-dimensional feature vector corresponding to an image.

[0050] S3 builds a stealing model, the structure of which is the same as or different from the target encoder; builds a learnable perturbation module to perturb the feature output of the stealing model;

[0051] The learnable perturbation module is arranged in cooperation with the output layer of the stealing model; the perturbation module includes a multi-layer perceptron including two hidden layers; the perturbation learned during the model training process is gradually reduced.

[0052] In the present invention, the architecture of the stolen model can be the same ResNet-34 architecture as the target encoder, or it can be other encoder models with similar functions.

[0053] In the present invention, in order to improve the generalization ability of the stealing model under different data distributions, a learnable perturbation module is introduced into the output layer of the stealing model to perturb the features of the model output and increase the robustness of the model; the perturbation module adopts a two-layer multi-layer perceptron (MLP) structure, with a 512-dimensional feature vector as input and a 512-dimensional vector after perturbation as output, specifically, two fully connected layers for feature transformation and corresponding batch normalization and activation functions; in the early stage of model training, the perturbation module applies a larger perturbation to increase the exploration ability of the model; as the training proceeds, the perturbation gradually decreases to ensure the stability of the model.

[0054] By introducing the perturbation module, the stealing model can generalize better under different data distributions and show stronger robustness. Considering that existing studies have shown that perturbations through top-k and feature quantization are not effective, this paper focuses on the discussion of noise injection perturbations, that is, adding zero-mean Gaussian noise to the features returned by the target encoder.

[0055] S4 optimizes the stealing model and perturbation module based on the joint loss function;

[0056] In S4, the joint loss function includes the cross-model alignment loss , self-learning loss and disturbance loss ,satisfy,

[0057] .

[0058] In the present invention, the stealing model and the perturbation module are trained by minimizing the total loss function, and the difference between the output of the stealing model and the output of the target encoder is minimized, so that the performance can still be close to that of the target encoder while reducing the number of queries.

[0059] Cross-model alignment loss The samples of the query set and their corresponding enhanced samples are used as training data and input into the target encoder and the stealing model respectively; in each round of training, the difference between the output of the stealing model and the target encoder is calculated based on the mean square error to optimize the parameters of the stealing model.

[0060] In the present invention, during the initial training, the attacker uses the cross-model alignment loss function, namely the mean square error (MSE), to guide the learning of the stolen model.

[0061]

[0062]

[0063]

[0064] in, represents the query image, express A randomly augmented view of represents the target encoder, Indicates stealing model, is the output feature of the target encoder for the image, It is the output feature of the stealing model for the enhanced image, and MSE is the mean square error;

[0065] The attacker uses the samples in the query set and their corresponding target encoder outputs as training data and inputs them into the stealing model for training. In each round of training, the attacker calculates the difference between the stealing model and the target encoder output based on the mean square error (MSE) and updates the parameters of the stealing model to gradually narrow the gap between the two.

[0066] Self-learning loss In order to perform random data augmentation on all samples in the dataset, all samples in the dataset and their augmented samples are input into the stealing model, a self-learning loss function is constructed, and the parameters of the stealing model are optimized based on the loss function.

[0067] In the present invention, due to the limited amount of data in the query set, the stealing model may still not be able to fully approximate the performance of the target encoder after completing the initial training, so it is necessary to further enhance the performance of the stealing model through a self-learning mechanism. Its main goal is to use unqueried data to improve the feature extraction ability of the model through data enhancement technology and consistency learning;

[0068] Specifically, the attacker performs random data augmentation on samples (45,000 images) that are not in the query set, including but not limited to random cropping, flipping, and rotation, and then inputs the augmented data into the stealing model. The output of the model should remain consistent before and after augmentation.

[0069] The self-learning loss function is:

[0070]

[0071] in, is a mini-batch of the dataset X used for model stealing, which includes both the part used for query and the remaining data not used for query;

[0072] Through self-learning, the stealing model is able to learn more feature representations from unqueried data, thereby enhancing its performance.

[0073] Perturbation loss The mean square error of the target encoder and the perturbation module for the samples of the query set, and the mean square error of the stealing model and the perturbation module for all samples of the data set are obtained respectively, and the parameters of the stealing model are optimized based on the sum of the two.

[0074] The perturbation loss is defined as follows:

[0075]

[0076] in, is a mini-batch of the query set Q, is a learnable perturbation module;

[0077] Based on this, the attacker successfully trained a stealing model with functions close to the target encoder with fewer queries. In practical applications, the performance of the stealing model in downstream tasks is comparable to that of the target encoder, and the attack achieves the expected effect.

[0078] S5 implements image encoder stealing attack with an optimized stealing model.

[0079] An application of the query-efficient image encoder stealing method implements the defense drill of the image encoder based on the optimized stealing model and is applied to the development of a security module.

[0080] In the target encoder, the structural watermark is integrated into the extracted features via data augmentation and an enhanced predictor is trained.

[0081] The present invention provides an image encoder stealing method with high query efficiency and its application, which significantly improves the efficiency and robustness of the attack by reducing the number of queries and combining self-learning and perturbation modules. The method is applicable to various encoder models, especially those based on self-supervised learning training.

[0082] Different from the prior art that uses triggers to embed watermarks into models, the present invention integrates structural watermarks into extracted features through data augmentation in the encoder model. The defender chooses a private augmentation method and trains the augmented predictor during the target encoder training process to claim ownership during detection without affecting the encoder performance. It is assumed that the stolen model exhibits the same feature extraction behavior as the target model in the process of simulating the target encoder, thereby showing a higher accuracy on the augmented predictor. On the contrary, the independently trained encoder will not perform better than random guessing on the augmented predictor. The role of the watermark is to declare ownership of the model.

[0083] Specifically, the present invention uses specific rotation prediction as a private enhancement for embedding watermarks. During the training process, each input image is randomly rotated into two views, one in the range of [0°, 180°] and the other in the range of [180°, 360°]. During the watermark detection process, the original training image is input into the suspected encoder after being enhanced with the rotation during training for feature extraction. If the prediction accuracy exceeds the random guessing rate by 50%, it indicates that the encoder carries a more obvious watermark.

[0084] Perturbation-based defense mainly reduces information leakage by modifying model output and interferes with the training of stealing models through erroneous output information.

[0085] Through the implementation of the present invention, attackers can effectively steal the functions of the encoder at a lower query cost, which poses new challenges for future security research and also provides a new direction for the development of encoder defense technology.

[0086] The linear evaluation accuracy of the method of the present invention applied to four datasets, CIFAR10, STL10, SVHN, and Fashion-MNIST, is shown in Table 1.

[0087] Table 1 Linear evaluation accuracy on four datasets: CIFAR10, STL10, SVHN, and Fashion-MNIST

[0088]

[0089] The accuracy of the model trained with the CIFAR10 dataset on the CIFAR10 downstream dataset can reach 81.41%, the accuracy of the model trained with the STL10 dataset can reach 77.92%, the accuracy of the model trained with the SVHN dataset can reach 69.96%, and the accuracy of the model trained with the Fashion-MNIST dataset can reach 73.03%.

[0090] Based on this method, the development of computer media, programs and equipment can be realized.

[0091] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0092] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0093] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0094] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0095] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0096] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A query-efficient image encoder stealing method, characterized in that: The method obtains output features of part of the data from the target encoder with a preset query amount, and uses the output features as external supervision signals of the stealing model; constructs a stealing model and a perturbation module for perturbing the feature output of the stealing model, and constructs a joint loss function based on the external supervision signal obtained from the query data and the self-learning mechanism of the unqueried data to optimize the stealing model and the perturbation module; Image encoder stealing attack is implemented with the optimized stealing model.

2. The method for stealing an image encoder with high query efficiency according to claim 1, characterized in that: The method comprises the following steps: S1 divides the dataset into a query set and an unqueried set; S2 queries the target encoder to obtain the feature vector of the sample in the query set; S3 builds a stealing model, the structure of which is the same as or different from the target encoder; builds a learnable perturbation module to perturb the feature output of the stealing model; S4 optimizes the stealing model and perturbation module based on the joint loss function; S5 implements image encoder stealing attack with an optimized stealing model.

3. The image encoder stealing method with high query efficiency according to claim 2, characterized in that: In S1, the samples in the query set are less than those in the non-query set.

4. The method for stealing an image encoder with high query efficiency according to claim 2, characterized in that: The learnable perturbation module is arranged in cooperation with the output layer of the stealing model; the perturbation module includes a multi-layer perceptron including two hidden layers; the perturbation learned during the model training process is gradually reduced.

5. The method for stealing an image encoder with high query efficiency according to claim 2, characterized in that: In S4, the joint loss function includes the cross-model alignment loss , self-learning loss and disturbance loss ,satisfy, 。 6. The method for stealing an image encoder with high query efficiency according to claim 5, characterized in that: Cross-model alignment loss The samples of the query set and their corresponding enhanced samples are used as training data and input into the target encoder and the stealing model respectively; in each round of training, the difference between the output of the stealing model and the target encoder is calculated based on the mean square error to optimize the parameters of the stealing model.

7. The method for stealing an image encoder with high query efficiency according to claim 5, characterized in that: Self-learning loss In order to perform random data augmentation on all samples in the dataset, all samples in the dataset and their augmented samples are input into the stealing model, a self-learning loss function is constructed, and the parameters of the stealing model are optimized based on the loss function.

8. The method for stealing an image encoder with high query efficiency according to claim 5, characterized in that: Perturbation loss The mean square error of the target encoder and the perturbation module for the samples of the query set, and the mean square error of the stealing model and the perturbation module for all samples of the data set are obtained respectively, and the parameters of the stealing model are optimized based on the sum of the two.

9. An application of the query efficient image encoder stealing method according to any one of claims 1 to 8, characterized in that: Based on the optimized theft model, the defense drill of the image encoder is implemented and applied to the development of security modules.

10. The use according to claim 9, characterized in that: In the target encoder, the structural watermark is integrated into the extracted features via data augmentation and an enhanced predictor is trained.

Images