Split-learning system and adversarial example creation method based on split-learning

Abstract

A split learning system and a method for generating adversarial examples based on split learning are disclosed. During forward propagation, the server of the split learning system acquires the input layer output from the client and inputs pseudo-training data into the shadow input layer to obtain the shadow input layer output. A similarity loss is calculated, which characterizes the similarity between the input layer output and the shadow input layer output. During backpropagation, the server layer gradient information and the similarity gradient information calculated based on the similarity loss are calculated, and the fused gradient information is returned to the client. This invention addresses the problem of missing parts in the shared model through shadow model training and evaluates it through a local adversarial attack that generates adversarial examples. Shadow model training requires only a small amount of unlabeled, similar data, and adversarial examples can be generated by modifying the intermediate outputs of natural samples. The above attack process is low-cost and effective, demonstrating the vulnerability of existing split models.

Description

Technical Field

[0001] This disclosure relates to the field of deep learning, and more particularly to a split learning system and a method for creating adversarial examples based on split learning. Background Technology

[0002] In recent years, artificial intelligence has ushered in a new wave of development, with machine learning playing a central role. Training a high-performing machine learning model requires collecting a large amount of high-quality data. However, in many application scenarios, privacy protection prevents the collection of private user data for centralized model training on servers, hindering the widespread adoption of machine learning.

[0003] To address this, multi-party collaborative modeling, which enables participants to collaboratively train a model without disclosing data, has been proposed to overcome data privacy concerns. A significant and common scenario in multi-party collaborative modeling is split learning (SL). Split learning mitigates privacy issues by storing sensitive raw data locally on the client and exchanging intermediate computations only with the server. However, while split learning holds broad promise for protecting data privacy, model security remains a concern.

[0004] Therefore, it is necessary to verify the security of the model based on split learning. Summary of the Invention

[0005] One technical problem this disclosure aims to solve is to provide a split-learning system and a method for generating adversarial examples based on split-learning. The server in this system can train a shadow input layer and generate adversarial examples sufficient to cause the trained model to misclassify, without the client's knowledge and with extremely low computational and data collection costs, thus posing a challenge to the security of split-learning models.

[0006] According to a first aspect of this disclosure, a split learning system is provided, comprising a client and a server, wherein an input layer of a neural network model is disposed on the client, and a server layer of the neural network model is disposed on the server, characterized in that a shadow input layer is further disposed on the server, and is used in the i-th iteration of the training phase of the neural network model to: during forward propagation, acquire the input layer output from the client, wherein the input layer output is generated by inputting training data into the input layer; input pseudo-training data into the shadow input layer to acquire the shadow input layer output; calculate a similarity loss, the similarity loss being used to characterize the similarity between the input layer output and the shadow input layer output; during backpropagation, calculate server layer gradient information and similarity gradient information calculated based on the similarity loss, and return fused gradient information, which integrates the server layer gradient information and the similarity gradient information, to the client, wherein the similarity gradient information is used to update the parameters of the shadow input layer, and the fused gradient information is used by the client to update the parameters of the input layer.

[0007] Optionally, the client also has an output layer of the neural network model. The input layer, the server layer, and the output layer constitute a complete neural network model. The server is used to: during forward propagation, take the output of the input layer as the input of the server layer to generate the server layer output and provide the server layer output to the client; and during back propagation, obtain the gradient information of the output layer and calculate the gradient information of the server layer based on the gradient information of the output layer, wherein the gradient information of the output layer is generated based on the server layer output and the label data of the training data.

[0008] Optionally, the client and the server complete the training of the neural network model after a predetermined number of iterations, and the server is used during the inference phase of the neural network model to: use the trained shadow input layer and the server layer to create adversarial examples from natural samples for inputting into the input layer.

[0009] Optionally, using the trained shadow input layer and server layer to create adversarial samples for inputting into the input layer includes: feeding the natural samples and perturbed samples with added perturbations into an adversarial sample creation network formed by connecting the shadow input layer and the server layer; obtaining intermediate calculation results of the natural samples and the perturbed samples output by the adversarial sample creation network; and generating the adversarial samples based on the intermediate calculation results of the natural samples and the perturbed samples.

[0010] Optionally, generating the adversarial sample based on the intermediate calculation results of the natural sample and the intermediate calculation results of the perturbation sample includes: finding an optimized perturbation value within the perturbation budget that maximizes the cosine distance between the intermediate calculation results of the natural sample and the intermediate calculation results of the perturbation sample; and adding the optimized perturbation value to the natural sample to obtain the adversarial sample.

[0011] Optionally, the input layer and the server layer constitute a complete neural network model, the output of the server layer is used as the output of the neural network model, and the server is used to: calculate a loss function based on the output of the server layer and the label data of the training data, and generate the gradient information of the output layer based on the loss function.

[0012] Optionally, the pseudo-training data is similar data to the training data collected by the server based on the structure of the server layer.

[0013] According to a second aspect of this disclosure, a method for creating adversarial examples based on split learning is provided, applied to a server of a split learning system, and the method includes: during the neural network model training phase: during forward propagation, acquiring the input layer output from a client, wherein the input layer output is generated by inputting training data into the input layer; inputting pseudo-training data into the shadow input layer to obtain the shadow input layer output; calculating a similarity loss, the similarity loss being used to characterize the similarity between the input layer output and the shadow input layer output; during back propagation, calculating server layer gradient information and similarity gradient information calculated based on the similarity loss, and returning fused gradient information, which integrates the server layer gradient information and the similarity gradient information, to the client; and during the neural network model inference phase: creating adversarial examples using an adversarial example creation network formed by connecting the shadow input layer and the server layer.

[0014] Optionally, creating adversarial examples using an adversarial example creation network consisting of the shadow input layer and the server layer includes:

[0015] The natural samples and the perturbed samples with added perturbations are fed into an adversarial sample creation network consisting of the shadow input layer and the server layer.

[0016] Obtain intermediate computation results of the natural samples and the perturbation samples output by the adversarial sample generation network; and

[0017] The adversarial sample is generated based on the intermediate calculation results of the natural sample and the intermediate calculation results of the perturbation sample.

[0018] According to a third aspect of this disclosure, a computing device is provided, comprising: a processor; and a memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method described in the first aspect above.

[0019] According to a fourth aspect of this disclosure, a non-transitory machine-readable storage medium is provided, on which executable code is stored, which, when executed by a processor of an electronic device, causes the processor to perform the method described in the first aspect above.

[0020] Therefore, this invention addresses the issue of missing parts in shared models through shadow model training and evaluates local adversarial attacks by generating adversarial examples. Shadow model training requires only a small amount of unlabeled, similar data, and adversarial examples can be generated by modifying the intermediate outputs of natural samples. The aforementioned attack process is low-cost and effective, demonstrating the vulnerability of existing splitting models. Attached Figure Description

[0021] The above and other objects, features and advantages of this disclosure will become more apparent from the more detailed description of exemplary embodiments thereof taken in conjunction with the accompanying drawings, wherein like reference numerals generally denote like parts.

[0022] Figure 1 The diagram illustrates the training of a model using conventional split learning and conventional split learning.

[0023] Figure 2 An example of constructing adversarial samples for adversarial attacks is shown.

[0024] Figure 3 A schematic diagram of the components of the split learning system that enables the present invention is shown.

[0025] Figure 4 A training diagram of a neural network model according to an embodiment of the present invention is shown.

[0026] Figure 5 A schematic flowchart of an adversarial example creation method based on split learning according to an embodiment of the present invention is shown.

[0027] Figure 6 A schematic diagram of a computing device is shown, which can be used to implement the above-described adversarial example creation method based on split learning according to an embodiment of the present invention. Detailed Implementation

[0028] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0029] The outstanding performance of artificial neural networks, especially deep neural networks (DNNs) with multiple hidden layers, is driven by the increasing model complexity and availability of large datasets, as large DNN models are better able to capture and store knowledge data from massive amounts of data.

[0030] However, training DNN models with millions or billions of parameters is extremely expensive, requiring sufficiently high-performance machines; and the powerful generalization capabilities of DNNs rely on a large amount of high-quality training data. In real-world applications, data is often scattered across different locations, and direct access to or collection of this data raises data privacy concerns. For example, in the era of the Internet of Things (IoT), edge devices are generating and collecting massive amounts of data, but the sensitivity of this data and the limited computing power of edge devices make unlocking the potential of the collected data challenging. Therefore, distributed training, which splits data and computation across multiple computing nodes, has attracted considerable attention in recent years.

[0031] Split learning (SL) has emerged as a promising distributed solution to address the aforementioned problems. Split learning enables collaborative model training by dividing the entire network into sub-networks computed by different participants (i.e., clients and servers). It mitigates privacy concerns by keeping sensitive raw data locally on the client and exchanging intermediate computations only with the server. During training, the server bears most of the computational cost, making split learning lightweight and scalable for clients with limited computing resources. In split learning, the model is divided into two sub-models: an input layer and a server layer. The client trains the server layer. Then, the output of the input layer is sent to the server, which uses the server layer to complete the remaining forward propagation process without evaluating the private raw data from the client. The backpropagation process is performed in a similar manner. However, ordinary split learning requires the client to expose labels to the server. To further reduce privacy leaks, a U-shaped split learning configuration has been proposed. In this configuration, the last few layers of the complete model, the output layer, are further split and handed over to the client. This eliminates the need for label sharing.

[0032] Figure 1The diagram illustrates conventional split learning and its model training. As shown on the left, in conventional split learning (a), a complete neural network model is divided into two parts: an input layer and a server layer. The input layer corresponds to the top layers of the model and is located on the client side to receive the input training data and generate intermediate computation results. The server layer is located on the server and includes all layers below the input layer (i.e., also the output). The server layer receives the intermediate computation results and continues computation, comparing the generated output with the labels to calculate the loss function. Then, it calculates the gradient based on the loss function using the backpropagation algorithm and updates the parameters on the server layer. The gradient calculated by the server layer is passed to the client, which then calculates the gradient of the input layer and updates the parameters. Obviously, in conventional split learning, the server needs to know the label corresponding to each round of data input, thus increasing the risk of privacy leakage.

[0033] As shown on the right side of the figure, in the conventional split learning corresponding to (b), a complete neural network model is divided into three parts: the input layer, the server layer, and the output layer. The input layer corresponds to the top layers of the model and is located on the client side. It receives the input training data and generates the first intermediate computation result. The server layer is located on the server and includes all intermediate layers except those below the input layer and above the output layer. The server layer receives the first intermediate computation result and continues computation to generate the second intermediate computation result. The output layer corresponds to the bottom layers of the model and is located on the client side. It receives the second intermediate computation result to generate the model output. The training data labels are stored locally on the client side and are used to compare with the model output to calculate the loss function. Then, based on the loss function, the output layer gradient is calculated using the backpropagation algorithm to update the output layer parameters. The output layer gradient is provided to the server to calculate the server layer gradient. The calculated server layer gradient is used to update the server layer parameters and is passed to the client. The client then calculates the input layer gradient and updates the parameters. Clearly, in U-shaped split learning, the server is unaware of the labels corresponding to each round of data input, which better protects data privacy on the client side.

[0034] While the solution of this invention is applicable to conventional split learning, it is even more suitable for verifying the vulnerabilities of split learning with a U-shaped configuration. First, this type of split learning is more advantageous because it allows for the protection of data and labels. Second, launching attacks in this setup is more challenging because the available server resources are much more limited compared to conventional split learning. Third, attacks developed in U-shaped split learning are more general and can be seamlessly integrated into conventional split learning.

[0035] Much work on split learning has focused on verifying whether it can truly protect privacy. However, validation regarding model security in split learning is lacking. Widespread deployment of insecure models in practice can cause significant losses, especially in critical sectors such as security and healthcare.

[0036] Typically, model security refers to the desire to keep publicly available models trained on them, as attackers could exploit this information to launch adversarial attacks against model-based applications. By accessing the model, adversarial attacks can create adversarial noise that is difficult for humans to detect, and then impose this noise onto natural samples, generating adversarial examples that cause the model to malfunction.

[0037] Figure 2 This illustrates an example of constructing adversarial examples for adversarial attacks. As shown in the figure, the original panda image on the left is input into a trained image classification model, which infers that the image contains a panda with over 50% confidence. However, after an attacker adds some specially designed noise to generate an adversarial example, the image on the right is obtained. The image on the right (i.e., the adversarial example obtained by adding specially designed noise to the original image) still appears to the human eye as a panda image, and the two images are indistinguishable to the human eye. However, the neural network will produce an unexpected output on this adversarial example, for example, inferring with extremely high confidence that the image contains a gibbon.

[0038] The vulnerability of deep neural networks to adversarial attacks is a major constraint on their deployment in high-security scenarios such as autonomous driving and healthcare. Since adversarial attacks can expose the blind spots of deep neural networks, developing effective adversarial attacks has become a fundamental task in the field of deep learning security.

[0039] Despite the impressive performance of DNNs on many complex tasks, the internal workings of DNNs remain opaque to humans, and this functionality can be maliciously explored to launch adversarial attacks against DNNs.

[0040] Adversarial attacks inject small amounts of noise into natural samples along the vulnerable direction of the model to create adversarial examples, and numerous studies have shown that DNNs are significantly vulnerable to adversarial examples. Typically, if the DNN is allowed full accessibility, the input gradient direction is considered the vulnerable direction of the model, i.e., the gradient of the loss function with respect to the data is used. Natural samples serve as adversarial noise. This scenario is also known as a white-box scenario. In traditional training paradigms, the model trainer is usually the model deployer, and the trained model is unlikely to be leaked to untrusted individuals. Therefore, launching attacks in a black-box scenario is more common in practical applications. Black-box attacks mainly rely on the transferability of adversarial examples, i.e., adversarial examples created locally on a proxy model can sometimes also fool another unknown model. Furthermore, transfer-based black-box attacks specifically follow two stages: 1) collecting a sufficient amount of data and its labels and training the proxy model from scratch, and 2) creating adversarial examples locally. For the first stage, it is difficult to obtain a training dataset similar to the target model in the real world, and powerful computing capabilities are also required. Therefore, adversarial attacks are generally difficult to launch.

[0041] In traditional DNN training paradigms, the model trainer is typically the model deployer, and the trained model is unlikely to be leaked to an untrusted server. In contrast, for split learning, untrusted servers are common, and a portion of the model is delivered to the server. Therefore, this invention presents a feasible and low-cost scheme for creating adversarial examples on untrusted servers, revealing the vulnerability of split learning to adversarial attacks.

[0042] In this invention, the proxy model training method commonly used in adversarial attacks is abandoned. Instead, the deep involvement of the server is required in split learning. A shadow input layer is co-trained on the server while the client's input layer is being trained. Since the server has a server layer, its structure allows it to determine the data type of the training data (e.g., whether the training data is image or speech), and it can easily collect similar data. This collected similar data (hereinafter referred to as "pseudo-training data") can be provided to the shadow input layer. Because neural network models extract more generalized features from training data at higher layers, the input data of the shadow input layer can be co-trained to a considerable extent even if it differs from the actual training data input to the input layer. The trained shadow input layer can be merged with the server layer to form an adversarial example creation network, used to add fabricated small perturbations to natural samples to obtain adversarial examples. In a preferred embodiment, the solution of this invention is applied to U-shaped split learning. Since the output layer is located on the client, the network output obtained by merging the shadow input layer and the server layer actually corresponds to the intermediate calculation results. At this point, the perturbation can be optimized and adversarial examples can be created by maximizing the distance between the intermediate calculation results of the natural samples and the intermediate calculation results after adding the perturbation (within the perturbation budget).

[0043] Figure 3 A schematic diagram illustrating the composition of the split learning system capable of implementing the present invention is shown. As shown, the split learning system 300 may include a server 310 and multiple clients 320. The server 310 can provide model training services and intermediate layer computation services for the inference phase to the multiple clients 320. In the illustrated example, the client 320 has the input layer and output layer of the neural network model, while the server 310 has a server layer corresponding to the intermediate layer of the neural network. In other examples, the client 320 may have the input layer of the neural network model, while the server 310 may have a server layer corresponding to the intermediate and output layers of the neural network.

[0044] In this invention, to achieve low-cost adversarial attacks, a shadow input layer is also deployed on the server 310. During the neural network model training phase, the shadow input layer is co-trained on the server in a manner unknown to the client. The trained shadow input layer can then be combined with the trained server layer for the creation of adversarial examples.

[0045] Figure 4 A training graph of a neural network model according to an embodiment of the present invention is shown. Figure 4As shown, for simplicity, the split learning system 400 includes only one server 410 and one client 420. Model training may include T iterations (shown below in conjunction with Table 1), and in the i-th iteration of the neural network model training phase, the server 410 is used to: during forward propagation, acquire the input layer output from the client, wherein the input layer output is generated by inputting training data into the input layer. Before backpropagation in the server layer, the server 410 may input pseudo-training data into the shadow input layer to obtain the shadow input layer output and calculate a similarity loss. The pseudo-training data is similar data to the training data collected by the server based on the structure of the server layer, for example, both being image data and both being speech data. The similarity loss is used to characterize the similarity between the input layer output and the shadow input layer output. In one embodiment, the similarity loss may be the distance between the input layer output and the shadow input layer output (e.g., L2 norm).

[0046] During backpropagation, server 410 can calculate server layer gradient information and similarity gradient information calculated based on the similarity loss, and return the fused gradient information, which combines the server layer gradient information and the similarity gradient information, to the client. Here, the similarity gradient information is used to update the parameters of the shadow input layer, and the fused gradient information is used by the client to update the parameters of the input layer.

[0047] Figure 4 The example illustrates a U-shaped configuration where the client includes an output layer. That is, the client 420 also houses the output layer of the neural network model; the input layer, the server layer, and the output layer constitute a complete neural network model.

[0048] During forward propagation, server 410 uses the input layer output obtained from client 420 as input to the server layer to generate server layer output, and provides the server layer output to client 420. During backward propagation, server 410 obtains the output layer gradient information transmitted by client 420 and calculates the server layer gradient information based on the output layer gradient information. This output layer gradient information can be generated based on the server layer output and the label data of the training data.

[0049] Specifically, during the training of the split learning model for the U-shaped configuration, in the forward propagation phase, the client 420 uses local training data to input the model's input layer (e.g., extracting data x and label y from the client-owned dataset D1, with data x being input into the input layer located on the client), and computes the output of the input layer. (Here, the input layer corresponds to) And send o1 to the server.

[0050] Server computing server layer output (Here, the server layer corresponds to) ), and pass o2 to the output layer on the client (here, the output layer corresponds to ).

[0051] Unlike conventional split learning operations, after obtaining the output o1 of the input layer (and before the subsequent backpropagation calculation), the server extracts pseudo-training data x′ from its own dataset D2, feeds it into the shadow output layer, and calculates the output of the shadow output layer. And calculate the output o1 of the output layer and the output of the shadow output layer. The similarity loss between them. In one embodiment, the similarity loss L sim It can be the distance between two outputs, such as the L2 norm, i.e. Furthermore, it should be understood that the server is unaware of the specific training data input by the client. However, based on its own server layer architecture, the server can determine the type of training data (e.g., image data) and can collect similar data as pseudo-training data. Since the collected data does not require labels (the shadow input layer only extracts generalized information, such as contour information in an image), the cost of collecting unlabeled data is very low. The similarity loss L obtained from this is... sim It can be used to calculate the fused gradient in the subsequent backpropagation.

[0052] Similarly, in the forward propagation, the client's output layer receives the server layer output from the server. And thus, the output of the output layer (which is also the training output of the model) is obtained. Subsequently, the loss is calculated based on a predetermined loss function (e.g., cross-entropy loss function) and the label y corresponding to the training data x. And execute the backpropagation algorithm to output the gradient of the layer. (can also be expressed as) The output layer gradient is sent to the server. Parameter updates in this iteration.

[0053] The server receives the output layer gradient. Next, calculate its own server gradient. Simultaneously calculate the similarity gradient. The server can then calculate the fusion gradient g = g1 + αg2 (α is a hyperparameter that adjusts the similarity, which can be equal to 1 in a default setting) and return g to the client. The server gradient g1 can be used in the server layer. Parameter updates in this iteration.

[0054] Unlike existing technologies, the gradient received by the client is not the server gradient, but a fused gradient that incorporates similarity gradients. This fused gradient is then used to calculate the client's own input layer gradient for parameter updates. This means that the client's gradient calculation for the input layer actually includes the influence of similarity gradients (i.e., it causes the client's input layer to move closer to the shadow input layer). Similarly, the parameters of the shadow input layer updated using similarity gradients will also move closer to the input layer. (See the application examples, especially the description in Table 1, for details).

[0055] In addition, although not in Figure 4 As shown, however, in one embodiment, a complete neural network model can also be constructed from an input layer and a server layer. That is, the server layer is used as the output of the neural network model (corresponding to...). Figure 1 (as shown in part (a)). At this time, server 410 is used to: calculate a loss function based on the label data of the server layer output and the training data, and generate the gradient information of the output layer based on the loss function.

[0056] At this point, the neural network to be trained only includes those corresponding to... The input layer (located on the client side) and corresponding to The server layer (located on the server) in the U-shaped setup has its output layer on the client side incorporated into the server layer. While the server layer does not execute the training data input for each iteration, it knows the corresponding label y. During the forward propagation phase, the client 420 uses its local training data to input the model's input layer (e.g., extracting data x and label y from the client-owned dataset D1, with data x being input to the input layer on the client), and computes the output of the input layer. Then send o1 to the server.

[0057] Server computing server layer output (Here, the server layer output is the model output), and calculate the loss, and calculate the server layer gradient g1 based on the loss, which is used for parameter updates on the server layer.

[0058] Similarly, after obtaining the output o1 of the input layer, the server will extract pseudo-training data x′ from its own dataset D2, and send it to the shadow output layer to calculate the output of the shadow output layer. And calculate the output o1 of the output layer and the output of the shadow output layer. Similarity loss between sim Similarity loss is used to calculate the similarity gradient. The server then calculates the fused gradient g = g1 + αg2 and returns g to the client. The client receives the fused gradient and uses it to calculate its own input layer gradient for parameter updates. This allows the input layer parameters to gradually approach the shadow output layer over multiple iterations.

[0059] Client 420 and server 410 complete the training of the neural network model after a predetermined number of iterations (e.g., T times), and the server, during the inference phase of the neural network model, is used to: create adversarial examples from natural samples for input to the input layer using the trained shadow input layer and server layer. Figure 4 In this example, the server layer does not include an output layer, so the server cannot generate adversarial examples based on the labels. In this case, the optimized perturbation value can be directly obtained based on the intermediate results generated by the server layer. Therefore, generating adversarial examples from natural samples for input to the input layer using the trained shadow input layer and the server layer includes: feeding the natural samples and perturbated samples into an adversarial example generation network formed by connecting the shadow input layer and the server layer; obtaining the intermediate calculation results of the natural samples and the perturbated samples output by the adversarial example generation network; and generating the adversarial examples based on the intermediate calculation results of the natural samples and the perturbated samples.

[0060] Furthermore, generating the adversarial sample based on the intermediate calculation results of the natural sample and the intermediate calculation results of the perturbation sample includes: finding an optimized perturbation value within the perturbation budget that maximizes the cosine distance between the intermediate calculation results of the natural sample and the intermediate calculation results of the perturbation sample; and adding the optimized perturbation value to the natural sample to obtain the adversarial sample.

[0061] When creating local adversarial examples, the server can specifically perform the following iterative operations (e.g., a total of K iterations) to determine the optimized perturbation value δ:

[0062] First, initialize the adversarial noise δ for the natural sample x (where x represents a natural sample, not a training sample), for example, as a zero vector. Then, feed both the natural sample x and the perturbated sample x+δ into the surrogate model composed of the trained shadow input layer and server layer, obtaining the server layer output respectively. and Since the proxy model in the U-shaped configuration application does not include an output layer, o2 and o′2 are intermediate calculation results. Subsequently, the loss L between the two can be calculated. attack In one embodiment, This is equivalent to the cosine similarity between two intermediate calculation results.

[0063] Therefore, δ can be optimized within the interference budget ∈, such that L attack The more, the better. In other words, the optimization goal of the K iterations is to maximize the distance between the proxy model outputs of natural samples and perturbation samples, while ensuring that δ does not exceed the perturbation budget (to avoid visually noticeable differences between the created adversarial examples and natural samples). This makes it easier for the created adversarial examples to achieve successful misclassification attacks when input to the client's input layer (see the application examples below, especially Table 2).

[0064] Therefore, this invention uses a shadow input layer instead of the input layer, and combines the shadow input layer and the server layer as a proxy model. A similarity loss is also added to improve the similarity between the input layer and the shadow input layer, resulting in better attack performance.

[0065] This invention can also be implemented as a method for creating adversarial examples based on split learning. Figure 5 A schematic flowchart illustrating a method for creating adversarial examples based on split learning according to an embodiment of the present invention is shown. The method is applied to a server of a split learning system, such as server 410.

[0066] During forward propagation, in step S510, the input layer output from the client is obtained. The input layer output is generated by inputting training data into the input layer.

[0067] In step S520, pseudo-training data is input into the shadow input layer to obtain the shadow input layer output.

[0068] In step S530, a similarity loss is calculated. This similarity loss is used to characterize the similarity between the input layer output and the shadow input layer output.

[0069] Subsequently, during backpropagation, in step S540, the server layer gradient information and the similarity gradient information calculated based on the similarity loss are calculated, and the fused gradient information that combines the server layer gradient information and the similarity gradient information is returned to the client.

[0070] Steps S510-S540 above are implemented during the neural network model training phase, thereby obtaining the input layer, server layer, and co-training input layer and shadow input layer through T iterations of training.

[0071] Therefore, in the subsequent neural network model inference stage, in step S550, adversarial examples can be created using an adversarial example creation network formed by connecting the shadow input layer and the server layer.

[0072] In the U-shaped configuration, the adversarial example creation network formed by connecting the shadow input layer and the server layer is not a complete network capable of outputting classification results (because it does not include an output layer). In this case, creating adversarial examples using the adversarial example creation network formed by connecting the shadow input layer and the server layer includes: feeding the natural samples and perturbation samples with added perturbations into the adversarial example creation network formed by connecting the shadow input layer and the server layer; obtaining the intermediate calculation results of the natural samples and the perturbation samples output by the adversarial example creation network; and generating the adversarial examples based on the intermediate calculation results of the natural samples and the perturbation samples.

[0073] Specifically, when training the split learning model for the U-shaped configuration, during the forward propagation phase, the client uses local training data to input the model's input layer and calculates the output of the input layer. (Here, the input layer corresponds to) And send o1 to the server.

[0074] Server computing server layer output And pass o2 to the output layer on the client (here, the output layer corresponds to...) ).

[0075] Unlike conventional split learning operations, after obtaining the output o1 of the input layer (and before the subsequent backpropagation calculation), the server extracts pseudo-training data x′ from its own dataset D2, feeds it into the shadow output layer, and calculates the output of the shadow output layer. And calculate the output o1 of the output layer and the output of the shadow output layer. Similarity loss between sim .

[0076] Similarly, in the forward propagation, the client's output layer receives the server layer output from the server. And thus, the output of the output layer (which is also the training output of the model) is obtained. Subsequently, the loss is calculated based on a predetermined loss function (e.g., cross-entropy loss function) and the label y corresponding to the training data x. And execute the backpropagation algorithm to output the gradient of the layer. Send to the server. The output layer gradient can be used in the output layer. Parameter updates in this iteration.

[0077] The server receives the output layer gradient. Next, calculate its own server gradient. Simultaneously calculate the similarity gradient. The server can then calculate the fused gradient g = g1 + αg2 and return g to the client. The server gradient g1 can be used at the server layer. Parameter updates in this iteration.

[0078] Figure 6 A schematic diagram of a computing device 600, according to an embodiment of the present invention, is shown for implementing the above-described adversarial example creation method based on split learning. The computing device 600 can, in particular, be implemented as a server with considerable processing power.

[0079] See Figure 6 The computing device 600 includes a memory 610 and a processor 620.

[0080] Processor 620 may be a multi-core processor or may contain multiple processors. In some embodiments, processor 620 may include a general-purpose main processor and one or more special-purpose coprocessors, such as a graphics processing unit (GPU), a digital signal processor (DSP), etc. In some embodiments, processor 620 may be implemented using custom circuitry, such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0081] Memory 610 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by the processor 620 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 610 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, memory 610 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-high-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0082] The memory 610 stores executable code, which, when processed by the processor 620, enables the processor 620 to execute the adversarial example creation method based on split learning described above.

[0083] Application examples

[0084] This invention proposes an attack method called SLADV that effectively utilizes the split learning characteristic. SLADV consists of two phases: shadow model training and local adversarial attack. Shadow model training and local adversarial attack correspond to the model training phase and the model deployment phase, respectively. Tables 1 and 2 below summarize these two phases.

[0085] 1. Shadow Model Training

[0086] The primary goal of this phase is to train a proxy model, which is then used to generate adversarial examples to attack the target model. Common transfer-based attacks assume that the entire label domain of the target model is accessible and that a large amount of data with the same distribution as the target model's training set is collected. The attacker then constructs a proxy model with a similar architecture to the target model and trains it on the collected data. During the attack phase, adversarial examples generated by reducing the proxy model's confidence in the true labels onto the original samples through noise can deceive the target model.

[0087] Similarly, SLADV also trains a proxy model to launch attacks. In split learning, the server holds the server layer, which can be maliciously explored to reduce the cost of launching an attack. An obvious approach is to create and train shadow input and shadow output layers (compatible with the server layer), and then concatenate the shadow input, server, and shadow output layers as a proxy model. However, this invention does not use this obvious structure because, for cheap attacks, attackers do not query the true labels of the collected data. Without labels, loss cannot be computed, therefore the shadow input and shadow output layers are untrainable without supervision. Furthermore, even if the labels of the collected data were available, attack problems still exist. Specifically, since the label domain of the target model is inaccessible, there may be a significant gap between the label domain of the collected data and the label domain of the target model, indicating poor similarity between the proxy model and the target model, leading to a low attack success rate.

[0088] Therefore, this invention completely avoids the use of a shadow output layer and can only rely on a shadow input layer and a server layer. Since SLADV cannot routinely generate adversarial examples by reducing the prediction probability of the corresponding label in the proxy model, intermediate settlement results are preferred as an alternative (see section 2 below).

[0089] In split learning, the client sends the output of the input layer to the server; therefore, this invention uses the output of the input layer as the supervision signal for the shadow input layer. Specifically, in each iteration, an additional difference loss term, L2-norm, is introduced to measure the difference between the outputs of the input layer and the shadow input layer relative to their respective data. Thus, through the introduction of this loss term, the similarity between the input layer and the shadow input layer increases with iteration. Since the difference loss term is calculated by the server, it is unknown to the client. In each backpropagation process, the server simply fuses the gradients of the original loss and the difference loss, and then returns the fused gradient to the client. The client cannot detect gradient tampering simply by observing the returned gradient. Formally, by representing the client and server data as D1 and D2 respectively, the forward process can be summarized as follows:

[0090]

[0091] in This is the shadow input layer, L(·,·) is the regular loss function, x,y ~ D1, x′ ~ D2. During backpropagation, the following steps are executed sequentially:

[0092] • Client through Update θ3 and Send back to the server;

[0093] • The servers respectively pass through Update θ2 and θ'1, and These are sent separately to the client, where α is the fusion factor;

[0094] • Client through Update θ1.

[0095] The entire process is summarized in Table 1.

[0096]

[0097]

[0098] Table 1

[0099] Besides utilizing the difference between the outputs of the input layer and the shadow input layer as a supervision signal, another embodiment can also leverage the output difference of the server layer. However, implementations utilizing server layer output differences perform worse because models tend to extract general features at shallower layers and more specific features at deeper layers. It has been observed that forcibly aligning the server layer outputs can lead to model performance degradation due to the non-negligible differences in data distribution between the client and server. In contrast, aligning the input layer outputs makes more sense, as the input layer typically extracts features shared by different types of data.

[0100] 2. Local counter-attacks

[0101] After the shadow model is trained, the trained shadow input layer serves as a replacement for the input layer. Here, the shadow input layer and the server layer are combined as a proxy model. Since the resulting proxy model lacks an output layer, this indicates that common techniques for creating adversarial examples in transfer-based attacks cannot be used in this invention, as these methods require probability prediction. Therefore, SLADV does not create adversarial examples by reducing prediction probabilities, but rather by perturbing intermediate outputs. Intuitively, since the output layer needs to predict based on the server layer's output, the target model can be fooled by changing the server layer's output. Furthermore, if the shadow input layer and the input layer are sufficiently similar, this noise, which causes a significant change in the combined output of the shadow input layer and the server layer, may also have a consistent effect on the combined output of the input layer and the server layer. The inventors have demonstrated the effectiveness of the above idea under normal conditions (mild regularization). In practice, SLADV achieves this idea by optimizing noise that reduces the cosine similarity between the intermediate outputs of perturbed and undisturbed samples. Mathematically, this can be achieved by initializing δ to a zero vector, and SLADV iterates over K iterations according to the following update equation:

[0102]

[0103] Where β is the update step size, and ∈ is the given perturbation budget. If the input exceeds the perturbation budget, Clip∈{·} will plot the input onto the ∈-ball (usually the ∞-norm).

[0104]

[0105] Table 2

[0106] The security verification scheme for the split-learning model according to the present invention has been described in detail above with reference to the accompanying drawings. The present invention uses a highly challenging split-learning setup (where an untrusted server only stores the intermediate layers of the entire model, i.e., the attacker has minimal available resources) to evaluate the robustness of split-learning against adversarial attacks. Existing adversarial attacks typically focus on centralized training setups rather than collaborative setups, indicating an inability to effectively utilize the characteristics of split-learning and leading to an overestimation of robustness. To address this issue, delving into the underlying layers of split-learning, the present invention proposes an attack method called SLADV that fully utilizes the characteristics of split-learning. SLADV comprises two phases: shadow model training to address the problem of missing parts of the shared model, and a local adversarial attack that generates adversarial examples for evaluation. Shadow model training requires only some unlabeled, similar data, and in the second phase, SLADV changes the intermediate output of natural samples to generate adversarial examples. The above attack process is low-cost and effective.

[0107] Furthermore, the method according to the present invention can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing the steps defined in the above-described method of the present invention.

[0108] Alternatively, the present invention can also be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) storing executable code (or computer program, or computer instruction code) thereon, which, when executed by a processor of an electronic device (or computing device, server, etc.), causes the processor to perform the various steps of the method described above according to the present invention.

[0109] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both.

[0110] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0111] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A split learning system, comprising a client and a server, wherein the client has an input layer and an output layer of a neural network model, and the server has a server layer of the neural network model, characterized in that, The input layer, the server layer, and the output layer constitute a complete neural network model. A shadow input layer is also deployed on the server and is used in the i-th iteration of the neural network model training phase for: During forward propagation, the input layer output from the client is obtained, wherein the input layer output is generated by inputting training data into the input layer; Pseudo-training data is input into the shadow input layer to obtain the shadow input layer output; Calculate the similarity loss, which is used to characterize the similarity between the output of the input layer and the output of the shadow input layer; During backpropagation, the server-layer gradient information and the similarity gradient information calculated based on the similarity loss are computed, and the fused gradient information, which combines the server-layer gradient information and the similarity gradient information, is returned to the client. The similarity gradient information is used to update the parameters of the shadow input layer, and the fusion gradient information is used by the client to update the parameters of the input layer. The server is configured to: during forward propagation, use the input layer output as the input of the server layer to generate the server layer output, and provide the server layer output to the client; and during backpropagation, acquire the output layer gradient information, and calculate the server layer gradient information based on the output layer gradient information, wherein the output layer gradient information is generated based on the server layer output and the label data of the training data.

2. The system as claimed in claim 1, wherein, The client and the server complete the training of the neural network model after a predetermined number of iterations, and the server is used during the inference phase of the neural network model to: Natural samples are used to create adversarial samples for input into the input layer using the trained shadow input layer and server layer.

3. The system as described in claim 2, wherein, Creating adversarial examples from natural samples for input into the input layer using the trained shadow input layer and server layer includes: The natural samples and the perturbed samples with added perturbations are fed into an adversarial sample creation network consisting of the shadow input layer and the server layer. Obtain intermediate computation results of the natural samples and the perturbation samples output by the adversarial sample generation network; and The adversarial sample is generated based on the intermediate calculation results of the natural sample and the intermediate calculation results of the perturbation sample.

4. The system of claim 3, wherein, Generating the adversarial examples based on the intermediate calculation results of the natural samples and the intermediate calculation results of the perturbation samples includes: Find the optimal perturbation value within the perturbation budget that maximizes the cosine distance between the intermediate calculation results of the natural sample and the intermediate calculation results of the perturbation sample; and The optimized perturbation value is added to the natural sample to obtain the adversarial sample.

5. The system as claimed in claim 1, wherein, The input layer and the server layer constitute a complete neural network model. The output of the server layer is used as the output of the neural network model, and the server is used for: The loss function is calculated based on the label data of the server layer output and the training data, and the gradient information of the output layer is generated based on the loss function.

6. The system of claim 1, wherein, The pseudo-training data is similar to the training data collected by the server based on the structure of the server layer.

7. A method for creating adversarial examples based on split learning, applied to a server of a split learning system, wherein a shadow input layer is further deployed on the server, and the method includes: During the neural network model training phase: During forward propagation, the input layer output from the client is acquired, wherein the input layer output is generated by inputting training data into the input layer; the client has the input layer and output layer of the neural network model deployed on it. Pseudo-training data is input into the shadow input layer to obtain the shadow input layer output; Calculate the similarity loss, which is used to characterize the similarity between the output of the input layer and the output of the shadow input layer; During backpropagation, the server-layer gradient information and the similarity gradient information calculated based on the similarity loss are computed, and the fused gradient information, which combines the server-layer gradient information and the similarity gradient information, is returned to the client; and In the inference stage of the neural network model: adversarial examples are generated using an adversarial example creation network consisting of the shadow input layer and the server layer; The server is configured to: during forward propagation, use the input layer output as the input of the server layer to generate the server layer output, and provide the server layer output to the client; and during backpropagation, acquire the output layer gradient information, and calculate the server layer gradient information based on the output layer gradient information, wherein the output layer gradient information is generated based on the server layer output and the label data of the training data.

8. The method of claim 7, wherein, Creating adversarial examples using an adversarial example creation network consisting of the shadow input layer and the server layer includes: Natural samples and perturbed samples with added perturbations are fed into an adversarial sample creation network consisting of the shadow input layer and the server layer. Obtain intermediate computation results of the natural samples and the perturbation samples output by the adversarial sample generation network; and The adversarial sample is generated based on the intermediate calculation results of the natural sample and the intermediate calculation results of the perturbation sample.

9. A computing device, comprising: processor; And a memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in claim 7 or 8.

10. A non-transitory machine-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method as described in claim 7 or 8.

Images