Image classification visual watermarking method based on federal learning

Abstract

The invention discloses an image classification visual watermarking method based on federated learning, which is characterized in that a specific visual watermark identifier of a client is generated by constructing a transposition model shared with image classification model parameters, and watermark embedding and image classification task performance are jointly optimized by adopting a comparative learning mechanism. And the watermark is embedded while the accuracy of the image classification task is ensured. According to the method, the watermark extraction vector can be converted into the human-recognizable watermark image through the transpose model, and whether the suspicious model comes from the specific client is judged based on the generated watermark image, so that an intuitive and reliable verification method is provided for intellectual property protection of the federated learning model.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence security and intellectual property protection technology, specifically to a visual watermarking method for embedding and verifying ownership of model contributions for multiple clients within a federated learning framework. Background Technology

[0002] Image classification is a core task in computer vision, aiming to automatically identify and label the categories of major content in an image using algorithmic models. With breakthroughs in deep learning technology, image classification models based on deep neural networks have achieved remarkable accuracy and are widely used in critical scenarios such as medical image diagnosis, autonomous driving environmental perception, and industrial quality inspection. The powerful performance of these models heavily relies on large amounts of high-quality, diverse training data. However, in practical applications, factors such as data privacy, security regulations, and trade secrets make it increasingly difficult to use datasets scattered across various institutions for model training.

[0003] To address this contradiction, federated learning emerged. Federated learning is a distributed machine learning framework that allows multiple participants (called clients) to train models independently on their local private data, only uploading model updates to a central server for aggregation, thereby collaboratively training a powerful global model. This approach effectively avoids the direct sharing of raw data, providing a feasible path for training high-performance image classification models using distributed data while protecting privacy. Under the federated learning paradigm, the final global image classification model is the culmination of the collaborative efforts of all participants, containing significant commercial value and intellectual property. However, this collaborative model also introduces new intellectual property risks. The central server, as the manager of the training process, has complete control over the global model. A dishonest or malicious server might deliberately weaken or eliminate the contribution traces of a specific client by tampering with model parameters, or even openly claim exclusive ownership of the final model. Such infringement not only harms the legitimate rights and interests of the clients but also undermines the fairness and sustainability of the federated learning ecosystem. Summary of the Invention

[0004] This invention aims to overcome the shortcomings of existing technologies and proposes a visual watermarking method for image classification based on federated learning. It seeks to provide each client participating in the training of the federated image classification model with a highly discriminative, secure, and intuitively verifiable ownership proof mechanism, thereby effectively protecting their intellectual property rights.

[0005] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: The present invention discloses an image classification visualization watermarking method based on federated learning, characterized by its applicability to a system consisting of a central server and... Client In the federated learning system that comprises, Indicates the first For a single client, the image classification and visualization watermarking method is performed according to the following steps: Step 1: Define the total number of global training communication rounds for the federated learning system as follows: The number of local training rounds on the client side is Initialize the current communication round t =1, the central server initializes its own first... t- Parameters of the global image classification model in one communication round And distribute it to all clients; Step 2: Define the first Client The local dataset is ,in, express The first in j Image sample, express The true label, , Indicates the total number of categories. Indicates the first Local datasets for each client The total number of image samples in the data; the first Client Specify its own watermark image and random seed ; Step 3: Client Through its own random seed Generate the first indivual dimensional watermark extraction vector and based on Generate the first An enhanced set of vectors ,in, express The first in k One augmented vector sample, Indicates the first Client The total number of augmented vector samples; the first Client Based on cosine similarity threshold The first An enhanced set of vectors Divided into the first a set of positive sample vectors and the negative sample vector set ; Step 4: Client Receive parameters from the global image classification model and initialize its own first t Local image classification model under each communication round parameters ,in, e Indicates the local training round index; Step 5: Client According to the t Local image classification model under each communication round Build a structure containing parameters The t Circular transpose model ; Step 6: Client Will Input to local image classification model The predicted category is then processed to obtain the predicted category, which, together with the true category label, establishes the first... t Cross-entropy loss for image classification task in each communication round is used to train the local image classification model. and obtained the first t The first communication round After each local training round, parameters are included. Local image classification model ; No. Client Will Input to transpose model The process is performed to obtain the reconstructed watermark image. ,based on and The reconstructed watermark image With watermark image Together, we will build the first t The contrastive loss of the watermarking task in each communication round is used to train the transposed model. and obtained the first t The first communication round After each local training round, parameters are included. transpose model Statistics of batch normalized layers; Step 7: Client The first t Parameters trained in each communication round Uploaded to the central server, the first Each client stores its watermark extraction vector locally. Watermarked images and training with parameters transpose model Statistics of the normalized layer in the middle batch; Step 8, the central server uses equation (7) to calculate the first... t Parameters of the global image classification model in each communication round : (7) In equation (7), For the first Client The weights; Step 9: The central server will... t Parameters of the global image classification model in each communication round After being distributed to all clients, t +1 is assigned to t ,like t ≤ T If the result is positive, return to step 4 and execute sequentially; otherwise, it indicates the end of federated training and the final parameters are obtained. Watermark embedding in a global image classification model; Step 10: According to and structural similarity threshold , No. Client Verify whether the pirated image classification model infringes on your copyright.

[0006] The image classification secure aggregation method in a multi-layer federated learning environment described in this invention is also characterized in that step 3 includes: Step 3.1: The Client Based on random seed Generate the first using a hash function Watermark extraction vector ; Step 3.2: The Client The first is generated by perturbation with Gaussian noise. The number of vector samples is Enhanced vector set , among which, the An enhanced vector sample σ To control noise intensity, Indicates a Gaussian distribution; Step 3.3: Calculate using equation (1) and cosine similarity : (1) Step 3.4: If > Then Classified to the a set of positive sample vectors Otherwise, Classified to the negative sample vector set middle.

[0007] Furthermore, step 5 includes: Step 5.1: The Client Traversal The type and key parameters of each layer in the code are defined, and the data is transposed according to the transpose rules. Each layer in the process is transformed to obtain the transformed layer; Step 5.2: According to... By reversing the connection methods of each layer, the transformed layers are assembled to form a transposed model. .

[0008] Furthermore, the transpose rule in step 5.1 includes: like If the current layer type is a linear layer, then set the linear layer to a transposed linear layer, and set the calculation method of the transposed linear layer to... ,in, W This represents the weight matrix of the linear layer. This represents the bias vector of the linear layer. This represents the input of a linear layer. This represents the output of a linear layer; like If the current layer is a standard convolutional layer, then the standard convolutional layer is set as a transposed convolutional layer, and the kernel parameters of the transposed convolutional layer are the same as those of the standard convolutional layer. At the same time, the stride and padding parameters are set according to the shape of the input data of the standard convolutional layer. like If the current layer is a batch normalization layer, then the output of the batch normalization layer is calculated using equation (2). : (2) In equation (2), and This represents the two parameters to be learned in the batch normalization layer. This represents a constant that prevents division by zero. This indicates the calculation of the mean. This indicates variance calculation. This represents the input to the batch normalization layer; like If the current layer in the model is a pooling layer, then the stride and padding parameters of the previous transposed convolutional layer at the current layer position are modified according to the shape of the input data of the pooling layer, while the convolutional kernel parameters remain unchanged.

[0009] Furthermore, step 6 includes: Step 6.1: Define and initialize the first t Local training round index under each communication round e =1; No. Client According to the received number t- Parameters of the global image classification model aggregated in one communication round Initialize the local image classification model In the e Parameters under -1 round of local training Transpose model In the e Parameters under -1 round of local training ; Step 6.2: The Client Will Input with Local image classification model The forward propagation is performed to obtain the first... e Predicted categories under local training Therefore, the first equation (3) is used to construct the second equation. e Cross-entropy loss under round-local training : (3) Step 6.3: The Client Will Input with transpose model The forward propagation is performed to obtain the reconstructed watermark image. Thus, the first equation (4) is used to construct the second equation. e Comparative loss of watermarking task under local training : (4) In equation (4), express The first in One augmented vector sample The corresponding reconstructed watermark image, where SSIM represents the structural similarity function; and This represents two conditional indicator functions; the function takes a value of 1 when the corresponding condition is met, and a value of 0 otherwise. This represents the maximum structural similarity boundary of negative samples; Step 6.4: Construct the first step using equation (5) e Joint losses under local training : (5) In equation (5), λ represents the weight of the loss; Step 6.5: Construct the first step using equation (6). e Parameters under local training Update formula: (6) In equation (6), Indicates the first e Parameters under -1 round of local training This represents the loss function with respect to all learnable parameters. The gradient; Indicates the local learning rate; Step 6.6: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require e+ 1 Assigned to e Then, return to step 6.2 and execute sequentially to process the joint loss. Perform backpropagation until... So far, we have obtained the first... Local image classification model trained in rounds and transpose model parameters and transpose model Statistics of the normalized layer in the middle batch.

[0010] Furthermore, step 10 includes: Step 10.1: Obtain the first Client The parameters of the pirated image classification model that is believed to infringe on the copyright of the local image classification model. ; Step 10.2: The third-party validator will transfer the data with parameters according to the transposition rules. The pirated image classification model is reconstructed, and the first... Client The statistics of the batch normalized layer are obtained, thus yielding parameters. The transpose model; Step 10.3: Place the first Client Watermark extraction vector Input with parameters The reconstructed watermark image is obtained by processing the transposed model. ; Step 10.4: Calculation and Structural similarity between ,like If the result is positive, the verification is successful; the image classification model is identified as pirated and infringes on the copyright of the original image. Client Ownership must be verified; otherwise, verification fails, and the pirated image classification model is classified as non-pirated. This represents the structural similarity threshold.

[0011] The present invention provides an electronic device, including a memory and a processor, characterized in that the memory is used to store a program that supports the processor in executing the image classification method, and the processor is configured to execute the program stored in the memory.

[0012] The present invention provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when run by a processor, executes the steps of the image classification method.

[0013] Compared with existing technologies, the beneficial effects of this invention are reflected in: 1. This invention introduces a transposed model that shares parameters with the main task model, and reverses the image classification task into an image generation task by designing transposition rules. Through joint optimization, the embedding of a visual watermark and intuitive verification are achieved while ensuring the model's classification accuracy. 2. This invention generates watermark extraction vectors with low similarity between each client by using hash encoding, ensuring that the watermarks are embedded in different regions of the model feature space, thereby achieving natural differentiation of watermarks and avoiding conflicts after aggregation. 3. This invention introduces contrastive learning into the watermark training process, enabling the model to effectively distinguish between legitimate watermark keys and illegitimate forged keys, thereby significantly improving the watermark's ability to resist reverse engineering. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the process of the present invention; Figure 2 The figure shows the experimental results of the performance of the present invention under different attack scenarios. Detailed Implementation

[0015] In this embodiment, a visual watermarking method for image classification based on federated learning can provide each client participating in federated learning with credible ownership proof of its model contribution. For example... Figure 1 As shown, this method is applied to systems consisting of a central server and... Client In the federated learning system that comprises, Indicates the first One client, and follow these steps: Step 1: Define the total number of global training communication rounds for the federated learning system as follows: The number of local training rounds on the client side is Initialize the current communication round t =1, the central server initializes its own first... t- Parameters of the global image classification model in one communication round And it is distributed to all clients; in this embodiment, T Take 80, Take 3, total number of clients N Take 10. This embodiment uses the CIFAR-10 dataset to train and evaluate the VGG13 model. The CIFAR-10 dataset consists of 60,000 images in 10 categories, with an image size of 32. 32, each category contains 5000 training examples and 1000 test examples.

[0016] Step 2: Define the first Client The local dataset is ,in, express The first in j Image sample, express The true label, , Indicates the total number of categories. Indicates the first Local datasets for each client The total number of image samples in the data; the first Client Specify its own watermark image and random seed In this embodiment, to simulate the problem of data imbalance among clients in real-world scenarios, the following approach is adopted: A Dirichlet distribution with a resolution of 0.8 was used to divide 50,000 training images into Non-IID data and distribute them to 10 clients; the watermark image size was the same as the input image for the image classification task, and the watermark image size was 32. 32; The random seed is a string arbitrarily selected by the client.

[0017] Step 3: Client Through its own random seed Generate the first indivual dimensional watermark extraction vector and based on Generate the first An enhanced set of vectors ,in, express The first in k An enhanced vector sample, Indicates the first Client The total number of augmented vector samples; the first Client Based on cosine similarity threshold The first An enhanced set of vectors Divided into the first a set of positive sample vectors and the negative sample vector set ; Step 3.1: The Client Based on random seed Generate the first using a hash function Watermark extraction vector The hash function used in this example is SHA-512; the dimension of the watermark extraction vector is the same as the number of categories in the image classification task, so in this example the dimension of the watermark extraction vector is 10.

[0018] Step 3.2: The Client The first is generated by perturbation with Gaussian noise. The number of vector samples is Enhanced vector set , among which, the An enhanced vector sample σ To control noise intensity, This represents a Gaussian distribution; in this embodiment, σ takes values ​​from the set [0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.3, 0.5, 0.7, 0.9, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0]. Ten perturbation vectors are generated for each noise intensity. The size of the generated vector set is 220.

[0019] Step 3.3: Calculate using equation (1) and cosine similarity : (1) Step 3.4: If > Then Classified to the a set of positive sample vectors Otherwise, Classified to the negative sample vector set In this embodiment... We set the value to 0.95. The purpose of dividing the sample into negative samples is to simulate potential illegal extraction vectors, thereby improving the security of the scheme.

[0020] Step 4: Client Receive parameters from the global image classification model and initialize its own first t Local image classification model under each communication round parameters ,in, e Indicates the local training round index; Step 5: Client According to the t Local image classification model under each communication round Build a structure containing parameters The t Circular transpose model ; Step 5.1: The Client Traversal The type and key parameters of each layer in the code are defined, and the data is transposed according to the transpose rules. Each layer in the process is transformed to obtain the transformed layer. In this embodiment, the types of layers include: linear layers, convolutional layers, batch normalization layers, etc., and the key parameters include input and output dimensions, convolutional kernel size, stride, etc.

[0021] Step 5.1.1: Define the transpose rules, including: like The current layer is a linear layer, and its calculation is as follows: Then, the linear layer is set as the transposed linear layer, and the calculation method of the transposed linear layer is set as the inverse operation. ,in, W This indicates that the weight matrix of the linear layer is reused. This represents the bias vector of the linear layer. This represents the input of a linear layer. It represents the output of the linear layer; however, the operational logic is reversed.

[0022] like If the current layer is a standard convolutional layer, then the standard convolutional layer is set as a transposed convolutional layer, and the kernel parameters of the transposed convolutional layer are the same as those of the standard convolutional layer. At the same time, the stride and padding parameters are set according to the shape of the input data of the standard convolutional layer to achieve the inverse transformation of spatial size.

[0023] like If the current layer is a batch normalization layer, then the batch normalization layer (BN) will directly reuse the learnable parameters of the original BN layer. and However, the runtime statistics (mean, variance) are maintained independently by the watermarking task, and the output of the batch normalization layer is calculated using equation (2). : (2) In equation (2), and This represents the two parameters to be learned in the BN layer. This represents a small constant to prevent division by zero. This indicates the calculation of the mean. This indicates variance calculation. This represents the input to the BN layer.

[0024] like If the current layer in the transposed model is a pooling layer, then the stride and padding parameters of the previous transposed convolutional layer at the current layer position are modified according to the shape of the input data of the pooling layer, while the convolutional kernel parameters remain unchanged. In this example, the transposition of the pooling layer approximates the upsampling effect by adjusting parameters such as the stride and padding of the transposed convolutional layer, thereby reversing the spatial downsampling brought about by pooling.

[0025] Step 5.2: According to... By reversing the connection methods of each layer, the transformed layers are assembled to form a transposed model. .

[0026] Step 6: Client Will Input to local image classification model The predicted category is then processed to obtain the predicted category, which, together with the true category label, establishes the first... t Cross-entropy loss for image classification task in each communication round is used to train the local image classification model. and obtained the firstt The first communication round After each local training round, parameters are included. Local image classification model ; No. Client Will Input to transpose model The process is performed to obtain the reconstructed watermark image. ,based on and The reconstructed watermark image With watermark image Together, we will build the first t The contrastive loss of the watermarking task in each communication round is used to train the transposed model. and obtained the first t The first communication round After each local training round, parameters are included. transpose model Statistics of batch normalized layers.

[0027] Step 6.1: Define and initialize the first t Local training round index under each communication round e =1; No. Client According to the received number t- Parameters of the global image classification model aggregated in one communication round Initialize the local image classification model In the e Parameters under -1 round of local training Transpose model In the e Parameters under -1 round of local training .

[0028] Step 6.2: The Client Will Input with Local image classification model Perform forward propagation to obtain the predicted category under the e-th round of local training. Therefore, the cross-entropy loss under the e-th round of local training is constructed using equation (3). : (3) Step 6.3: The Client Will Input with transpose model The forward propagation is performed to obtain the reconstructed watermark image. Thus, the contrast loss of the watermarking task under the local training of the e-th round is constructed using equation (4). : (4) In equation (4), express The first in One augmented vector sample The corresponding reconstructed watermark image, where SSIM represents the structural similarity function; and This represents a conditional indicator function; the value is 1 if the corresponding condition is met, and 0 otherwise. This represents the maximum structural similarity boundary of negative samples. In this embodiment, contrastive learning is introduced into the watermark training process, forcing the model to learn the feature differences between positive samples (close to the real watermark extraction vector) and negative samples (potentially illegal vectors). This makes the model's output space highly non-convex for forgery attacks. When attackers perform reverse engineering to forge watermarks, they are likely to fall into the negative sample region during initialization, making it difficult to break through the barrier between positive and negative samples during training. Therefore, it is extremely difficult for attackers to successfully reverse engineer a visually credible watermark.

[0029] Construct the first using equation (5) e Joint losses under local training : (5) In equation (5), λ represents the weight of the loss; Step 6.5: Construct the first step using equation (6). e Parameters under local training Update formula: (6) In equation (6), Indicates the first e Parameters under -1 round of local training This represents the loss function with respect to all learnable parameters. The gradient; This represents the local learning rate; in this embodiment... Using 0.001, the SGD optimizer is applied because the two models share parameters. This backpropagation process considers the impact of both image classification and watermarking tasks on parameter updates, and the gradient incorporates the learning signals from both tasks.

[0030] Step 6.6: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require e+ 1 Assigned toe Then, return to step 6.2 and execute sequentially to process the joint loss. Perform backpropagation until... So far, we have obtained the first... Local image classification model trained in rounds and transpose model parameters and transpose model Statistics of the batch normalization layer. In this embodiment, since the two tasks are completely different, sharing the statistical information of the batch normalization layer would greatly disturb the image classification task and even cause training failure. Therefore, we calculate the statistical information of each task in the two models separately, which is beneficial to the stability of the training process. Note: The statistical information of the batch normalization layer is a non-learning parameter.

[0031] Step 7: Client The first t Parameters trained in each communication round Uploaded to the central server, number Each client stores its watermark extraction vector locally. Watermarked images and training with parameters transpose model Statistics of the normalized layer in the middle batch; Step 8, the central server uses equation (7) to calculate the first... t Parameters of the global image classification model in each communication round : In equation (7), For the first Client The weights are determined by the aggregation method used in this embodiment, which is FedAvg.

[0032] Step 9: The central server will... t Parameters of the global image classification model in each communication round After being distributed to all clients, t +1 is assigned to t ,like t ≤ T If the result is positive, return to step 4 and execute sequentially; otherwise, it indicates the end of federated training and the final parameters are obtained. Watermark embedding in a global image classification model; Step 10: According to and structural similarity threshold , No. Client Verify whether the pirated image classification model infringes on its own ownership. In this embodiment, the first... Client The copyright holder of the model.

[0033] Step 10.1: Obtain the first Client The parameters of the pirated image classification model that is believed to infringe on the copyright of the local image classification model. ; Step 10.2: The third-party validator will transfer the data with parameters according to the transposition rules. The pirated image classification model is reconstructed, and the first... Client The statistics of the batch normalized layer are obtained, thus yielding parameters. The transpose model; Step 10.3: Place the first Client Watermark extraction vector Input with parameters The reconstructed watermark image is obtained by processing the transposed model. ; Step 10.4: Calculation and Structural similarity between ,like If the result is positive, the verification is successful; the image classification model is identified as pirated and infringes on the copyright of the original image. Client Ownership must be verified; otherwise, verification fails, and the pirated image classification model is classified as non-pirated. This represents the structural similarity threshold. In this embodiment... The threshold is 0.7; the higher the threshold, the more difficult it is for the attack to succeed. As shown in Table 1, attackers can reverse engineer the actual extraction vector even when the client watermark is known and the actual watermark is unknown. The success rate is 0.

[0034] Table 1. Experimental results of attack success rate of the present invention in two reverse engineering scenarios. In this embodiment, Figure 2This study demonstrates the robustness of the invention against model modification attacks (including pruning, fine-tuning, and quantization). When the pruning rate is below 0.6, the model accuracy and SSIM are almost unaffected; however, when the pruning rate reaches 0.8, both drop sharply, indicating that the embedded watermark can resist moderate parameter reduction but is sensitive to extreme sparsity. During fine-tuning over 30 training epochs, accuracy and SSIM only fluctuate slightly, proving that the watermark remains stable during model re-optimization and does not hinder model convergence. The watermark exhibits robustness to low-precision deployment. Once the bit width exceeds 4 bits, both accuracy and SSIM remain at high levels, indicating that the watermark information is effectively preserved even after the model parameters are represented with low precision.

[0035] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program for the image classification method described above, and the processor runs the program stored in the memory.

[0036] In this embodiment, a readable storage medium is used to store computer program instructions, which are read by a processor and executed to perform the steps of the image classification method described above.

[0037] In summary, the method of this invention meets several key requirements: 1. It can clearly distinguish the contributions of different clients and avoid ownership confusion; 2. It has sufficient security to resist various attacks from malicious parties, especially reverse engineering; 3. The ownership proof process should be clear and intuitive, easy for humans to understand and arbitrate.

Claims

1. A federated learning based image classification visual watermarking method, characterized in that, Applied to by a central server and Client In the federated learning system that comprises, Indicates the first For a single client, the image classification and visualization watermarking method is performed according to the following steps: Step 1: define the total communication rounds of global training of the federated learning system as and the local training rounds of the client as , initialize the current communication round t =1, the center server initializes the parameters of the global image classification model in the first t- communication round and distributes them to all clients; ​ Step 2: Define the first Client The local dataset is ,in, express The first in j Image sample, express The true label, , Indicates the total number of categories. Indicates the first Local datasets for each client The total number of image samples in the data; the first Client Specify its own watermark image and random seed ; Step 3: the first client generates the first watermark extraction vector of the dimension based on the random seed of itself, and generates the first enhanced vector set based on the first watermark extraction vector , wherein represents the first k enhanced vector sample in , and represents the total number of enhanced vector samples of the first client ; the first client divides the first enhanced vector set into the first positive sample vector set and the first negative sample vector set according to the cosine similarity threshold value. Step 4: Client Receive parameters from the global image classification model and initialize its own first t Local image classification model in each communication round parameters ,in, e Indicates the local training round index; Step 5: Client According to the t Local image classification model in each communication round Build a structure containing parameters The t Circular transpose model ; Step 6: Client Will Input to local image classification model The predicted category is then processed to obtain the predicted category, which, together with the true category label, establishes the first... t Cross-entropy loss for image classification task in each communication round is used to train the local image classification model. and obtained the first t The first communication round After each local training round, parameters are included. Local image classification model ; No. Client Will Input to transpose model The process is performed to obtain the reconstructed watermark image. ,based on and The reconstructed watermark image With watermark image Together, we will build the first t The contrastive loss of the watermarking task in each communication round is used to train the transposed model. and obtained the first t The first communication round After each local training round, parameters are included. transpose model Statistics of batch normalized layers; Step 7: Client The first t Parameters trained in each communication round Uploaded to the central server, the first Each client stores its watermark extraction vector locally. Watermarked images and training with parameters transpose model Statistics of the normalized layer in the middle batch; Step 8, the center server calculates the parameters of the global image classification model under the i-th communication round by using formula (7) t : θi= θi-1- η∑j=1n∇L(θi-1; xj, yi) ​ (7) In formula (7), is the weight of the th client . Step 9: The central server will... t Parameters of the global image classification model in each communication round After being distributed to all clients, t +1 is assigned to t ,like t ≤ T If the result is positive, return to step 4 and execute sequentially; otherwise, it indicates the end of federated training and the final parameters are obtained. Watermark embedding in a global image classification model; Step 10: According to and structural similarity threshold , No. Client Verify whether the pirated image classification model infringes on your copyright.

2. The image classification secure aggregation method in a multi-layer federated learning environment according to claim 1, characterized in that, The step 3 comprises: Step 3.1: The Client Based on random seed Generate the first using a hash function Watermark extraction vector ; Step 3.2: The Client The first is generated by perturbation with Gaussian noise. The number of vector samples is Enhanced vector set , among which, the An enhanced vector sample σ To control noise intensity, Indicates a Gaussian distribution; Step 3.3: Calculate using formula (1) with the cosine similarity of : (1) Step 3.4: If > Then Classified to the a set of positive sample vectors Otherwise, Classified to the negative sample vector set middle.

3. The image classification secure aggregation method in a multi-layer federated learning environment according to claim 1, characterized in that, The step 5 comprises: Step 5.1: The first client traverses the type of each layer in the first model and its key parameters, and converts each layer in the first model according to the transpose rule to obtain each converted layer. Step 5.2: The second client traverses the type of each layer in the second model and its key parameters, and converts each layer in the second model according to the transpose rule to obtain each converted layer.​​​ Step 5.2: Assemble each converted layer in the reverse order of the way the layers were connected in the original model to form a transposed model .​ 4. The image classification secure aggregation method in a multi-layer federated learning environment according to claim 3, characterized in that, The transposition rule in the step 5.1 comprises: like If the current layer type is a linear layer, then set the linear layer to a transposed linear layer, and set the calculation method of the transposed linear layer to... ,in, W This represents the weight matrix of the linear layer. This represents the bias vector of the linear layer. This represents the input of a linear layer. This represents the output of a linear layer; like If the current layer is a standard convolutional layer, then the standard convolutional layer is set as a transposed convolutional layer, and the kernel parameters of the transposed convolutional layer are the same as those of the standard convolutional layer. At the same time, the stride and padding parameters are set according to the shape of the input data of the standard convolutional layer. like If the current layer is a batch normalization layer, then the output of the batch normalization layer is calculated using equation (2). : (2) In formula (2), and denote two parameters to be learned of the batch normalization layer, denotes a constant to prevent division by zero, denotes mean calculation, denotes variance calculation, denotes input of the batch normalization layer; like If the current layer in the model is a pooling layer, then the stride and padding parameters of the previous transposed convolutional layer at the current layer position are modified according to the shape of the input data of the pooling layer, while the convolutional kernel parameters remain unchanged.

5. The federated learning based image classification visual watermarking scheme of claim 1, wherein, The step 6 comprises: Step 6.1: Define and initialize the first t Local training round index under each communication round e =1; The first client According to the received first t- 1 communication round of the global image classification model aggregated parameters , initialize the local image classification model In the first e -1 round of local training parameters , transpose model In the first e -1 round of local training parameters ; Step 6.2: the first client inputs the local image classification model with the local image classification model to perform forward propagation to obtain the predicted class under the first round of local training e ; thereby constructing the cross-entropy loss under the first round of local training e using formula (3) : (3) Step 6.3: The Client Will Input with transpose model The forward propagation is performed to obtain the reconstructed watermark image. Thus, the first equation (4) is used to construct the second equation. e Comparative loss of watermarking task under local training : (4) In formula (4), denotes the first enhancement vector sample corresponding to the reconstructed watermark image, SSIM denotes a structural similarity function; and denote two conditional indicator functions, which take the value 1 when the corresponding condition is met, and otherwise take the value 0, denotes the maximum structural similarity boundary of the negative sample; Step 6.4: Constructing the first e Joint loss under local training : (5) In the formula (5), λ represents the weight of loss; Step 6.5: Constructing the first e parameters under local training update formula: (6) In formula (6), denotes the first e -1 round of local training under the parameters, denotes the gradient of the loss function with respect to all learnable parameters ; denotes the local learning rate; Step 6.6: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] e+ 1 Assigned to e Then, return to step 6.2 and execute sequentially to process the joint loss. Perform backpropagation until... So far, we have obtained the first... Local image classification model trained in rounds and transpose model parameters and transpose model Statistics of the normalized layer in the middle batch.

6. The image classification secure aggregation method in a multi-layer federated learning environment according to claim 1, characterized in that, The step 10 comprises: Step 10.1: Obtain the first client parameters of a pirated image classification model that is considered to infringe the copyright of the local image classification model ; Step 10.2: The third-party validator will transfer the data with parameters according to the transposition rules. The pirated image classification model is reconstructed, and the first... Client The statistics of the batch normalized layer are obtained, thus yielding parameters. The transpose model; Step 10.3: The watermark extraction vector of the first client is input into the transposed model with parameters to get the reconstructed watermark image. ;​​​​ Step 10.4: Calculation and Structural similarity between ,like If the result is positive, the verification is successful; the image classification model is identified as pirated and infringes on the copyright of the original image. Client Ownership must be verified; otherwise, verification fails, and the pirated image classification model is classified as non-pirated. This represents the structural similarity threshold.

7. An electronic device comprising a memory and a processor, characterized in that The memory is configured to store a program supporting the processor to execute the image classification method in any one of claims 1-5, and the processor is configured to execute the program stored in the memory.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is configured to execute the steps of the image classification method in any one of claims 1-5 when the computer program is run by the processor.

Images