Federal learning method based on quantum transfer learning

Abstract

The invention discloses a federated learning method based on quantum transfer learning, relates to the cross technical field of quantum computing and federated learning, and aims to solve the problem of performance bottleneck of traditional federated learning under non-independent identically distributed data. The local model of each client is fused with a classic convolutional layer and a quantum convolutional layer; during local training, firstly, classic features are extracted by a classic convolutional layer, and then the classic features are coded and input into a quantum convolutional layer to generate quantum features; and the two features are spliced and then classified, and model parameters are uploaded. And the server side adopts a FedAvg algorithm to aggregate parameters so as to update the global model. According to the method, a transfer learning mechanism is introduced to optimize the initialization of the quantum layer, and an experimental result shows that the classification performance and robustness of the method are remarkably superior to those of a traditional federated CNN model on a non-IID image data set, and the method is particularly suitable for a privacy protection cooperative computing scene.

Description

Technical Field

[0001] This invention relates to the intersection of quantum computing and federated learning (FL), specifically to a quantum transfer learning federated method for improving image classification performance of non-independent identically distributed (non-IID) data in a distributed environment, which is particularly suitable for privacy-sensitive scenarios such as medical image diagnosis and financial risk control. Background Technology

[0002] Federated learning is an emerging distributed machine learning paradigm that allows multiple clients to collaboratively train a model without sharing local data, thus protecting data privacy. However, traditional federated learning models perform poorly when faced with non-independent and identically distributed (non-IID) data, which is prevalent in the real world, often exhibiting problems such as slow model convergence and decreased accuracy.

[0003] Meanwhile, quantum computing, with its quantum superposition and entanglement properties, can significantly improve computational parallelism and shows great potential in the field of machine learning. While existing quantum convolutional neural networks (QCNNs) have shown promise in image classification, they have failed to address the problem of insufficient model generalization ability caused by the heterogeneity of client-side data in federated learning scenarios.

[0004] Furthermore, while quantum transfer learning (QTL) enhances model adaptability through knowledge transfer, its integration with federated learning has not been systematically studied, particularly regarding how to effectively combine classical and quantum networks and address the challenges of heterogeneous environments in federated learning. Therefore, developing a novel technical solution that integrates quantum computing and federated learning and effectively addresses the challenges of non-IID data is of significant theoretical and practical importance. Summary of the Invention

[0005] The present invention provides a federated learning method based on quantum transfer learning, which integrates quantum convolutional layers into the classical convolutional neural network structure and adopts a transfer learning strategy to achieve efficient training and performance improvement of the model in a multi-client distributed environment.

[0006] To solve the above problems, the technical solution adopted by the present invention is as follows:

[0007] A federated learning method based on quantum transfer learning includes the following steps:

[0008] Step 1: Initialize a hybrid global model containing classical convolutional layers and quantum convolutional layers on the server side;

[0009] Step 2: Each client performs the following local training operations based on its local data: extract image features using classical convolutional layers to obtain classical flattened features; encode the classical flattened features into quantum states and input them into quantum convolutional layers; the quantum convolutional layers perform parameterized quantum gate operations on the quantum states and measure them to obtain quantum outputs; concatenate the quantum outputs with the classical flattened features and input them into fully connected layers for classification to update the local model parameters;

[0010] Step 3: Each client uploads the updated local model parameters to the server;

[0011] Step 4: The server aggregates all local model parameters uploaded by clients to update the hybrid global model.

[0012] Furthermore, in step 2, the method of encoding the classical flattened features into quantum states includes: encoding the i-th component x of the classical flattened feature vector... i The initial quantum state |0> is acted upon through a rotation gate RY around the Y-axis.

[0013] Furthermore, in step 2, the parameterized quantum gate operation includes the RY gate and the CR entanglement gate.

[0014] Furthermore, in step 2, the measurement is performed by measuring the Pauli-Z expectation value of each qubit.

[0015] Furthermore, in step 2, during the process of updating the local model parameters, the Adam optimizer is used for local optimization, and L2 regularization is introduced to reduce the risk of overfitting.

[0016] Furthermore, in step 4, the server uses the FedAvg algorithm to perform a weighted average of the local model parameters uploaded by each client in order to update the hybrid global model.

[0017] Furthermore, in the local training operation on the client side, a transfer learning mechanism is introduced to initialize the parameters of the quantum convolutional layer based on the weights already trained in the classical convolutional layer.

[0018] Furthermore, the classic convolutional layer is followed by a max pooling layer for dimensionality reduction of the image features.

[0019] Furthermore, the local data of each client consists of non-independent, identically distributed image data.

[0020] Compared with the prior art, the beneficial effects of the present invention are:

[0021] (1) By splicing and fusing the features extracted by the classical convolutional layer with the quantum features generated by the quantum convolutional layer, a more robust feature representation is achieved. Experimental results show that, regardless of whether it is under IID or non-IID data conditions, the model of this invention is significantly better than the traditional federated CNN model in terms of classification accuracy, F1-score and other indicators, especially showing stronger robustness in non-IID scenarios.

[0022] (2) Quantum computing is used to efficiently model high-dimensional data. The parameterized operation of quantum circuits is used to explore more complex feature spaces, which improves the overall expressive power of the model. The classical features and quantum features are integrated, enabling the model to capture data information from different levels, thereby achieving better generalization performance.

[0023] (3) Combining the idea of ​​transfer learning, the quantum convolutional layer is initialized with the weights of the classic convolutional model that have been trained before training on the client side. This provides a better starting point for training the model and avoids the training difficulties that may be caused by random initialization, thereby effectively improving the training efficiency and convergence speed of the model.

[0024] (4) Running under the federated learning framework, it naturally meets the requirements of data privacy protection. Its superior performance in processing heterogeneous data makes it particularly suitable for application scenarios with uneven data distribution and high privacy sensitivity, such as medical image diagnosis, financial risk control, and distributed intelligent monitoring, and has broad application prospects.

[0025] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, embodiments of the present invention are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of the training and parameter update process in the federated learning framework of the embodiment;

[0028] Figure 2 This is a flowchart illustrating the structure of FedQTL, a federated learning model based on quantum transfer learning, as shown in the example. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0030] This invention discloses a federated learning model based on quantum transfer learning (referred to as FedQTL in this paper) and the corresponding learning method. This embodiment is developed using the PyTorch and PennyLane platforms, and the federated learning process is simulated using the Flower framework.

[0031] Please refer to Figure 1 The document illustrates the overall flow of the federated learning framework of this invention, which involves a central server and multiple clients. The server is responsible for initializing and aggregating the global model, while the clients train the model using local data.

[0032] Please refer to Figure 2 It demonstrates the detailed structure and data flow of the FedQTL model of the present invention on the client side, which integrates classical convolution modules and quantum convolution modules.

[0033] The specific implementation process of the federated learning method based on quantum transfer learning in this invention is as follows:

[0034] 1. Initialize the global model

[0035] At the start of the federated learning process, the server initializes a hybrid global model containing classical convolutional layers (as is known to those skilled in the art, neural network layers that perform standard mathematical convolution operations on classical computers) and quantum convolutional layers. The initialized model is then distributed to all clients participating in the training.

[0036] 2. Local training on the client side

[0037] After receiving the hybrid global model, each client performs local training based on its local dataset (e.g., MNIST or CIFAR-10). The local training process specifically includes:

[0038] (1) Input layer: The dimension of the input image data is I∈R B×C×H×W Where B is the batch size, C is the number of color channels, and H and W represent the height and width of the data, respectively.

[0039] (2) Classical Convolutional Layer and Pooling Layer: The input image data first passes through a classical convolutional layer, which consists of K1 convolutional kernels of size 3×3, stride 1, and padding 1. After the convolution operation, the ReLU activation function is applied to obtain the feature map F1.

[0040] F1=ReLU(I*K1+b1) (1)

[0041] Where * represents the convolution operation, and b1 is the bias term.

[0042] Subsequently, feature map F1 is passed through a max pooling layer with a 2×2 pooling kernel and a stride of 2 for downsampling to extract the main features:

[0043] P1 = max(F1) (2)

[0044] The output of the pooling layer is flattened, forming a one-dimensional classic flattened feature x. flat :

[0045] x flat =Flatten(P1) (3)

[0046] (3) Quantum convolutional layer: Classical flattening feature x flat It is fed into a quantum convolutional layer for further processing. This process includes data encoding, parameterizing quantum circuits, and measurement.

[0047] Data encoding is the process of encoding a classic vector x. flat Encoding into quantum states. For example, the components of a vector can be encoded into the rotation angle of a qubit through a series of rotation gates (such as RY gates).

[0048] Parameterized quantum circuits are used to evolve encoded quantum states. These parameterized quantum circuits contain quantum gates with trainable parameters, such as RY rotation gates and CR entanglement gates, to achieve complex feature transformations.

[0049] Measurement refers to measuring the quantum state after the circuit has evolved, such as measuring the Pauli-Z expectation value of each qubit to obtain the quantum output vector. <z>.

[0050] (4) Feature assembly and classification:

[0051] Output vector of quantum convolution layer <z>Compared with the classical flattened feature x obtained in the previous steps flat Perform a concatenation operation to form a more expressive fused feature vector x. cat :

[0052] x cat =Concat(x) flat , <z>(4)

[0053] Concat represents the feature splicing and fusion operation.

[0054] Fusion feature vector x cat It is then fed into subsequent fully connected layers. For example, a fully connected layer will input x... cat The hidden representation F2 mapped to 64 dimensions:

[0055] F2=ReLU(P·W1+b2) (5)

[0056] Where W1 is the weight matrix and b2 is the bias term.

[0057] The final classification layer maps the hidden representation F2 to an n-dimensional output (where n is the number of classes in the dataset) to obtain the prediction result.

[0058]

[0059] Where W2 is the weight matrix and b3 is the bias term.

[0060] In local training, a transfer learning mechanism is introduced to optimize the training process. Specifically, the initial weights w of the quantum convolutional layer... initial It is based on the weights w of a pre-trained classic convolutional network (CNN). CNN This setting provides a better starting point for model training:

[0061] w initial =w CNN (7)

[0062] Quantum convolutional layers work in conjunction with classical convolutional layers through parameter sharing, enhancing the model's ability to handle complex data distributions.

[0063] In each round of local training, the Adam optimizer is used for local optimization, and L2 regularization can be introduced to reduce the risk of overfitting.

[0064] 3. Upload local model weights

[0065] After completing a preset number of rounds of local training (e.g., 20 rounds), the client will update the local model parameters. Uploaded to the server.

[0066] 4. Model aggregation

[0067] After receiving the local model parameters from each client, the server uses the FedAvg algorithm to perform a weighted average and aggregate them to generate a new generation of global model.

[0068]

[0069] This updated global model will be distributed to the client in the next round of communication, and the above process will be repeated cyclically.

[0070] Experimental Results and Analysis:

[0071] The performance of this invention was validated through experiments on the MNIST and CIFAR-10 datasets, simulating scenarios with both independent and identically distributed (IID) and non-independent and identically distributed (non-IID) data. Five clients were used in the experiments. Evaluation metrics included accuracy, F1-score, precision, and recall.

[0072] Table 1 summarizes the highest performance metrics of the FedQTL model and the traditional FedCNN model (FedCNN) under IID conditions.

[0073] Table 1. Performance comparison of FedQTL model and FedCNN model under IID conditions.

[0074]

[0075] Table 2 summarizes the performance of the two models under the more challenging non-IID condition.

[0076] Table 2 Performance comparison of FedQTL model and FedCNN model under non-IID conditions

[0077]

[0078] As can be seen from the data in Tables 1 and 2, the FedQTL model of this invention has a significant advantage in all evaluation metrics under both IID and non-IID conditions.

[0079] Accuracy Analysis: On the MNIST dataset, FedQTL significantly outperforms FedCNN under both IID and non-IID conditions, achieving 100% accuracy, especially under the IID condition. On the CIFAR-10 dataset, FedQTL still performs best under the IID condition, indicating its good generalization ability to complex data. Under the non-IID condition, FedCNN's accuracy drops significantly, while FedQTL remains relatively stable.

[0080] Comprehensive metric comparison: In the MNIST IID scenario, FedQTL's F1-score, Precision, and Recall are all 1.000; FedCNN's F1-score is 0.859. Under non-IID conditions, FedCNN's F1-score drops to 0.437, while FedQTL remains at 0.502, demonstrating stronger robustness against data heterogeneity. In the CIFAR-10 dataset, FedQTL's F1-score is 0.959 under IID conditions, significantly outperforming FedCNN's 0.436.

[0081] In summary, the FedQTL of this invention outperforms the classic federated CNN model in terms of classification accuracy, robustness, and convergence efficiency, indicating that this invention is particularly suitable for real-world application scenarios with varying data distributions. It has broad application prospects, especially in fields requiring data privacy protection, such as medical image analysis, distributed intelligent monitoring, and privacy-preserving collaborative computing.

[0082] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.< / z> < / z> < / z>

Claims

1. A federated learning method based on quantum transfer learning, characterized in that, Includes the following steps: Step 1: Initialize a hybrid global model containing classical convolutional layers and quantum convolutional layers on the server side; Step 2: Each client performs the following local training operations based on its local data: extract image features using classical convolutional layers to obtain classical flattened features; encode the classical flattened features into quantum states and input them into quantum convolutional layers; the quantum convolutional layers perform parameterized quantum gate operations on the quantum states and measure them to obtain quantum outputs; concatenate the quantum outputs with the classical flattened features and input them into fully connected layers for classification to update the local model parameters; Step 3: Each client uploads the updated local model parameters to the server; Step 4: The server aggregates all local model parameters uploaded by clients to update the hybrid global model.

2. The federated learning method based on quantum transfer learning according to claim 1, characterized in that, In step 2, the method of encoding classical flattened features into quantum states includes: encoding the i-th component x of the classical flattened feature vector. i The initial quantum state |0> is acted upon through a rotation gate RY around the Y-axis.

3. The federated learning method based on quantum transfer learning according to claim 1, characterized in that, In step 2, the parameterized quantum gate operation includes the RY gate and the CR entanglement gate.

4. The federated learning method based on quantum transfer learning according to claim 1, characterized in that, In step 2, the measurement method is to measure the Pauli-Z expectation value of each qubit.

5. The federated learning method based on quantum transfer learning according to claim 1, characterized in that, In step 2, during the process of updating the local model parameters, the Adam optimizer is used for local optimization, and L2 regularization is introduced to reduce the risk of overfitting.

6. The federated learning method based on quantum transfer learning according to claim 1, characterized in that, In step 4, the server uses the FedAvg algorithm to perform a weighted average of the local model parameters uploaded by each client in order to update the hybrid global model.

7. The federated learning method based on quantum transfer learning according to claim 1, characterized in that, In the local training operation on the client side, a transfer learning mechanism is introduced to initialize the parameters of the quantum convolutional layer based on the weights that have been trained in the classical convolutional layer.

8. The federated learning method based on quantum transfer learning according to claim 1, characterized in that, The classic convolutional layer is followed by a max pooling layer, which is used to reduce the dimensionality of the image features.

9. The federated learning method based on quantum transfer learning according to claim 1, characterized in that, The local data on each client consists of non-independent, identically distributed image data.

Images