Quantum federal collaborative distillation method and system, electronic equipment and storage medium

Abstract

The invention discloses a quantum federal collaborative distillation method, which comprises the following steps of: coordinating and serially connecting a whole federal learning process through a federal server, selecting clients participating in training in each round through a non-put-back sub-sampling method to realize first privacy amplification of privacy, providing a public non-label data set and finishing aggregation of soft labels; the obfuscator anonymously obfuscates the received information through a shuffling method, so that the federated server cannot associate any piece of information with a sender of the federated server, and second privacy amplification of differential privacy is realized under the condition that collusion with the federated server is avoided; and a client performs quantum model training on the local data set, predicts the public label-free data set, adds noise, encrypts and uploads the public label-free data set, so that knowledge distillation is realized, and third privacy amplification of differential privacy is realized by smoothing the same cluster sample. The invention also provides a quantum federal collaborative distillation system, electronic equipment and a storage medium.

Description

Technical Field

[0001] The present invention relates to the field of data privacy protection technology, and in particular to a quantum federated collaborative distillation method, system, electronic device and storage medium. Background Art

[0002] Federated learning is a new distributed machine learning paradigm that breaks through the data silos formed by data security, industry competition, and user privacy. Federated learning can integrate distributed data and use it to train machine learning models by only transmitting model parameters. Although only transmitting model parameters will not directly leak local data privacy, studies have shown that attackers can recover the training data of the model through model parameters (even model gradients). Therefore, federated learning requires additional privacy protection technologies to enhance protection capabilities.

[0003] Quantum Federated Learning (QFL) is a distributed machine learning framework that combines quantum computing and federated learning. In QFL, multiple clients (such as mobile devices, sensor networks, or edge computing nodes) jointly train a global quantum model while keeping data local. This approach allows the model to be updated without direct access to the original data, thereby achieving efficient model training while protecting user privacy.

[0004] However, in the prior art, the classic federated learning method is called federated averaging, which achieves information exchange by uploading model parameters or updates, but this will introduce a huge communication overhead. In addition, since the data owned by each participant is heterogeneous (usually non-independent and identically distributed), the performance of the classic federated averaging method will be greatly reduced under heterogeneous data. Another drawback of federated averaging is that when the computing power of the clients participating in the training is inconsistent, the clients cannot select a machine learning model that meets the computing power for training according to their own situation, which reduces the universality of the method. Summary of the invention

[0005] In view of the above problems, the present invention provides a quantum federal collaborative distillation method, system, electronic device and storage medium, in order to solve one of the above problems.

[0006] According to a first aspect of the present invention, a quantum federated collaborative distillation method is provided, comprising the following steps:

[0007] The federated server randomly samples M clients through the non-replacement subsampling method to obtain N clients participating in the federated training, achieving the first level of privacy amplification, where M and N are both positive integers, and N <M;

[0008] The federated server generates a public unlabeled data set, and sends the public unlabeled data set to the N clients participating in the federated training, and shares the quantum key with the clients in a quantum key distribution (QKD) manner; in this scheme, the federated server is used to coordinate and connect the entire federated learning process in series, select the clients participating in each round of training through the no-replacement subsampling method to achieve the first level of privacy amplification and provide the public unlabeled data set and complete the aggregation of soft labels;

[0009] Each client participating in the federated training uses its own local data set to train its own local quantum classifier, and obtains N trained local quantum classifiers;

[0010] Each trained local quantum classifier predicts the public unlabeled data set to obtain the prediction results, and encrypts the prediction results and uploads them to the obfuscator. In this solution, the obfuscator anonymizes and obfuscates the received information through the shuffling method, so that the federal server cannot associate any information with its sender, and the second privacy amplification of differential privacy is achieved without collusion with the federal server.

[0011] The obfuscator uses a shuffle method to anonymize and shuffle the prediction results from N clients participating in the federated training, and uploads the obtained N processing results to the federated server, wherein the shuffle method anonymizes the encrypted messages so that the attacker cannot associate any message with its sender, thereby achieving a second level of privacy amplification; the federated server decrypts and aggregates the N processing results to obtain soft labels, and transmits the soft labels to the clients participating in the next round of federated training, wherein the soft labels serve as labels of the public unlabeled data set; the above-mentioned federated server generates a public unlabeled data set, and sends the public unlabeled data set to the N clients participating in the federated training, and at the same time shares the quantum key with the clients in a quantum key distribution (QKD) manner, including:

[0012] The federated server determines the round of federated training and obtains the determination result;

[0013] The federated server generates keys and public unlabeled datasets;

[0014] The federated server shares the quantum key with the client in the form of quantum key distribution (QKD);

[0015] The federated server performs clustering processing on the public unlabeled data set to obtain a clustered unlabeled data set;

[0016] If the result is the first round of federated training, the key and the clustered unlabeled data set are sent to N clients participating in the federated training through the federated server;

[0017] If it is determined that the result is not the first round of federated training, the federated server sends the soft labels, keys and clustered unlabeled data sets of the previous round of federated training to N clients participating in the federated training. The clients perform quantum model training on the local data sets, make predictions on the public unlabeled data sets, and upload them with noise and encryption, thereby realizing knowledge distillation and achieving the third level of privacy amplification of differential privacy by smoothing samples of the same cluster.

[0018] Each client participating in the federated training uses its own local data set to train its own local quantum classifier, and obtains N trained local quantum classifiers including:

[0019] The clients participating in the federated training obtain the round information of the federated training through the federated server;

[0020] Each client participating in the federated training maintains and cools down a quantum device that can run a quantum classifier;

[0021] When the round information is the first round of federated training, the client participating in the federated training uses its own local data set as the training data set and uses it to train its own local quantum classifier to obtain a trained local quantum classifier;

[0022] When the round information is not the first round of federated training, the client participating in the federated training uses the soft label sent from the federated server in the previous federated training round as the label of the received public unlabeled dataset, adds the public dataset with the soft label to the local dataset, and uses the new local dataset to train the local quantum classifier to obtain a trained local quantum classifier.

[0023] According to an embodiment of the present invention, each trained local quantum classifier predicts a public unlabeled data set to obtain a prediction result, and encrypts the prediction result and uploads it to the obfuscator, including:

[0024] The client participating in the federated training uses its own trained local quantum classifier to predict the public unlabeled data set and obtain the prediction results;

[0025] The client participating in the federated training obtains the clustering results of the federated server, and performs smoothing and privacy processing on the prediction results based on the clustering results to obtain the processing results;

[0026] The client participating in the federated training encrypts the processing result with a key and uploads the encrypted prediction result to the obfuscator.

[0027] The client participating in the federated training obtains the clustering results of the federated server and performs smoothing and privacy processing on the prediction results based on the clustering results. The processing results include:

[0028] The client participating in the federated training obtains the clustering results from the federated server;

[0029] The clients participating in the federated training obtain the average value of the prediction results belonging to the same clustering result;

[0030] The client participating in the federated training adds Laplace noise to the average value and uses the noisy average value as the label of the prediction result to obtain the processing result.

[0031] The above-mentioned federal server decrypts and aggregates the N processing results to obtain soft labels including:

[0032] The federal server decrypts the N processing results to obtain N decrypted results;

[0033] According to a first preset threshold, the federal server performs a first screening on the N decryption results to obtain a screening result;

[0034] The federal server performs weighted averaging on the screening results to obtain a weighted average result;

[0035] According to a second preset threshold, the federal server performs a second screening on the weighted average result to obtain the soft label.

[0036] According to a second aspect of the present invention, a federated collaborative distillation system based on a differential privacy scrambling model is provided, comprising:

[0037] The sampling module is used by the federated server to randomly sample M clients through the no-replacement subsampling method to obtain N clients participating in the federated training, where M and N are both positive integers, and N <M;

[0038] A generation and distribution module, configured for a federated server to generate a public unlabeled data set, and send the public unlabeled data set to the N clients participating in the federated training, while sharing a quantum key with the clients in a quantum key distribution (QKD) manner;

[0039] The classifier training module is used for each client participating in the federated training to train its own local quantum classifier using its own local data set to obtain N trained local quantum classifiers;

[0040] The prediction processing module is used for each trained local quantum classifier to predict the public unlabeled data set, obtain the prediction results, and encrypt the prediction results and upload them to the obfuscator;

[0041] The privacy amplification module is used for the obfuscator to use the shuffle method to anonymize and shuffle the prediction results from N clients participating in the federated training, and upload the N processed results to the federated server. The shuffle method is used to achieve privacy amplification.

[0042] The soft label acquisition module is used for the federated server to decrypt and aggregate N processing results to obtain soft labels, and transmit the soft labels to the clients participating in the next round of federated training, where the soft labels serve as labels for the public unlabeled datasets.

[0043] According to a third aspect of the present invention, there is provided an electronic device, comprising:

[0044] processor;

[0045] a storage device for storing one or more programs,

[0046] Among them, when the program is executed by the processor, any of the above-mentioned quantum federal collaborative distillation methods can be implemented.

[0047] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which executable instructions are stored, and when the instructions are executed by a processor, any of the above-mentioned quantum federal collaborative distillation methods can be implemented.

[0048] A quantum computer device having a plurality of controllable quantum bits thereon, which can be used to execute any of the quantum federal collaborative distillation methods described above.

[0049] The quantum federated collaborative distillation method provided by the present invention obtains a quantum classifier model with good performance by training with distributed stored data while providing client-level differential privacy with practical protection significance; security is ensured by using key generation methods such as QKD; at the same time, knowledge distillation is realized by using unlabeled data as a carrier, which greatly reduces the communication overhead of federated learning, alleviates the performance loss caused by data heterogeneity, and enables models of different structures to be integrated and trained. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the specific embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 is a flow chart of a quantum federated collaborative distillation method according to an embodiment of the present invention;

[0052] Figure 2 is a flow chart of obtaining a key and a public unlabeled data set by a client participating in federated training according to an embodiment of the present invention;

[0053] Figure 3 is a flow chart of obtaining a clustered unlabeled data set according to an embodiment of the present invention;

[0054] Figure 4 is a flow chart of obtaining N trained local quantum classifiers according to an embodiment of the present invention;

[0055] Figure 5 is a flowchart of obtaining and uploading prediction results according to an embodiment of the present invention;

[0056] Figure 6 is a flowchart of obtaining processing results according to an embodiment of the present invention;

[0057] Figure 7 is a flowchart of obtaining a soft tag according to an embodiment of the present invention;

[0058] Figure 8 is a structural diagram of a quantum federation collaborative distillation system according to an embodiment of the present invention;

[0059] Fig. 9 is a diagram of a system training process of a quantum federated collaborative distillation method according to an embodiment of the present invention;

[0060] Fig.10 A block diagram of an electronic device of a quantum federated collaborative distillation method according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0061] The technical solutions in the specific embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] Differential privacy, as a privacy protection technology that can provide rigorous mathematical proofs and different levels of privacy protection capabilities, is often used in federated learning.

[0063] In the existing technology, differential privacy methods for federated learning can be mainly divided into two types: differential privacy scrambling model and centralized differential privacy.

[0064] Among them, the centralized differential privacy-based federated learning method assumes that there is a trusted server. After several rounds of training, the local client calculates the parameter update and then transmits the parameter update directly to the server. After the server receives the parameter updates from all participants in that round, it first clips the parameter updates according to the selected threshold and calculates the weighted average, and then adds Gaussian noise with a specified privacy budget to meet the definition of differential privacy. The server then sends the noisy parameter updates to the clients participating in the next round of training. Since the noise addition operation is performed on a trusted server, it is called centralized differential privacy. This model requires that the server must be honest and trustworthy, which is difficult to meet in real scenarios.

[0065] In addition, the federated learning method based on the differential privacy scrambling model does not make any security assumptions about other parties. After several rounds of training on the local data set, the local client calculates the parameter update, and then adds noise to the parameter update to satisfy differential privacy. The local client then sends the parameter update with noise to the server. After receiving all the parameter updates, the server obtains the parameter update for this round by weighted average and sends it to the clients participating in the next round of training. Since differential privacy processing is performed before the local client uploads the update, the server does not need to be trusted, and the security is higher than centralized differential privacy. However, compared with the centralized model, the error caused by the noise introduced by the differential privacy scrambling model is larger, which leads to poor data availability and affects the performance of the model.

[0066] At present, the main difficulties of existing technologies are: how to reduce the impact of noise introduced by the differential privacy scrambling model on the model, taking into account both method performance and privacy protection strength; the classic federated averaging method has a large performance loss under heterogeneous data, how to eliminate or reduce the performance loss caused by heterogeneous data through technical means; the federated averaging method cannot jointly train machine learning models of different structures, how to integrate models of different structures for training together.

[0067] In response to the above problems, the present invention provides a quantum federated collaborative distillation method, system, electronic system and storage medium. The method is based on a differential privacy scrambling model, and can improve the model performance loss caused by heterogeneous data while ensuring the differential privacy of distributed data. In addition, the method uses the public unlabeled samples provided by the server as knowledge carriers to achieve fusion training of different structural models. The above method provided by the present invention can be applied in sample classification, regression prediction, image recognition, gene analysis, natural language and other fields.

[0068] Figure 1 is a flow chart of a quantum federated collaborative distillation method according to an embodiment of the present invention.

[0069] like Figure 1As shown, the above method includes operations S110 to S160.

[0070] In operation S110, the federated server randomly samples M clients by a subsampling method without replacement to obtain N clients participating in the federated training, where M and N are both positive integers, and N <M。

[0071] In operation S120, the federation server generates a public unlabeled dataset, sends the public unlabeled dataset to N clients participating in the federation training, and shares the quantum key with the clients through QKD.

[0072] Each client participating in federated training receives a quantum key and a public unlabeled dataset.

[0073] In operation S130, each client participating in the federated training trains its own local quantum classifier using its own local data set to obtain N trained local quantum classifiers.

[0074] Each client participating in the federated training includes a local quantum classifier, and each client participating in the federated training has a local data set. After the above operations, each client participating in the federated training has a trained local quantum classifier.

[0075] In operation S140, each trained local quantum classifier predicts the public unlabeled data set to obtain a prediction result, and encrypts the prediction result and uploads it to the obfuscator.

[0076] In operation S150, the obfuscator uses a shuffle method to anonymize and shuffle the prediction results from N clients participating in the federated training, and uploads the obtained N processing results to the federated server, wherein the shuffle method is used to achieve privacy amplification.

[0077] The above shuffle method (or privacy blanket or obfuscation method) anonymizes the prediction results and then shuffles the order of the anonymized results. The federated server receives the shuffled and anonymized prediction results.

[0078] In operation S160, the federation server decrypts and aggregates the N processing results to obtain soft labels, and transmits the soft labels to the clients participating in the next round of federation training, wherein the soft labels serve as labels of the public unlabeled data sets.

[0079] The quantum federated collaborative distillation method provided by the present invention obtains a quantum classifier model with good performance by training with distributed stored data while providing client-level differential privacy with practical protection significance; security is ensured by using key generation methods such as QKD; at the same time, knowledge distillation is realized by using unlabeled data as a carrier, which greatly reduces the communication overhead of federated learning, alleviates the performance loss caused by data heterogeneity, and enables models of different structures to be integrated and trained.

[0080] Figure 2 This is a flowchart of a client participating in federated training obtaining a key and a public unlabeled dataset according to an embodiment of the present invention.

[0081] like Figure 2 As shown, the above-mentioned federation server generates a key and a public unlabeled dataset, and sends the key and the public unlabeled dataset to N clients participating in the federation training, including operations S210 to S250.

[0082] In operation S210, the federation server determines the round of federation training and obtains a determination result.

[0083] In operation S220, the federation server makes the unlabeled dataset public and shares the key through QKD.

[0084] In operation S230 , the federation server performs clustering processing on the public unlabeled data set to obtain a clustered unlabeled data set.

[0085] In operation S240 , when it is determined that the result is the first round of federated training, the key and the clustered unlabeled data set are sent to N clients participating in the federated training through the federated server.

[0086] In operation S250, when it is determined that the result is not the first round of federated training, the federated server sends the soft labels of the previous round of federated training and the clustered unlabeled data set to the N clients participating in the federated training and distributes the quantum keys.

[0087] During the above operation, the federated server module first selects N clients participating in this round of federated learning by subsampling without replacement. Then the federated server determines whether the current round is the first federated communication round (i.e., the first round of federated training, or the first federated round): if so, the federated server generates a public unlabeled data set and distributes quantum keys, clusters the public data set by a clustering method, and then sends the encryption key, clustering results, and public data set (Note: the clustering results and public data set are the clustered unlabeled data sets obtained in the above operation S230) to the selected client; if it is not the first federated round (i.e., the first round of federated training), the federated server sends the soft label obtained in the previous round to the selected client (i.e., the client participating in the federated training).

[0088] Figure 3 4 is a flow chart of obtaining a clustered unlabeled data set according to an embodiment of the present invention.

[0089] like Figure 3 As shown, the above-mentioned federal server performs clustering processing on the public unlabeled data set to obtain the clustered unlabeled data set, including operations S310 to S320.

[0090] In operation S310, the federation server uses a convolutional neural network to perform feature extraction on the public unlabeled data set to obtain an extraction result.

[0091] In operation S320 , the federation server clusters the extraction results using a K-means clustering method to obtain a clustered unlabeled data set.

[0092] The clustered unlabeled dataset has the same data type as the local dataset of the client participating in the federated training. The same data type can be data with the same data content, for example, both are pictures describing people, animals, plants or other abstract things. Since a quantum classifier is used, such data must first be encoded into a quantum state.

[0093] The method used to cluster public unlabeled datasets is to first extract data features using a convolutional neural network, and then cluster these features using K-means. Since the clustering results are not completely correct, the method selects a larger K value to reduce the impact of clustering errors on subsequent results. The server only needs to select the same type of unlabeled data as the local federation client to form a public dataset.

[0094] Figure 4 4 is a flow chart of obtaining N trained local quantum classifiers according to an embodiment of the present invention.

[0095] like Figure 4As shown, each client participating in the federated training trains its own local classifier using its own local data set to obtain N trained local quantum classifiers, including operations S410 to S430.

[0096] In operation S410, the client participating in the federated training obtains round information of the federated training through the federated server.

[0097] In operation S420, when the round information is the first round of federated training, the client participating in the federated training uses its own local data set as a training data set to train its own local quantum classifier to obtain a trained local quantum classifier.

[0098] In operation S430, when the round information is not the first round of federated training, the client participating in the federated training uses the soft label sent from the federated server in the previous federated training round as the label of the received public unlabeled data set, adds the public data set with the soft label to the local data set, and uses the new local data set to train the local quantum classifier to obtain a trained local quantum classifier.

[0099] Figure 5 is a flowchart of obtaining and uploading prediction results according to an embodiment of the present invention.

[0100] like Figure 5 As shown, each of the trained local quantum classifiers predicts the public unlabeled data set to obtain a prediction result, and encrypts the prediction result and uploads it to the obfuscator, including operations S510 to S530.

[0101] In operation S510, the client participating in the federated training uses its own trained local quantum classifier to predict the public unlabeled data set to obtain a prediction result.

[0102] In operation S520, the client participating in the federated training obtains the clustering result of the federated server, and performs smoothing and privacy processing on the prediction result according to the clustering result to obtain a processing result.

[0103] In operation S530, the client participating in the federated training encrypts the processing result using the key and uploads the encrypted prediction result to the obfuscator.

[0104] Figure 6 is a flowchart of obtaining processing results according to an embodiment of the present invention.

[0105] like Figure 6As shown, the client participating in the federated training obtains the clustering result of the federated server, and performs smoothing and privacy processing on the prediction result according to the clustering result, and the processing result includes operations S610 to S630.

[0106] In operation S610 , a client participating in federated training obtains a clustering result of a federated server.

[0107] In operation S620 , the clients participating in the federated training obtain an average value of prediction results belonging to the same clustering result.

[0108] In operation S630, the client participating in the federated training adds Laplace noise to the average value, and uses the average value with noise as a label of the prediction result to obtain a processing result.

[0109] In the above method, the client participating in the federated training first determines whether it is the first federated round (i.e., the first round of federated training): if so, the client participating in the federated training uses the local data set as the training set for this round; if not, the client participating in the federated training uses the received soft label as the label of the public data set and adds it to the local data set as the training set for this round. Subsequently, the N clients selected to participate in the federated training train their respective quantum classifier models locally at the same time, and then use their respective models to make predictions on the public data set, and smooth and privacy-process the predictions based on the clustering results. Finally, the prediction results are encrypted with a key and sent to the obfuscator. After the client participating in the federated training uses the model to make predictions on the public data set, in order to protect the privacy of the local data, the client averages the sample prediction labels of the same cluster based on the clustering results, and adds Laplace noise that meets the requirements. Sharing a noisy prediction label for the same cluster can reduce privacy consumption and achieve privacy amplification.

[0110] Figure 7 is a flow chart of obtaining a soft tag according to an embodiment of the present invention.

[0111] like Figure 7 As shown, the above-mentioned federal server decrypts and aggregates N processing results to obtain a soft label, including operations S710 to S740.

[0112] In operation S710, the federal server decrypts the N processing results to obtain N decrypted results.

[0113] In operation S720, according to a first preset threshold, the federal server performs a first screening on the N decryption results to obtain a screening result.

[0114] In operation S730 , the federation server performs weighted averaging on the screening results to obtain a weighted average result.

[0115] In operation S740, according to a second preset threshold, the federal server performs a second screening on the weighted average result to obtain a soft label.

[0116] After receiving all the messages, the federated server first screens the prediction results according to threshold T1 to remove labels with low availability. Then, it aggregates them by weighted average to obtain Y, and then screens Y for the second time according to threshold T2 to obtain the final prediction result of this round, which is used as the soft label of the public dataset to participate in the next round of federated training.

[0117] Figure 8 is a structural diagram of a federated collaborative distillation system according to an embodiment of the present invention.

[0118] like Figure 8 As shown, the system 800 includes a sampling module 810 , a generation module 820 , a classifier training module 830 , a prediction processing module 840 , a privacy amplification module 850 and a soft label acquisition module 860 .

[0119] Sampling module 810 is used for the federated server to randomly sample M clients by a subsampling method without replacement to obtain N clients participating in the federated training, where M and N are both positive integers, and N <M。

[0120] The generation and distribution module 820 is used for the federation server to generate a public unlabeled data set, send the key and the public unlabeled data set to N clients participating in the federation training, and distribute the quantum key through QKD.

[0121] The classifier training module 830 is used for each client participating in the federated training to train its own local quantum classifier using its own local data set to obtain N trained local quantum classifiers.

[0122] The prediction processing module 840 is used for each trained local quantum classifier to predict the public unlabeled data set, obtain the prediction result, and encrypt the prediction result and upload it to the obfuscator.

[0123] The privacy amplification module 850 is used for the obfuscator to use the shuffle method to anonymize and shuffle the prediction results from N clients participating in the federated training, and upload the obtained N processing results to the federated server, wherein the shuffle method is used to achieve privacy amplification.

[0124] The soft label acquisition module 860 is used for the federated server to decrypt and aggregate N processing results to obtain soft labels, and transmit the soft labels to the clients participating in the next round of federated training, wherein the soft labels serve as labels of the public unlabeled data sets.

[0125] Fig. 9 is a diagram of the system training process of the quantum federated collaborative distillation method according to an embodiment of the present invention.

[0126] Combine the following Fig. 9 The above method and system provided by the present invention are further described in detail.

[0127] like Fig. 9 As shown in the figure, the federated server module is used to coordinate and connect the entire federated learning process, and subsample the clients participating in each round of training to achieve the first level of privacy amplification, while providing unlabeled data sets and completing the aggregation of soft labels. The obfuscator module anonymizes and obfuscates the received information, so that the server cannot associate any information with its sender, and achieves the second level of differential privacy amplification without collusion with the server. The federated client module is the main undertaker of model training and is also the object of differential privacy protection. The client trains the model on the local data set, then makes predictions on the public data set and uploads it with noise encryption, thereby achieving knowledge distillation, and achieves the third level of differential privacy amplification by smoothing the same cluster samples.

[0128] Fig.10 A block diagram of an electronic device of a quantum federated collaborative distillation method according to an embodiment of the present invention is schematically shown.

[0129] like Fig.10 As shown, the electronic device 1000 according to an embodiment of the present invention includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0130] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0131] According to an embodiment of the present invention, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the I / O interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1008 as needed.

[0132] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0133] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 1002 and / or RAM 1003 described above and / or one or more memories other than ROM 1002 and RAM 1003.

[0134] The present invention also provides a quantum device, which may include the hardware / software components described in the above embodiments; or may exist independently without being integrated into other systems. The above quantum device contains one or more quantum programs, and when the above one or more quantum programs are executed, the operation method of the quantum federation collaborative distillation method according to the embodiment of the present invention can be implemented.

[0135] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All of these combinations and / or combinations fall within the scope of the present invention.

Claims

1. A quantum federated collaborative distillation method, characterized in that: The steps include: The federated server randomly samples M clients through the non-replacement subsampling method to obtain N clients participating in the federated training, achieving the first level of privacy amplification, where M and N are both positive integers, and N <M; The federated server generates a public unlabeled data set, and sends the public unlabeled data set to the N clients participating in the federated training, while sharing the quantum key with the clients in a quantum key distribution manner; Each client participating in the federated training trains its own local quantum classifier using its own local data set to obtain N trained local quantum classifiers; Each of the trained local quantum classifiers predicts the public unlabeled data set, combines the clustering results, unifies the prediction results of each cluster in the form of a majority, and performs noise perturbation and encryption on the prediction results, and then uploads them to the obfuscator; The obfuscator uses a shuffling method to anonymize and shuffle the prediction results from the N clients participating in the federated training, and uploads the obtained N processing results to the federated server; wherein the shuffling method anonymizes the encrypted messages so that an attacker cannot associate any message with its sender, thereby achieving a second level of privacy amplification; The federated server decrypts and aggregates the N processing results to obtain soft labels, and transmits the soft labels to the clients participating in the next round of federated training, wherein the soft labels serve as labels for the public unlabeled dataset. The clients perform quantum model training on the local dataset, make predictions on the public unlabeled dataset, and upload it with noise and encryption, thereby realizing knowledge distillation, and realizing the third level of privacy amplification of differential privacy by smoothing samples of the same cluster.

2. The quantum federal collaborative distillation method according to claim 1, characterized in that: The federation server generates a public unlabeled data set, and sends the public unlabeled data set to the N clients participating in the federation training, and at the same time shares the quantum key with the client in a quantum key distribution manner, including: The federal server determines the round of federal training and obtains a determination result; The federal server generates the public unlabeled dataset; The federal server shares the quantum key with the client in a quantum key distribution manner; The federal server performs clustering processing on the public unlabeled data set to obtain a clustered unlabeled data set; When the judgment result is the first round of federated training, sending the key and the clustered unlabeled data set to the N clients participating in the federated training through the federated server; When the judgment result is not the first round of federated training, the federated server sends the soft labels of the previous round of federated training, the key, and the clustered unlabeled data set to the N clients participating in the federated training.

3. The quantum federal collaborative distillation method according to claim 2, characterized in that: The federal server performs clustering processing on the public unlabeled data set, and the clustered unlabeled data set obtained includes: The federal server uses a convolutional neural network to perform feature extraction on the public unlabeled data set to obtain an extraction result; The federated server clusters the extraction results using a K-means clustering method to obtain a clustered unlabeled data set; The clustered unlabeled data set and the local data set of the client participating in the federated training have the same data type.

4. The quantum federal collaborative distillation method according to claim 1, characterized in that: Each client participating in the federated training uses its own local data set to train its own local quantum classifier to obtain N trained local quantum classifiers, including: The client participating in the federated training obtains the round information of the federated training through the federated server; The clients participating in the federated training each maintain and cool down a quantum device that can run a quantum classifier; In the case where the round information is the first round of federated training, the client participating in the federated training uses its own local data set as a training data set and uses it to train its own local quantum classifier to obtain a trained local quantum classifier; When the round information is not the first round of federated training, the client participating in the federated training uses the soft label sent from the federated server in the previous federated training round as the label of the received public unlabeled data set, adds the public data set with the soft label to the local data set, and trains the local quantum classifier using the new local data set to obtain a trained local quantum classifier.

5. The quantum federal collaborative distillation method according to claim 1, characterized in that: Each of the trained local quantum classifiers predicts the public unlabeled data set to obtain a prediction result, and encrypts the prediction result and uploads it to the obfuscator, including: The client participating in the federated training uses its own trained local quantum classifier to predict the public unlabeled data set to obtain a prediction result; The client participating in the federated training obtains the clustering result of the federated server, and performs smoothing and privacy processing on the prediction result according to the clustering result to obtain a processing result; The client participating in the federated training uses the key to encrypt the processing result and uploads the encrypted prediction result to the obfuscator.

6. The quantum federal collaborative distillation method according to claim 5, characterized in that: The client participating in the federated training obtains the clustering result of the federated server, and performs smoothing and privacy processing on the prediction result according to the clustering result, and the processing result obtained includes: The client participating in the federated training obtains the clustering result of the federated server; The clients participating in the federated training obtain an average value of the prediction results belonging to the same clustering result; The client participating in the federated training adds Laplace noise to the average value, and uses the average value with noise as the label of the prediction result to obtain a processing result.

7. The quantum federal collaborative distillation method according to claim 1, characterized in that: The federated server decrypts and aggregates the N processing results to obtain soft labels including: The federal server decrypts the N processing results to obtain N decrypted results; According to a first preset threshold, the federal server performs a first screening on the N decryption results to obtain a screening result; The federal server performs weighted averaging on the screening results to obtain a weighted average result; According to a second preset threshold, the federal server performs a second screening on the weighted average result to obtain the soft label.

8. A quantum federation collaborative distillation system, characterized in that: include: The sampling module is used by the federated server to randomly sample M clients through the no-replacement subsampling method to obtain N clients participating in the federated training, where M and N are both positive integers, and N <M; A generation and distribution module, used for the federated server to generate a public unlabeled data set, and send the public unlabeled data set to the N clients participating in the federated training, and share the quantum key with the clients in a quantum key distribution manner; A classifier training module, used for each client participating in the federated training to train its own local quantum classifier using its own local data set to obtain N trained local quantum classifiers; A prediction processing module, used for each trained local quantum classifier to predict the public unlabeled data set, obtain a prediction result, and encrypt the prediction result and upload it to the obfuscator; A privacy amplification module, used for the obfuscator to perform anonymization and random order processing on the prediction results from the N clients participating in the federated training using the shuffle method, and upload the obtained N processing results to the federated server, wherein the shuffle method is used to achieve privacy amplification; A soft label acquisition module is used for the federated server to decrypt and aggregate the N processing results to obtain soft labels, and transmit the soft labels to the clients participating in the next round of federated training, wherein the soft labels serve as labels for the public unlabeled data set.

9. An electronic device, characterized in that: include: processor; A storage device for storing a program, Wherein, when the program is executed by the processor, the quantum federated collaborative distillation method described in any one of claims 1 to 7 can be implemented.

10. A computer-readable storage medium, characterized in that: Executable instructions are stored thereon, and when the instructions are executed, the quantum federal collaborative distillation method described in any one of claims 1 to 7 can be implemented.

11. A quantum computer device, characterized in that: There are a number of controllable quantum bits thereon, which can be used to execute the quantum federal collaborative distillation method described in any one of claims 1 to 7.

Images