Federated learning modeling optimization methods, devices, readable storage media, and program products

Abstract

This application discloses a federated learning modeling optimization method, device, readable storage medium, and program product, applied to a first device. The federated learning modeling optimization method includes: acquiring first sample features generated by a feature extraction model and second sample features generated by a feature generation model to be trained; judging the first sample features and second sample features through a feature discrimination model to be trained, iteratively optimizing the feature generation model to be trained, and obtaining a feature generation model; sending the feature generation model and classification model to a second device, so that the second device can iteratively optimize a global feature generation model and a global classification model based on the feature generation models and classification models sent by each first device, so that the first device can construct a target feature extraction model and a target classification model based on the iteratively optimized global feature generation model and global feature classification model. This application solves the technical problem of the risk of leaking the data privacy of participating parties in federated learning methods.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology in financial technology (Fintech), and more particularly to a federated learning modeling optimization method, device, readable storage medium, and program product. Background Technology

[0002] With the continuous development of fintech, especially internet fintech, more and more technologies (such as distributed systems and artificial intelligence) are being applied in the financial field. However, the financial industry is also placing higher demands on technology, such as on the distribution of tasks to be completed.

[0003] With the continuous development of computer software and applications of artificial intelligence, big data, and cloud services, in federated learning scenarios, each participant currently maintains a local model and a global model. Each participant learns locally specific knowledge through its local model and shares knowledge across all participants through the global model. Then, each participant aggregates its local and global models. However, a drawback of this approach is that the federated server can infer the participants' original data from their global models, thus posing a risk of data leakage. In other words, existing federated learning methods carry the risk of compromising participant data privacy. Summary of the Invention

[0004] The main purpose of this application is to provide a federated learning modeling optimization method, device, readable storage medium, and program product, aiming to solve the technical problem that existing federated learning methods pose a risk of leaking the data privacy of participating parties.

[0005] To achieve the above objectives, this application provides a federated learning modeling optimization method, which is applied to a first device, and includes:

[0006] Obtain the trained feature extraction model and classification model, and extract the first training sample, the first noisy data, and the first true classification label corresponding to the first training sample;

[0007] Obtain the first sample features generated by the feature extraction model for the first training sample, and the second sample features generated by the feature generation model to be trained for the first noisy data and the first true classification label;

[0008] The first sample features and the second sample features are discriminated by a feature discrimination model to be trained. The feature generation model to be trained is iteratively optimized while the feature extraction model is fixed to obtain a feature generation model.

[0009] The feature generation model and the classification model are sent to the second device, so that the second device can iteratively optimize and aggregate the global classification model obtained by the first device based on the feature generation model sent by each of the first devices, and iteratively optimize and aggregate the global feature generation model obtained by the first device based on the classification model sent by each of the first devices, to obtain the target global feature generation model and the target global classification model.

[0010] The system receives the target global feature generation model and target global classification model fed back by the second device, and iteratively optimizes the feature extraction model and the classification model based on the target global feature generation model and the target global classification model to obtain the target feature extraction model and the target classification model.

[0011] This application provides a federated learning modeling optimization method, which is applied to a second device, and the federated learning modeling optimization method includes:

[0012] Receive feature generation models and classification models sent by each first device, and aggregate the feature generation models into a global feature generation model and aggregate the classification models into a global classification model;

[0013] Extract the noise data and the corresponding real classification labels;

[0014] Based on the noise data and the true classification label, the global classification model is iteratively optimized while keeping the feature generation models fixed, to obtain the target global classification model;

[0015] Based on the noise data and the true classification labels, the global feature generation model is iteratively optimized while keeping the classification models fixed, to obtain the target global feature generation model;

[0016] The target global classification model and the target global feature generation model are fed back to the first device, so that the first device can iteratively optimize the feature extraction model and the classification model corresponding to the feature generation model based on the target global feature generation model and the target global classification model, so as to obtain the target feature extraction model and the target classification model.

[0017] This application also provides a federated learning modeling optimization device, which is a virtual device and is applied to a first device. The federated learning modeling optimization device includes:

[0018] The extraction module is used to obtain the trained feature extraction model and classification model, and to extract the first training sample, the first noisy data, and the first true classification label corresponding to the first training sample.

[0019] The generation module is used to obtain the first sample features generated by the feature extraction model for the first training sample, and the second sample features generated by the feature generation model to be trained for the first noisy data and the first real classification label.

[0020] The first iterative optimization module is used to perform feature discrimination on the first sample features and the second sample features through the feature discrimination model to be trained, and to iteratively optimize the feature generation model to be trained while fixing the feature extraction model, so as to obtain the feature generation model.

[0021] The sending module is used to send the feature generation model and the classification model to the second device, so that the second device can iteratively optimize and aggregate the global classification model obtained by the first device based on the feature generation model sent by each of the first devices, and iteratively optimize and aggregate the global feature generation model obtained by the first device based on the classification model sent by each of the first devices, to obtain a target global feature generation model and a target global classification model.

[0022] The second iterative optimization module is used to receive the target global feature generation model and the target global classification model fed back by the second device, and iteratively optimize the feature extraction model and the classification model based on the target global feature generation model and the target global classification model to obtain the target feature extraction model and the target classification model.

[0023] This application also provides a federated learning modeling optimization device, which is a virtual device and is applied to a second device. The federated learning modeling optimization device includes:

[0024] An aggregation module is used to receive feature generation models and classification models sent by each first device, and to aggregate the feature generation models into a global feature generation model and the classification models into a global classification model.

[0025] The extraction module is used to extract noise data and the corresponding real classification labels of the noise data;

[0026] The first iterative optimization module is used to iteratively optimize the global classification model based on the noise data and the real classification label, while keeping the feature generation models fixed, to obtain the target global classification model.

[0027] The second iterative optimization module is used to iteratively optimize the global feature generation model based on the noise data and the real classification label, while keeping each of the classification models fixed, to obtain the target global feature generation model.

[0028] The feedback module is used to feed back the target global classification model and the target global feature generation model to the first device, so that the first device can iteratively optimize the feature extraction model and the classification model corresponding to the feature generation model based on the target global feature generation model and the target global classification model to obtain the target feature extraction model and the target classification model.

[0029] This application also provides a federated learning modeling optimization device, which is a physical device. The federated learning modeling optimization device includes: a memory, a processor, and a program of the federated learning modeling optimization method stored in the memory and executable on the processor. When the program of the federated learning modeling optimization method is executed by the processor, it can implement the steps of the federated learning modeling optimization method as described above.

[0030] This application also provides a readable storage medium storing a program that implements a federated learning modeling optimization method. When the program is executed by a processor, it implements the steps of the federated learning modeling optimization method as described above.

[0031] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the federated learning modeling optimization method described above.

[0032] This application provides a federated learning modeling optimization method, device, readable storage medium, and program product, which, compared to the prior art, involves each participant maintaining a local model and a global model in a federated learning scenario.Each participant learns local-specific knowledge through a local model, shares knowledge across all participants through a global model, and then aggregates the local and global models. This application first obtains a trained feature extraction model and a classification model, and extracts a first training sample, first noisy data, and the first true classification label corresponding to the first training sample. Then, it obtains the first sample features generated by the feature extraction model for the first training sample, and the second sample features generated by the feature generation model to be trained for the first noisy data and the first true classification label. Finally, it uses the feature discrimination model to be trained to perform feature discrimination on the first sample features and the second sample features. With the feature extraction model fixed, the training feature generation model is iteratively optimized to obtain the feature generation model. That is, by forming a generative adversarial network between the feature generation model and the feature discrimination model, the feature generation model can accurately convert the first noisy data and the first true classification label into sample features. This achieves the goal of constructing a feature generation model with the same output sample features as the feature extraction model. Simultaneously, the feature generation model is not directly constructed based on the original sample data of the participants. Then, the feature generation model and the classification model are sent to a second device, so that the second device can iteratively optimize and aggregate the full set of features obtained from each classification model based on the feature generation models sent by the first device. The global classification model allows it to learn knowledge from all federated participants. Based on the classification models sent by each of the first devices, it iteratively optimizes and aggregates the feature generation models to obtain a global feature generation model. This enables the global feature generation model to learn knowledge from all federated participants, achieving the goal of sharing knowledge from all federated participants. This results in a target global feature generation model and a target global classification model. Since the feature generation model is not directly constructed based on the participants' original sample data, the second device cannot deduce the original sample data from the first device. Therefore, the second device receives the target global feature generation model and target global classification model fed back by the second device and, based on the target global features... The generator model and the target global classification model are iteratively optimized, and the feature extraction model and the classification model are improved so that the feature extraction model and the classification model can learn the knowledge of all participants in the target global feature generation model and the target global classification model, thereby obtaining the target feature extraction model and the target classification model. This achieves the goal of enabling the local model to learn the model knowledge of the global model, that is, to share the knowledge of all participants through the global model, while not leaking the original sample data of the local model. Therefore, it overcomes the technical defect that the federated server can infer the original data of each participant through the global model of each participant, thus reducing the risk of leakage of the privacy data of the participants in federated learning. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating the first embodiment of the federated learning modeling optimization method of this application;

[0036] Figure 2 A schematic diagram illustrating the process of constructing a feature generation model for the first device in the federated learning modeling optimization method of this application;

[0037] Figure 3 A flowchart illustrating the process of constructing a target feature extraction model and a target classification model for the first device in the federated learning modeling optimization method of this application;

[0038] Figure 4 This is a flowchart illustrating the second embodiment of the federated learning modeling optimization method of this application;

[0039] Figure 5 A flowchart illustrating the process of constructing a target global feature extraction model and a target global classification model for the second device in the federated learning modeling optimization method of this application;

[0040] Figure 6 A flowchart illustrating the process of constructing a global feature classification model for the target in the federated learning modeling optimization method of this application;

[0041] Figure 7 A flowchart illustrating the process of constructing a global feature classification model for the target in the federated learning modeling optimization method of this application;

[0042] Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the federated learning modeling optimization method in the embodiments of this application.

[0043] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0044] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0045] This application provides a federated learning modeling optimization method applied to a first device. In the first embodiment of the federated learning modeling optimization method of this application, refer to... Figure 1 The federated learning modeling optimization method includes:

[0046] Step S10: Obtain the trained feature extraction model and classification model, and extract the first training sample, the first noise data, and the first true classification label corresponding to the first training sample;

[0047] In this embodiment, it should be noted that the federated learning modeling optimization method is applied to a federated learning scenario, which can be a horizontal federated learning scenario. The first device is a participant in the federated learning, the second device is a federated server for the federated learning, the feature extraction model and the classification model are local models trained iteratively on the first device, and the first noisy data is mixed with the first real classification label as input to the feature generation model to be trained. Thus, even if the second device reverse-engineers the privacy data of the participant based on the trained feature generation model, it will only obtain a mixture of the first noisy data and the first real classification label, and will not be able to obtain the first real classification label of the participant, thereby reducing the risk of leakage of the first real classification label.

[0048] Furthermore, the local iterative training process for the feature extraction model and the classification model is as follows:

[0049] Extract local training samples and their corresponding local classification labels. Then, pass the local training samples sequentially through the feature extraction model and the classification model to be trained to obtain the training output classification labels. Based on the training output classification labels and the local classification labels corresponding to the local training samples, calculate the model loss. If the model loss converges, use the feature extraction model and the classification model to be trained as the feature extraction model and the classification model to be trained as the classification model. If the model loss does not converge, update the feature extraction model and the classification model to be trained based on the model gradient calculated by the model loss, and return to the execution step: extract local training samples and their corresponding local classification labels.

[0050] Step S20: Obtain the first sample features generated by the feature extraction model for the first training sample, and the second sample features generated by the feature generation model to be trained for the first noisy data and the first real classification label;

[0051] In this embodiment, specifically, based on the feature extraction model, features are extracted from the first training sample to obtain the first sample features, and based on the feature generation model to be trained, the first noisy data and the first real classification label are converted into the second sample features.

[0052] Step S30: The first sample features and the second sample features are discriminated using the feature discrimination model to be trained. The feature generation model to be trained is iteratively optimized while the feature extraction model is fixed to obtain the feature generation model.

[0053] In this embodiment, specifically, based on the feature discrimination model to be trained, feature discrimination is performed on the first sample feature and the second sample feature respectively to determine whether the first sample feature and the second sample feature come from the feature generation model or the feature extraction model, thereby obtaining feature discrimination prediction labels. Then, based on the feature discrimination prediction labels and the feature discrimination true labels corresponding to the first sample feature and the second sample feature, the feature discrimination loss is calculated. Based on the feature discrimination loss, the feature generation model to be trained is iteratively updated while keeping the feature extraction model fixed, to obtain the feature generation model. Figure 2 The diagram shows a flowchart of the first device constructing a feature generation model in an embodiment of this application. In this flowchart, x represents the first training sample (i.e., data), z represents the first noise data (i.e., noise), y represents the first true classification label (i.e., label), f represents the first sample feature, and g represents the second sample feature. L is used to predict labels based on features. gan The feature discrimination loss is used.

[0054] The step of iteratively updating the feature generation model to be trained based on the feature discrimination loss, while keeping the feature extraction model fixed, to obtain the feature generation model includes:

[0055] Determine whether the feature discrimination loss has converged. If it has converged, use the feature generation model to be trained as the feature generation model. If it has not converged, keep the feature extraction model unchanged, and update the feature generation model and the feature discrimination model to be trained based on the model gradient calculated by the feature discrimination loss. Then return to the execution step: extract the first training sample, the first noise data, and the first true classification label corresponding to the first training sample to perform the next round of model training iteration.

[0056] The step of performing feature discrimination on the first sample features and the second sample features using a feature discrimination model to be trained, and iteratively optimizing the feature generation model to be trained while fixing the feature extraction model, to obtain the feature generation model includes:

[0057] Step S31: Perform feature discrimination on the first sample features and the second sample features using the feature discrimination model to be trained, and calculate the feature discrimination loss;

[0058] In this embodiment, it should be noted that the feature discrimination loss can be either cross-entropy loss or generative adversarial network loss.

[0059] Specifically, based on the feature discrimination model to be trained, feature discrimination is performed on the first sample feature and the second sample feature respectively to obtain the first feature discrimination prediction result corresponding to the first sample feature and the second feature discrimination prediction result corresponding to the second sample feature. Then, based on the first feature discrimination prediction result, the first feature discrimination true label corresponding to the first sample feature, the second feature discrimination prediction result and the second feature discrimination true label corresponding to the second sample feature, the feature discrimination loss is calculated.

[0060] Furthermore, if the feature discrimination loss is a generative adversarial network (GAN) loss, then the feature discrimination loss needs to be calculated based on a preset GAN loss function, which is as follows:

[0061]

[0062] Where V is the generative adversarial network loss, G represents the feature generation model to be trained, D represents the feature discrimination model to be trained, Q represents the feature extraction model, and x ~ P data Let z represent the first training sample, z represent the first noisy data and the first true classification label, Q(x) represent the first sample feature, and G(z) represent the second sample feature. The true label of the first feature is 1, and the true label of the second feature is 0. Then, the feature generation model and the feature discrimination model to be trained are optimized by the feature discrimination loss. The loss of the feature generation model to be trained can be maximized while the loss of the feature discrimination model to be trained is minimized. The first feature discrimination prediction result is the probability that the first sample feature is generated by the feature extraction model, and the second feature discrimination prediction result is the probability that the second sample feature is generated by the feature extraction model.

[0063] The step of performing feature discrimination on the first sample features and the second sample features using the feature discrimination model to be trained, and calculating the feature discrimination loss, includes:

[0064] Step S311: Based on the feature discrimination model to be trained, perform binary classification on the first sample features and the second sample features respectively to obtain the binary classification result;

[0065] In this embodiment, it should be noted that the feature discrimination model to be trained is a binary classification model, which is used to perform binary classification on sample features to determine whether the sample features come from the feature generation model or the feature extraction model, that is, to determine whether the sample features come from real samples or from mixed fictitious samples composed of noise and labels. For example, assuming that the real binary classification labels (positive and negative sample labels) are labels 0 and 1 respectively, where 0 indicates that the corresponding sample feature is the output of the feature generation model, and 1 indicates that the corresponding sample feature is the output of the feature extraction model, the output of the feature discrimination model can be represented as the probability value that the corresponding sample feature is the output of the feature extraction model.

[0066] Specifically, based on the feature discrimination model to be trained, the first sample feature and the second sample feature are classified into two categories to determine whether the first sample feature and the second sample feature come from the feature discrimination model to be trained or from the feature extraction model, thereby obtaining the first binary classification prediction label corresponding to the first sample feature and the second binary classification prediction label corresponding to the second sample feature. The binary classification result includes the first binary classification prediction label corresponding to the first sample feature and the second binary classification prediction label corresponding to the second sample feature.

[0067] Step S312: Calculate the feature discrimination loss based on the binary classification result and the positive and negative sample labels corresponding to the first sample feature and the second sample feature.

[0068] In this embodiment, it should be noted that the positive and negative sample labels include the positive sample label corresponding to the first sample feature and the negative sample label corresponding to the second sample feature.

[0069] Specifically, the feature discrimination loss is calculated based on the difference between the first binary classification prediction label and the positive sample label, and the difference between the second binary classification prediction label and the negative sample label.

[0070] Step S32: Based on the feature discrimination loss, while keeping the feature extraction model fixed, iteratively update the feature discrimination model to be trained and the feature generation model to be trained to optimize the feature generation model to be trained and obtain the feature generation model.

[0071] In this embodiment, specifically, it is determined whether the feature discrimination loss has converged; if it has converged, the feature generation model to be trained is used as the feature generation model; if it has not converged, the feature discrimination model to be trained and the feature generation model to be trained are updated according to the model gradient calculated by the feature discrimination loss, and the execution step is returned: extracting the first training sample, the first noise data, and the first true classification label corresponding to the first training sample.

[0072] Step S40: The feature generation model and the classification model are sent to the second device, so that the second device can iteratively optimize and aggregate the global classification model obtained by the first device based on the feature generation model sent by each of the first devices, and iteratively optimize and aggregate the global feature generation model obtained by the first device based on the classification model sent by each of the first devices, to obtain the target global feature generation model and the target global classification model.

[0073] In this embodiment, specifically, the feature generation model and the classification model are sent to the second device. The second device then aggregates the feature generation models sent by the first devices to obtain a global feature generation model, and aggregates the classification models sent by the first devices to obtain a global classification model. Then, with the feature generation models fixed, the outputs of each feature generation model are classified according to the global classification model, and a first global classification loss is calculated. Based on the first global classification loss, the global classification model is iteratively optimized to enable it to indirectly learn the knowledge of all first devices through all feature generation models, thereby obtaining a target global classification model. With the classification models fixed, the outputs of the global feature generation model are classified according to each classification model, and a second global classification loss is calculated. Then, based on the second global classification loss, the global feature generation model is iteratively optimized to enable the global feature generation model to indirectly learn the knowledge of all first devices through all classification models, thereby obtaining the target global feature generation model. The specific process of the second device constructing the target global feature generation model and the target global classification model can be referred to the specific content of steps D10 to D40 below, which will not be detailed here. Since the second device does not directly possess the feature extraction model, but possesses the feature generation models of all participants, and the input of the feature generation model is a mixed fictitious sample of noisy data and labels, even if the second device reverse-engineers based on the feature generation model, it can only obtain the labels mixed with noisy data, and cannot reverse-engineer the specific labels and specific samples possessed by the first device, thereby protecting the data privacy of the first device.

[0074] Furthermore, since, with each of the aforementioned feature generation models fixed, the outputs of each feature generation model are classified according to the global classification model, a first global classification loss is calculated, and then, based on the first global classification loss, the global classification model is iteratively optimized to obtain a target global classification model. Thus, the target global classification model is compatible with all feature generation models; that is, for the outputs of all feature generation models, the target global classification model can make accurate classification decisions. Therefore, the target global classification model carries the classification model knowledge of all the first devices. Since, with each of the aforementioned classification models fixed, the outputs of the global feature generation model are classified according to each classification model, a second global classification loss is calculated, and then, based on the second global classification loss, the global feature generation model is iteratively optimized to obtain a target global feature generation model. Thus, the target global feature generation model is compatible with all classification models; that is, for the outputs of the target global feature generation model, each classification model can make accurate classification decisions. Therefore, the target global feature generation model carries the feature generation model knowledge of all the first devices, thereby achieving the purpose of aggregating the feature generation model knowledge of all federated participants into the target global feature generation model, and the purpose of aggregating the classification model knowledge of all federated participants into the target global classification model.

[0075] Step S50: Receive the target global feature generation model and target global classification model fed back by the second device, and iteratively optimize the feature extraction model and the classification model based on the target global feature generation model and the target global classification model to obtain the target feature extraction model and the target classification model.

[0076] In this embodiment, specifically, the target global feature generation model and the target global classification model fed back by the second device are received. Based on the target global feature generation model and the target global classification model, the feature extraction model and the classification model are iteratively optimized to enable the feature extraction model to learn the model knowledge of the target global feature generation model and to enable the classification model to learn the model knowledge of the target global classification model, thereby obtaining the target feature extraction model and the target classification model.

[0077] Furthermore, since each classification model can make accurate classification decisions based on the output of the target global feature generation model, the target global feature generation model learns the output feature distribution (global sample feature distribution) of all feature extraction models. The first device iteratively optimizes the feature extraction model and classification model based on the target global classification model and the target global feature generation model. The first device's local feature extraction model can learn the global sample feature distribution, while its local classification model can learn the global classification model knowledge. Therefore, the goal of knowledge sharing among all federated participants is achieved, realizing federated learning modeling. Simultaneously, because the feature discrimination model ensures a certain difference between the output of the feature generation model and the output of the feature extraction model, even if the second device uses the feature generation model to reverse-engineer the first device's privacy data, it can only obtain a mixed constructed sample of noisy data and labels, rather than a training sample.

[0078] The step of iteratively optimizing the feature extraction model and the classification model based on the target global feature generation model and the target global classification model to obtain the target feature extraction model and the target classification model includes:

[0079] Step S51: Perform knowledge distillation between the classification model and the target global classification model to optimize the classification model and obtain the target classification model;

[0080] Step S52: Perform knowledge distillation between the feature extraction model and the target global feature generation model to optimize the feature extraction model and obtain the target feature extraction model.

[0081] In this embodiment, specifically, with the target global feature generation model fixed, knowledge distillation is performed between the classification model and the target global classification model to enable the classification model to learn the model knowledge of the target global classification model and obtain the target classification model. With the target global classification model fixed, knowledge distillation is performed between the feature extraction model and the target global feature generation model to enable the feature extraction model to learn the model knowledge of the target global feature generation model and obtain the target feature extraction model.

[0082] In addition, such as Figure 3 The diagram shows a flowchart illustrating the construction of a target feature extraction model and a target classification model by the first device in an embodiment of this application. In this flowchart, x represents the first training sample (i.e., data), z represents the first noise data (i.e., noise), y represents the first true classification label (i.e., label), f represents the first sample feature, and g represents the second sample feature. The output of the feature discrimination model, For the output of the classification model, L ganFor the global feature discrimination loss, L task The global classification loss is given.

[0083] Additionally, it should be noted that in order to protect data privacy in federated learning, federated learning can currently be performed in a homomorphic encryption environment. However, homomorphic encryption involves extremely high computational overhead, while the data involved in the first and second devices in this application embodiment is plaintext data. Therefore, compared with the method of federated learning based on homomorphic encryption, the computational overhead is significantly reduced, thereby improving the efficiency of federated learning. Furthermore, to protect data privacy in federated learning, multi-party secure computation (MPC) can currently be used. However, MPC involves significant computational and communication overhead. In this embodiment, the data involved in the first and second devices are all complete plaintext data and do not involve the secret sharing process in MPC. Therefore, compared to MPC-based federated learning, the computational and communication overhead is significantly reduced, improving the efficiency of federated learning. Additionally, to protect data privacy in federated learning, differential privacy can currently be used. However, differential privacy requires adding noise to achieve privacy protection, which can affect the usability and accuracy of the model. In this embodiment, noise is not directly added to the feature extraction and classification models; instead, noise data and labels are input together into the feature generation model to simulate the output of the feature extraction model. Therefore, the accuracy and usability of the feature extraction and classification models are not affected. Thus, compared to differential privacy-based federated learning, the usability and accuracy of the federated learning model are improved.

[0084] Furthermore, the federated learning modeling optimization method can be used in the field of image processing, and the federated learning modeling optimization method further includes:

[0085] Step Q10: Obtain the trained image feature extraction model and image classification model, and extract the first training image sample, the first image noise data, and the first real image classification label corresponding to the first training image sample;

[0086] Step Q20: Obtain the first image sample features generated by the image feature extraction model for the first training image sample, and the second image sample features generated by the image feature generation model to be trained for the first image noise data and the first real image classification label.

[0087] Step Q30: The first image sample features and the second image sample features are discriminated using the image feature discrimination model to be trained. The image feature extraction model is fixed, and the image feature generation model to be trained is iteratively optimized to obtain the image feature generation model.

[0088] Step Q40: The image feature generation model and the image classification model are sent to the second device, so that the second device can iteratively optimize and aggregate the global image classification model obtained by the first device based on the image feature generation model sent by each of the first devices, and iteratively optimize and aggregate the global image feature generation model obtained by the first device based on the image classification model sent by each of the first devices, to obtain the target global image feature generation model and the target global image classification model.

[0089] Step Q50: Receive the target global image feature generation model and target global image classification model fed back by the second device, and iteratively optimize the image feature extraction model and the image classification model based on the target global image feature generation model and the target global image classification model to obtain the target image feature extraction model and the target image classification model.

[0090] In this embodiment, it should be noted that the feature extraction model can be an image feature extraction model, the classification model can be an image classification model, the first training sample can be a first training image sample, the first noise data can be first image noise data, the first true classification label can be a first true image classification label, the feature generation model to be trained can be a model to be trained image feature generation, the second sample feature can be a second image sample feature, the feature discrimination model to be trained can be a model to be trained image feature discrimination, the feature generation model can be an image feature generation model, the target global feature generation model can be a target global image feature generation model, the target global classification model can be a target global image classification model, the target feature extraction model can be a target image feature extraction model, and the target classification model can be a target image classification model. The specific implementation process of steps Q10 to Q50 can be referred to the content of steps S10 to S50, and will not be repeated here.

[0091] This application provides a method for constructing an image feature extraction model and an image classification model based on federated learning. First, a trained image feature extraction model and an image classification model are obtained, and a first training image sample, first image noise data, and a first real image classification label corresponding to the first training image sample are extracted. Then, first image sample features generated by the image feature extraction model for the first training image sample, and second image sample features generated by the image feature generation model to be trained for the first image noise data and the first real image classification label are obtained. Next, the first image sample features and the second image sample features are discriminated using the image feature discrimination model to be trained. With the image feature extraction model fixed, the image feature generation model to be trained is iteratively optimized to obtain the image feature generation model. That is, by forming a generative adversarial network between the image feature generation model and the image feature discrimination model, the image feature generation model can accurately convert the first image noise data and the first real image classification label into image sample features, achieving the goal of constructing an image feature generation model with the same output image sample features as the image feature extraction model, while the image feature generation model is not directly constructed based on the original image sample data of the participants. The image feature generation model and the image classification model are then sent to the second device. The second device iteratively optimizes and aggregates the image feature generation models sent by the first device to obtain a global image classification model, and iteratively optimizes and aggregates the image feature generation models sent by the first device to obtain a global image feature generation model. This allows the global image classification model to learn knowledge from all federated parties, and iteratively optimizes and aggregates the image feature generation models sent by the first device to obtain a global image feature generation model, achieving the goal of sharing knowledge from all federated parties, thus obtaining a target global image feature generation model and a target global image classification model. The second device then receives the target global image feature generation model and the target global image classification model fed back by the second device, and iteratively optimizes the image feature extraction model and the image classification model based on these models. This allows the image feature extraction model and the image classification model to learn knowledge from all participants in the target global image feature generation model and the target global image classification model, thus obtaining a target image feature extraction model and a target image classification model. This approach achieves the goal of sharing knowledge among all participants through a global model without leaking local original sample data. In other words, while realizing the construction of image feature extraction and image classification models based on federated learning, it also reduces the risk of each participant leaking its own original image sample data.It solves the "data silo" problem when building image feature extraction and image classification models, and improves the accuracy of image feature extraction and image classification models while protecting the data privacy of the original image sample data of all parties.

[0092] Compared to homomorphic encryption, building image processing models (image feature generation and image classification models) using federated learning significantly reduces computational overhead and improves efficiency. Compared to multi-party secure computation, federated learning significantly reduces computational and communication overhead, improving efficiency. Compared to differential privacy, this embodiment does not directly add noise to the image feature extraction and classification models; it merely uses noise data and labels as input to the image feature generation model to simulate the output of the image feature extraction model. Therefore, it does not affect the accuracy and usability of the image feature extraction and classification models, thus improving the usability and accuracy of the image processing models built using federated learning.

[0093] This application provides a federated learning modeling optimization method, which, compared to the prior art, involves each participant maintaining a local model and a global model in a federated learning scenario.Each participant learns local knowledge through a local model, shares knowledge across all participants through a global model, and then aggregates the local and global models. In this embodiment, the first step is to obtain a trained feature extraction model and a classification model, and extract a first training sample, first noisy data, and a first true classification label corresponding to the first training sample. Then, the first sample feature generated by the feature extraction model for the first training sample, and the second sample feature generated by the feature generation model to be trained for the first noisy data and the first true classification label are obtained. Finally, the first sample feature and the second sample feature are used to perform feature discrimination by the feature discrimination model to be trained. Under the condition of fixing the feature extraction model, the training feature generation model is iteratively optimized to obtain the feature generation model. That is, by forming a generative adversarial network between the feature generation model and the feature discrimination model, the feature generation model can accurately convert the first noisy data and the first true classification label into sample features. This achieves the purpose of constructing a feature generation model with the same output sample features as the feature extraction model. At the same time, the feature generation model is not directly constructed based on the original sample data of the participants. Then, the feature generation model and the classification model are sent to the second device, so that the second device can iteratively optimize and aggregate the classification models based on the feature generation models sent by the first device to obtain the desired feature generation model. The global classification model allows it to learn knowledge from all federated participants. Based on the classification models sent by each of the first devices, it iteratively optimizes and aggregates the feature generation models to obtain a global feature generation model. This enables the global feature generation model to learn knowledge from all federated participants, achieving the goal of sharing knowledge from all federated participants. This results in a target global feature generation model and a target global classification model. Since the feature generation model is not directly constructed based on the participants' original sample data, the second device cannot deduce the original sample data from the first device. Therefore, the second device receives the target global feature generation model and target global classification model fed back by the second device and, based on the target global feature generation model... The feature generation model and the target global classification model are iteratively optimized to enable the feature extraction model and the classification model to learn the knowledge of all participants in the target global feature generation model and the target global classification model, thereby obtaining the target feature extraction model and the target classification model. This achieves the goal of enabling local models to learn the model knowledge of the global model, that is, to share the knowledge of all participants through the global model, while not leaking the original sample data of the local model. Therefore, it overcomes the technical defect that the federated server can infer the original data of each participant through the global model of each participant, thus reducing the risk of leakage of the privacy data of the participants in federated learning.

[0094] Furthermore, referring to Figure 4 Based on the first embodiment of this application, in another embodiment of this application, the step of iteratively optimizing the feature extraction model and the classification model according to the target global feature generation model and the target global classification model to obtain the target feature extraction model and the target classification model includes:

[0095] Step A10: Use the target global classification model as the new classification model and the target global feature generation model as the new feature generation model, and extract the second training sample, the second noise data, and the second true classification label corresponding to the second training sample.

[0096] In this embodiment, specifically, the classification model is directly replaced with the target global classification model to obtain a new classification model, and the feature generation model is directly replaced with the target global feature generation model to obtain a new feature generation model. The second training sample, the second noise data, and the second true classification label corresponding to the second training sample are extracted.

[0097] Step A20: Based on the feature extraction model, feature extraction is performed on the second training sample to obtain the third sample feature; and based on the new feature generation model, the second noise data and the second true classification label are converted into the fourth sample feature.

[0098] Step A30: Classify the third sample features using the new classification model, and perform feature discrimination on the third sample features and the fourth sample features using the feature discrimination model. Iteratively optimize the new classification model and the feature extraction model to obtain the target feature extraction model and the target classification model.

[0099] In this embodiment, specifically, the third sample features are classified according to the new classification model to calculate the global classification loss, and the third sample features and the fourth sample features are discriminated according to the feature discrimination model to calculate the global feature discrimination loss. Then, the new classification model is iteratively optimized based on the model gradient calculated by the global classification loss to obtain the target classification model. Finally, the feature extraction model is iteratively optimized based on the model gradient calculated by the global classification loss and the global feature discrimination loss to obtain the target feature extraction model.

[0100] The steps of classifying the third sample features using the new classification model, and discriminating the third sample features and the fourth sample features using the feature discrimination model, and iteratively optimizing the new classification model and the feature extraction model to obtain the target feature extraction model and the target classification model include:

[0101] Step A31: If the target global classification model and the target global feature generation model satisfy the preset federated iteration termination condition, then the third sample feature is classified by the new classification model, and the third sample feature and the fourth sample feature are discriminated by the feature discrimination model. The new classification model and the feature extraction model are iteratively optimized to obtain the target feature extraction model and the target classification model.

[0102] Step A32: If the target global classification model and the target global feature generation model do not meet the preset federated iteration termination condition, then the third sample feature is classified by the new classification model, and the third sample feature and the fourth sample feature are discriminated by the feature discrimination model. The new classification model and the feature extraction model are iteratively optimized, and the execution step is returned: extract the first training sample, the first noise data, and the first true classification label corresponding to the first training sample.

[0103] In this embodiment, specifically, if a federated learning modeling end notification is received from the second device, it is determined that the target global classification model and the target global feature generation model meet the preset federated iteration end condition. Then, the third sample feature is classified using the new classification model, and the third sample feature and the fourth sample feature are discriminated using the feature discrimination model. The new classification model and the feature extraction model are iteratively optimized to obtain the target feature extraction model and the target classification model. If a federated learning modeling end notification is not received from the second device, it is determined that the target global classification model and the target global feature generation model do not meet the preset federated iteration end condition. Then, the third sample feature is classified using the new classification model, and the third sample feature and the fourth sample feature are discriminated using the feature discrimination model. The new classification model and the feature extraction model are iteratively optimized, and the execution step is returned: extracting the first training sample, the first noise data, and the first true classification label corresponding to the first training sample for the next round of federated learning iteration.

[0104] The steps of classifying the third sample features using the new classification model, and discriminating the third sample features and the fourth sample features using the feature discrimination model, and iteratively optimizing the new classification model and the feature extraction model to obtain the target feature extraction model and the target classification model include:

[0105] Step C10: Classify the features of the third sample using the new classification model and calculate the global classification loss;

[0106] In this embodiment, specifically, the features of the third sample are classified according to the new classification model to obtain a classification result. Based on the difference between the classification result and the second true classification label, a global classification loss is calculated using a preset loss function. The preset loss function can be an L2 loss function or a cross-entropy loss function.

[0107] Step C20: Using the feature discrimination model, perform feature discrimination on the features of the third sample and the features of the fourth sample, and calculate the global feature discrimination loss;

[0108] In this embodiment, it should be noted that the global feature discrimination loss can be either cross-entropy loss or generative adversarial network loss. The specific calculation process of generative adversarial network loss can be referred to the specific content in step S31 above, and will not be repeated here.

[0109] Specifically, based on the feature discrimination model, feature discrimination is performed on the third sample feature and the fourth sample feature respectively to obtain the third feature discrimination prediction label corresponding to the third sample feature and the fourth feature discrimination prediction label corresponding to the fourth sample feature. Then, based on the difference between the third feature discrimination prediction label and the feature discrimination true label corresponding to the third sample feature, and the difference between the third feature discrimination prediction label and the feature discrimination true label corresponding to the fourth sample feature, the global feature discrimination loss is calculated.

[0110] Step C30: Iteratively update the feature extraction model, the new classification model, the new feature generation model, and the feature discrimination model using the global classification loss and the global feature discrimination loss to optimize the feature extraction model and the new classification model, thereby obtaining the target feature extraction model and the target classification model.

[0111] In this embodiment, specifically, it is determined whether the global classification loss and the global feature discrimination loss have both converged. If they have both converged and a federated learning termination notification is received, the feature extraction model is used as the target feature extraction model and the new classification model is used as the target classification model. If they have both converged and no federated learning termination notification is received, the new feature generation model and the new classification model are sent to the second device for the next round of federated learning modeling iteration. If they have not converged, the new classification model is updated based on the model gradient calculated by the global classification loss relative to the new classification model; the feature extraction model is updated based on the model gradient calculated by the global classification loss and the global feature discrimination loss together relative to the feature extraction model; the feature discrimination model is updated based on the model gradient calculated by the global feature discrimination loss relative to the feature discrimination model; the new feature generation model is updated based on the model gradient calculated by the feature discrimination model relative to the new feature generation model, and the execution step is returned: extracting the second training sample, the second noise data, and the second true classification label corresponding to the second training sample for the next round of model training iteration.

[0112] This application provides an optimization method for a feature extraction model and a classification model. Specifically, it first uses the target global classification model as a new classification model and the target global feature generation model as a new feature generation model, extracting a second training sample, second noisy data, and the second true classification label corresponding to the second training sample. Then, based on the feature extraction model, it extracts features from the second training sample to obtain a third sample feature. Based on the new feature generation model, it converts the second noisy data and the second true classification label into a fourth sample feature. The new classification model then classifies the third sample feature, and the feature discrimination model discriminates the third and fourth sample features. This iterative optimization of the new classification model and the feature extraction model enables the new classification model to adapt to the new feature generation and feature extraction models, and allows the feature extraction model to learn the model knowledge of the target global feature generation model. This achieves the goal of sharing the classification model knowledge and feature generation model knowledge of all federated participants, realizing the construction of feature extraction and classification models based on federated learning. Furthermore, it avoids directly aggregating feature extraction models for federated learning modeling, thus protecting the data privacy of federated participants.

[0113] Furthermore, referring to Figure 5 In another embodiment of this application, the federated learning modeling optimization method is applied to a second device, and the federated learning modeling optimization method includes:

[0114] Step D10: Receive the feature generation model and classification model sent by each first device, and aggregate the feature generation models into a global feature generation model and aggregate the classification models into a global classification model;

[0115] Step D20: Extract the noise data and the corresponding real classification labels;

[0116] In this embodiment, it should be noted that the aggregation methods include averaging and weighted summation. Each first device needs to maintain a feature generation model and a classification model. The feature generation model is a model that simulates the output features of the corresponding feature extraction model based on noise data and real classification labels. The output features of the feature extraction model are the model output results obtained by extracting features from samples corresponding to real classification labels. The output of the feature generation model and the output of the feature extraction model can be distinguished by a feature discrimination model. Furthermore, for the same sample and its corresponding noise and label, the output of the feature generation model and the output of the corresponding feature extraction model can generate the same classification label through the classification model. The specific process of the first device constructing the feature generation model can be referred to the specific content of steps S10 to S30 above, and will not be repeated here.

[0117] Step D30: Based on the noise data and the true classification label, the global classification model is iteratively optimized while keeping the feature generation models fixed, to obtain the target global classification model;

[0118] In this embodiment, specifically, based on each of the feature generation models, the real classification labels of the noisy data are converted into sample features to obtain local sample features output by each feature generation model. Then, the local predicted classification labels and the real classification labels are classified by the global classification model, and a first target loss is calculated. Based on the first target loss, the global classification model is iteratively optimized while keeping the feature generation models fixed to obtain a target global classification model. Iterative optimization of the global classification model while keeping the feature generation models fixed can make the global classification model adapt to each of the feature generation models, thereby indirectly enabling the global classification model to learn the classification model knowledge of all participants, thus achieving the goal of constructing a target global classification model with global classification model knowledge.

[0119] The step of iteratively optimizing the global classification model based on the noise data and the true classification label, while keeping the feature generation models fixed, to obtain the target global classification model includes:

[0120] Step D31: Based on each of the feature generation models, the noisy data and the real classification label are converted into corresponding local sample features.

[0121] Step D32: Based on the global classification model, classify the features of each local sample to obtain the predicted local classification labels.

[0122] Step D33: Calculate the first target loss based on the difference between each of the local predicted classification labels and the true classification labels;

[0123] Step D34: Based on the first target loss, the global classification model is iteratively optimized while keeping the feature generation models fixed, to obtain the target global classification model.

[0124] In this embodiment, specifically, based on each of the feature generation models, the noisy data and the true classification labels are converted into corresponding local sample features; based on the global classification model, each of the local sample features is classified to obtain each local predicted classification label; based on the difference between each local predicted classification label and the true classification label, the local classification loss corresponding to each local predicted classification label is calculated, and then the local classification losses are aggregated to obtain the first target loss, wherein the aggregation method includes averaging and weighted summation, etc.; then, based on the model gradient calculated by the first target loss, while keeping each of the feature generation models fixed, the global classification model is updated, and it is determined whether the global classification model has reached a preset number of model iterations. If it has, the global classification model is used as the target global classification model; if it has not, the execution step is returned: extracting the noisy data and the true classification labels corresponding to the noisy data.

[0125] In addition, such as Figure 6 The diagram shown is a flowchart of constructing a target global feature classification model in an embodiment of this application. In this flowchart, the client is the first device, the server is the second device, the noise is the noise data, the label is the real classification label, and the global feature classification model is a global classification model.

[0126] Step D40: Based on the noise data and the true classification label, the global feature generation model is iteratively optimized while keeping the classification models fixed to obtain the target global feature generation model;

[0127] In this embodiment, specifically, based on the global feature generation model, the noisy data and the true classification label are converted into sample features to obtain global sample features. Then, based on each classification model, the global sample features are classified to obtain each global predicted classification label and the true classification label. A second target loss is calculated. Then, based on the second target loss, the global feature generation model is iteratively optimized while keeping each classification model fixed to obtain a target global feature generation model.

[0128] The step of iteratively optimizing the global feature generation model based on the noise data and the true classification labels, while keeping the classification models fixed, to obtain the target global feature generation model includes:

[0129] Step D41: Based on the global feature generation model, the noisy data and the real classification label are converted into corresponding global generated features.

[0130] Step D42: Based on each of the classification models, classify the globally generated features to obtain the globally predicted classification labels corresponding to each classification model;

[0131] Step D43: Calculate the second target loss based on the difference between each of the globally predicted classification labels and the true classification labels;

[0132] Step D44: Based on the second target loss, the global feature generation model is iteratively optimized while keeping the classification models fixed, to obtain the target global feature generation model.

[0133] In this embodiment, specifically, based on the global feature generation model, the noisy data and the true classification label are jointly converted into corresponding global generated features; based on each classification model, the global generated features are classified to obtain the global predicted classification label corresponding to each classification model; based on the difference between each global predicted classification label and the true classification label, the global classification loss corresponding to each global predicted classification label is calculated, and then the global classification losses are aggregated to obtain a second target loss; then, based on the model gradient calculated by the second target loss, the global feature generation model is updated while fixing each classification model, and it is determined whether the global feature generation model has reached a preset number of model iterations. If it has, the global feature generation model is used as the target global feature generation model; if it has not, the execution step is returned to: extracting the noisy data and the true classification label corresponding to the noisy data.

[0134] In addition, such as Figure 7The diagram shown is a flowchart of constructing a target global feature classification model in an embodiment of this application, wherein the client is the first device, the server is the second device, the noise is the noise data, and the label is the real classification label.

[0135] Furthermore, it should be noted that since the sample data of each first device is usually not independent and identically distributed, the local models built by each first device based on the sample data usually have certain differences. If the local models are directly aggregated as the global model, it will usually lead to a decrease in the performance of the global model. However, in this embodiment, after aggregating the local models to obtain the global model, the global model is further iteratively optimized to make the global model more compatible with each local model, that is, to make the global classification model more compatible with each feature generation model, and to make the global feature generation model more compatible with each classification model. In other words, the global classification model and the global feature generation model can better learn the knowledge of all participants, thus improving the performance of the global classification model and the global feature generation model. The local model includes the feature generation model and the classification model.

[0136] Step D50: The target global classification model and the target global feature generation model are fed back to the first device, so that the first device can iteratively optimize the feature extraction model and the classification model corresponding to the feature generation model based on the target global feature generation model and the target global classification model, to obtain the target feature extraction model and the target classification model.

[0137] In this embodiment, it should be noted that the specific process by which the first device iteratively optimizes the feature extraction model and the classification model corresponding to the feature generation model based on the target global feature generation model and the target global classification model to obtain the target feature extraction model and the target classification model can be referred to the specific content in step S50 above, and will not be repeated here.

[0138] Prior to the step of feeding back the target global classification model and the target global feature generation model to the first device, the federated learning modeling optimization method further includes:

[0139] Step E10: Determine whether the target global classification model and the target global feature generation model satisfy the preset federated iteration termination condition;

[0140] Step E20: If satisfied, notify each of the first devices that the federated iteration has ended, and execute the following steps: feed back the target global classification model and the target global feature generation model to the first device;

[0141] If step E30 is not satisfied, then proceed directly to step: feed back the target global classification model and the target global feature generation model to the first device.

[0142] In this embodiment, it should be noted that the preset federated iteration termination condition is the condition for determining whether to end federated learning modeling. The preset federated iteration termination condition can be reaching the maximum number of federated iterations or the convergence of the loss function, etc.

[0143] Specifically, it is determined whether the target global classification model and the target global feature generation model meet the preset federated iteration termination condition; if they do, each of the first devices is notified that the federated iteration has ended, and the following steps are executed: feeding back the target global classification model and the target global feature generation model to the first device so that the first device can optimize its local model to obtain the final target feature extraction model and the final target classification model; if they do not meet the condition, the following steps are executed directly: feeding back the target global classification model and the target global feature generation model to the first device so that the first device can optimize its local model to proceed to the next round of federated iteration.

[0144] Additionally, it should be noted that in the embodiments of this application, the first device can also classify the output of the feature generation model and the output of the feature extraction model simultaneously through a classification model, and iteratively update to obtain the feature generation model. However, during the training process, the classification model always makes the output of the feature generation model consistent with the output of the feature extraction model. Thus, the classification model can generate the same classification label for the corresponding output of the feature generation model and the output of the feature extraction model. However, the feature discrimination model causes a certain difference between the output of the feature generation model and the output of the feature extraction model, which will lead to the feature generation model being difficult to converge and is prone to overfitting on the local sample set. In the embodiments of this application, when constructing the feature generation model in steps S10 to S30, a classification model is not used to classify the output of the feature generation model to optimize the feature generation model. Instead, in steps D30 and D40, the global classification model is adapted to each feature generation model, and the global feature generation model is adapted to each classification model. This avoids the feature generation model from being difficult to converge and from being prone to overfitting on the local sample set, thus improving the efficiency of federated learning modeling. Similarly, if the feature generation model is an image feature generation model, it improves the efficiency of constructing the target image feature generation model and the target image classification model based on federated learning.

[0145] This application provides a federated learning modeling optimization method, which differs from existing technologies where each participant maintains a local model and a global model in a federated learning scenario. In this method, each participant learns local knowledge through its local model and shares knowledge from all participants through the global model. Then, each participant aggregates its local and global models. This application first receives feature generation models and classification models from each first device, aggregates the feature generation models into a global feature generation model, and aggregates the classification models into a global classification model. It then extracts noise data and the corresponding real classification labels. Based on the noise data and the real classification labels, and with the feature generation models fixed, it iteratively optimizes the global classification model, making it compatible with the feature generation models. This indirectly promotes the global classification model to further learn the model knowledge of all federated learning participants, improving its accuracy and obtaining a target global classification model. Finally, based on the noise data and the real classification labels, and with the classification models fixed, iteratively optimizes the global feature generation model, making the classification models compatible with the global feature generation model, indirectly promoting the global feature generation model to learn the model knowledge of all federated learning participants. By further learning the model knowledge of all participants in the federated learning process, the accuracy of the global feature generation model is improved, resulting in a target global feature generation model. This target global classification model and the target global feature generation model are then fed back to the first device. The first device uses these models to iteratively optimize the feature extraction model and the classification model corresponding to the feature generation model. This improves the accuracy of the feature extraction and classification models used as local models. Furthermore, since the first device does not directly send the feature extraction model to the second device for aggregation, but instead sends the feature generation model, even if the second device uses the feature generation model to infer the privacy data of the first device, it will only obtain the input of the feature generation model—a mixed sample of noisy data and labels. This protects the data privacy of the federated learning participants from leakage. Therefore, it overcomes the technical flaw of the federated server being able to infer the original data of each participant from their global models, thus reducing the risk of data leakage in federated learning.

[0146] Reference Figure 8 , Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application.

[0147] like Figure 8As shown, the federated learning modeling optimization device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to establish communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM or a stable, non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0148] Optionally, the federated learning modeling optimization device may also include a rectangular user interface, a network interface, a camera, RF (Radio Frequency) circuitry, sensors, audio circuitry, a WiFi module, etc. The rectangular user interface may include a display screen and an input submodule such as a keyboard; optionally, the rectangular user interface may also include standard wired or wireless interfaces. The network interface may optionally include standard wired or wireless interfaces (such as a Wi-Fi interface).

[0149] Those skilled in the art will understand that Figure 8 The structure of the federated learning modeling optimization device shown in the figure does not constitute a limitation on the federated learning modeling optimization device. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0150] like Figure 8 As shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, and a federated learning modeling optimization program. The operating system is a program that manages and controls the hardware and software resources of the federated learning modeling optimization device, supporting the operation of the federated learning modeling optimization program and other software and / or programs. The network communication module is used to enable communication between the various components within the memory 1005, as well as communication with other hardware and software in the federated learning modeling optimization system.

[0151] exist Figure 8 In the federated learning modeling optimization device shown, the processor 1001 is used to execute the federated learning modeling optimization program stored in the memory 1005 to implement the steps of the federated learning modeling optimization method described above.

[0152] The specific implementation of the federated learning modeling optimization device in this application is basically the same as the embodiments of the federated learning modeling optimization method described above, and will not be repeated here.

[0153] This application embodiment also provides a federated learning modeling optimization device, which is applied to a first device, and the federated learning modeling optimization device includes:

[0154] The extraction module is used to obtain the trained feature extraction model and classification model, and to extract the first training sample, the first noisy data, and the first true classification label corresponding to the first training sample.

[0155] The generation module is used to obtain the first sample features generated by the feature extraction model for the first training sample, and the second sample features generated by the feature generation model to be trained for the first noisy data and the first real classification label.

[0156] The first iterative optimization module is used to perform feature discrimination on the first sample features and the second sample features through the feature discrimination model to be trained, and to iteratively optimize the feature generation model to be trained while fixing the feature extraction model, so as to obtain the feature generation model.

[0157] The sending module is used to send the feature generation model and the classification model to the second device, so that the second device can iteratively optimize and aggregate the global classification model obtained by the first device based on the feature generation model sent by each of the first devices, and iteratively optimize and aggregate the global feature generation model obtained by the first device based on the classification model sent by each of the first devices, to obtain a target global feature generation model and a target global classification model.

[0158] The second iterative optimization module is used to receive the target global feature generation model and the target global classification model fed back by the second device, and iteratively optimize the feature extraction model and the classification model based on the target global feature generation model and the target global classification model to obtain the target feature extraction model and the target classification model.

[0159] Optionally, the generation module is further configured to:

[0160] The feature discrimination model to be trained is used to perform feature discrimination on the first sample features and the second sample features, and the feature discrimination loss is calculated.

[0161] Based on the feature discrimination loss, while keeping the feature extraction model fixed, the feature discrimination model to be trained and the feature generation model to be trained are iteratively updated to optimize the feature generation model to be trained, thereby obtaining the feature generation model.

[0162] Optionally, the generation module is further configured to:

[0163] Based on the feature discrimination model to be trained, the first sample feature and the second sample feature are classified into two categories respectively to obtain the classification result;

[0164] Based on the binary classification results and the positive and negative sample labels corresponding to the first sample features and the second sample features, the feature discrimination loss is calculated.

[0165] Optionally, the second iterative optimization module is further configured to:

[0166] The target global classification model is used as a new classification model, and the target global feature generation model is used as a new feature generation model. The second training sample, the second noise data, and the second true classification label corresponding to the second training sample are extracted.

[0167] Based on the feature extraction model, features are extracted from the second training sample to obtain the third sample features, and based on the new feature generation model, the second noisy data and the second true classification label are converted into the fourth sample features.

[0168] The new classification model is used to classify the features of the third sample, and the feature discrimination model is used to discriminate the features of the third sample and the features of the fourth sample. The new classification model and the feature extraction model are iteratively optimized to obtain the target feature extraction model and the target classification model.

[0169] Optionally, the second iterative optimization module is further configured to:

[0170] If the target global classification model and the target global feature generation model satisfy the preset federated iteration termination condition, then the third sample feature is classified by the new classification model, and the third sample feature and the fourth sample feature are discriminated by the feature discrimination model. The new classification model and the feature extraction model are iteratively optimized to obtain the target feature extraction model and the target classification model.

[0171] If the target global classification model and the target global feature generation model do not meet the preset federated iteration termination condition, then the third sample feature is classified by the new classification model, and the third sample feature and the fourth sample feature are discriminated by the feature discrimination model. The new classification model and the feature extraction model are iteratively optimized, and the execution step is returned: extract the first training sample, the first noisy data, and the first true classification label corresponding to the first training sample.

[0172] Optionally, the second iterative optimization module is further configured to:

[0173] The new classification model is used to classify the features of the third sample, and the global classification loss is calculated.

[0174] The feature discrimination model is used to perform feature discrimination on the features of the third sample and the features of the fourth sample, and the global feature discrimination loss is calculated.

[0175] The feature extraction model, the new classification model, the new feature generation model, and the feature discrimination model are iteratively updated using the global classification loss and the global feature discrimination loss to optimize the feature extraction model and the new classification model, thereby obtaining the target feature extraction model and the target classification model.

[0176] Optionally, the second iterative optimization module is further configured to:

[0177] Knowledge distillation is performed between the classification model and the target global classification model to optimize the classification model and obtain the target classification model;

[0178] Knowledge distillation is performed between the feature extraction model and the target global feature generation model to optimize the feature extraction model and obtain the target feature extraction model.

[0179] The specific implementation of the federated learning modeling optimization device in this application is basically the same as the embodiments of the federated learning modeling optimization method described above, and will not be repeated here.

[0180] This application embodiment also provides a federated learning modeling optimization device, which is applied to a second device, and the federated learning modeling optimization device includes:

[0181] An aggregation module is used to receive feature generation models and classification models sent by each first device, and to aggregate the feature generation models into a global feature generation model and the classification models into a global classification model.

[0182] The extraction module is used to extract noise data and the corresponding real classification labels of the noise data;

[0183] The first iterative optimization module is used to iteratively optimize the global classification model based on the noise data and the real classification label, while keeping the feature generation models fixed, to obtain the target global classification model.

[0184] The second iterative optimization module is used to iteratively optimize the global feature generation model based on the noise data and the real classification label, while keeping each of the classification models fixed, to obtain the target global feature generation model.

[0185] The feedback module is used to feed back the target global classification model and the target global feature generation model to the first device, so that the first device can iteratively optimize the feature extraction model and the classification model corresponding to the feature generation model based on the target global feature generation model and the target global classification model to obtain the target feature extraction model and the target classification model.

[0186] Optionally, the first iterative optimization module is further configured to:

[0187] Based on the feature generation models described above, the noisy data and the real classification labels are converted into corresponding local sample features.

[0188] Based on the global classification model, the features of each local sample are classified to obtain the predicted local classification labels.

[0189] The first target loss is calculated based on the difference between each of the locally predicted classification labels and the true classification labels;

[0190] Based on the first target loss, the global classification model is iteratively optimized while keeping the feature generation models fixed, to obtain the target global classification model.

[0191] Optionally, the second iterative optimization module is further configured to:

[0192] Based on the global feature generation model, the noisy data and the true classification label are jointly converted into corresponding global generated features;

[0193] Based on each of the classification models, the globally generated features are classified to obtain the globally predicted classification labels corresponding to each classification model;

[0194] The second objective loss is calculated based on the difference between each of the globally predicted classification labels and the true classification labels;

[0195] Based on the second target loss, the global feature generation model is iteratively optimized while keeping the classification models fixed, to obtain the target global feature generation model.

[0196] Optionally, the federated learning modeling optimization device is further used for:

[0197] Determine whether the target global classification model and the target global feature generation model satisfy the preset federated iteration termination condition;

[0198] If the conditions are met, notify each of the first devices that the federated iteration has ended, and execute the following steps: feed back the target global classification model and the target global feature generation model to the first device;

[0199] If the conditions are not met, proceed directly to step: feed back the target global classification model and the target global feature generation model to the first device.

[0200] The specific implementation of the federated learning modeling optimization device in this application is basically the same as the embodiments of the federated learning modeling optimization method described above, and will not be repeated here.

[0201] This application provides a readable storage medium that stores one or more programs, which can be executed by one or more processors to implement the steps of the federated learning modeling optimization method described in any of the above claims.

[0202] The specific implementation of the readable storage medium in this application is basically the same as the embodiments of the federated learning modeling optimization method described above, and will not be repeated here.

[0203] This application provides a computer program product, which includes one or more computer programs. The one or more computer programs can be executed by one or more processors to implement the steps of the federated learning modeling optimization method described in any of the above claims.

[0204] The specific implementation of the computer program product in this application is basically the same as the embodiments of the federated learning modeling optimization method described above, and will not be repeated here.

[0205] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A federated learning modeling optimization method, characterized in that, Applied to the first device, the federated learning modeling optimization method includes: Obtain the trained feature extraction model and classification model, and extract the first training sample, the first noisy data, and the first true classification label corresponding to the first training sample; Obtain the first sample features generated by the feature extraction model for the first training sample, and the second sample features generated by the feature generation model to be trained for the first noisy data and the first true classification label; The first sample features and the second sample features are discriminated by a feature discrimination model to be trained. The feature generation model to be trained is iteratively optimized while the feature extraction model is fixed to obtain a feature generation model. The feature generation model and the classification model are sent to a second device, which iteratively optimizes and aggregates the classification models to obtain a global classification model based on the feature generation models sent by the first devices, and iteratively optimizes and aggregates the feature generation models to obtain a global feature generation model based on the classification models sent by the first devices. With the feature generation models fixed, the outputs of each feature generation model are classified according to the global classification model, and a first global classification loss is calculated. Then, based on the first global classification loss, the global classification model is iteratively optimized to enable the global classification model to indirectly learn the knowledge of all first devices through all feature generation models, thereby obtaining a target global classification model. With the classification models fixed, the outputs of the global feature generation model are classified according to each classification model, and a second global classification loss is calculated. Then, based on the second global classification loss, the global feature generation model is iteratively optimized to enable the global feature generation model to indirectly learn the knowledge of all first devices through all classification models, thereby obtaining a target global feature generation model. The system receives the target global feature generation model and target global classification model fed back by the second device, and iteratively optimizes the feature extraction model and the classification model based on the target global feature generation model and the target global classification model to obtain the target feature extraction model and the target classification model.

2. The federated learning modeling optimization method as described in claim 1, characterized in that, The step of performing feature discrimination on the first sample features and the second sample features using a feature discrimination model to be trained, and iteratively optimizing the feature generation model to be trained while fixing the feature extraction model, to obtain the feature generation model includes: The feature discrimination model to be trained is used to perform feature discrimination on the first sample features and the second sample features, and the feature discrimination loss is calculated. Based on the feature discrimination loss, while keeping the feature extraction model fixed, the feature discrimination model to be trained and the feature generation model to be trained are iteratively updated to optimize the feature generation model to be trained, thereby obtaining the feature generation model.

3. The federated learning modeling optimization method as described in claim 2, characterized in that, The step of performing feature discrimination on the first sample features and the second sample features using the feature discrimination model to be trained, and calculating the feature discrimination loss, includes: Based on the feature discrimination model to be trained, the first sample feature and the second sample feature are classified into two categories respectively to obtain the classification result; Based on the binary classification results and the positive and negative sample labels corresponding to the first sample features and the second sample features, the feature discrimination loss is calculated.

4. The federated learning modeling and optimization method as described in claim 1, characterized in that, The step of iteratively optimizing the feature extraction model and the classification model based on the target global feature generation model and the target global classification model to obtain the target feature extraction model and the target classification model includes: The target global classification model is used as a new classification model, and the target global feature generation model is used as a new feature generation model. The second training sample, the second noise data, and the second true classification label corresponding to the second training sample are extracted. Based on the feature extraction model, features are extracted from the second training sample to obtain the third sample features, and based on the new feature generation model, the second noisy data and the second true classification label are converted into the fourth sample features. The new classification model is used to classify the features of the third sample, and the feature discrimination model is used to discriminate the features of the third sample and the features of the fourth sample. The new classification model and the feature extraction model are iteratively optimized to obtain the target feature extraction model and the target classification model.

5. The federated learning modeling optimization method as described in claim 4, characterized in that, The steps of classifying the third sample features using the new classification model, and discriminating the third sample features and the fourth sample features using the feature discrimination model, and iteratively optimizing the new classification model and the feature extraction model to obtain the target feature extraction model and the target classification model include: If the target global classification model and the target global feature generation model satisfy the preset federated iteration termination condition, then the third sample feature is classified by the new classification model, and the third sample feature and the fourth sample feature are discriminated by the feature discrimination model. The new classification model and the feature extraction model are iteratively optimized to obtain the target feature extraction model and the target classification model. If the target global classification model and the target global feature generation model do not meet the preset federated iteration termination condition, then the third sample feature is classified by the new classification model, and the third sample feature and the fourth sample feature are discriminated by the feature discrimination model. The new classification model and the feature extraction model are iteratively optimized, and the execution step is returned: extract the first training sample, the first noisy data, and the first true classification label corresponding to the first training sample.

6. The federated learning modeling optimization method as described in claim 4 or 5, characterized in that, The steps of classifying the third sample features using the new classification model, and discriminating the third sample features and the fourth sample features using the feature discrimination model, and iteratively optimizing the new classification model and the feature extraction model to obtain the target feature extraction model and the target classification model include: The new classification model is used to classify the features of the third sample, and the global classification loss is calculated. The feature discrimination model is used to perform feature discrimination on the features of the third sample and the features of the fourth sample, and the global feature discrimination loss is calculated. The feature extraction model, the new classification model, the new feature generation model, and the feature discrimination model are iteratively updated using the global classification loss and the global feature discrimination loss to optimize the feature extraction model and the new classification model, thereby obtaining the target feature extraction model and the target classification model.

7. The federated learning modeling optimization method as described in claim 1, characterized in that, The step of iteratively optimizing the feature extraction model and the classification model based on the target global feature generation model and the target global classification model to obtain the target feature extraction model and the target classification model includes: Knowledge distillation is performed between the classification model and the target global classification model to optimize the classification model and obtain the target classification model; Knowledge distillation is performed between the feature extraction model and the target global feature generation model to optimize the feature extraction model and obtain the target feature extraction model.

8. A federated learning modeling optimization method, characterized in that, Applied to a second device, the federated learning modeling optimization method includes: Receive feature generation models and classification models sent by each first device, and aggregate the feature generation models into a global feature generation model and aggregate the classification models into a global classification model; Extract the noise data and the corresponding real classification labels; Based on the noise data and the true classification label, the global classification model is iteratively optimized while keeping the feature generation models fixed, to obtain the target global classification model; Based on the noise data and the true classification labels, the global feature generation model is iteratively optimized while keeping the classification models fixed, to obtain the target global feature generation model; The target global classification model and the target global feature generation model are fed back to the first device, so that the first device can iteratively optimize the feature extraction model and the classification model corresponding to the feature generation model based on the target global feature generation model and the target global classification model, so as to obtain the target feature extraction model and the target classification model. The step of iteratively optimizing the global classification model based on the noise data and the true classification label, while keeping the feature generation models fixed, to obtain the target global classification model includes: Based on the feature generation models described above, the noisy data and the real classification labels are converted into corresponding local sample features. Based on the global classification model, the features of each local sample are classified to obtain the predicted local classification labels. The first target loss is calculated based on the difference between each of the locally predicted classification labels and the true classification labels; Based on the first target loss, the global classification model is iteratively optimized while keeping the feature generation models fixed, to obtain the target global classification model; The step of iteratively optimizing the global feature generation model based on the noise data and the true classification labels, while keeping the classification models fixed, to obtain the target global feature generation model includes: Based on the global feature generation model, the noisy data and the true classification label are jointly converted into corresponding global generated features; Based on each of the classification models, the globally generated features are classified to obtain the globally predicted classification labels corresponding to each classification model; The second objective loss is calculated based on the difference between each of the globally predicted classification labels and the true classification labels; Based on the second target loss, the global feature generation model is iteratively optimized while keeping the classification models fixed, to obtain the target global feature generation model.

9. The federated learning modeling optimization method as described in claim 8, characterized in that, Prior to the step of feeding back the target global classification model and the target global feature generation model to the first device, the federated learning modeling optimization method further includes: Determine whether the target global classification model and the target global feature generation model satisfy the preset federated iteration termination condition; If the conditions are met, notify each of the first devices that the federated iteration has ended, and execute the following steps: feed back the target global classification model and the target global feature generation model to the first device; If the conditions are not met, proceed directly to step: feed back the target global classification model and the target global feature generation model to the first device.

10. A federated learning modeling optimization device, characterized in that, The federated learning modeling optimization device includes: a memory, a processor, and a program stored in the memory for implementing the federated learning modeling optimization method. The memory is used to store programs that implement federated learning modeling optimization methods; The processor is configured to execute a program that implements the federated learning modeling optimization method, thereby implementing the steps of the federated learning modeling optimization method as described in any one of claims 1 to 7 or 8 to 9.

11. A readable storage medium, characterized in that, The readable storage medium stores a program that implements the federated learning modeling optimization method, which is executed by a processor to implement the steps of the federated learning modeling optimization method as described in any one of claims 1 to 7 or 8 to 9.

12. A program product, said program product being a computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the federated learning modeling optimization method as described in any one of claims 1 to 7 or 8 to 9.

Images