Image classification model federated learning training method and related device

Abstract

This invention belongs to the field of image processing and discloses a federated learning training method and related apparatus for an image classification model. The method includes: distributing the image classification model to be trained to each training party in the federated learning process; each training party performing local training based on local data, with labeled training parties using a label distribution-aware alignment loss function and unlabeled training parties using a consistency loss function; obtaining the model parameter update amounts and aggregating them separately to obtain labeled and unlabeled aggregation results; obtaining a learning performance evaluation value, weighting and aggregating the labeled and unlabeled aggregation results to obtain a global aggregation result, and updating the model. The label distribution-aware alignment loss function corrects training bias caused by class imbalance, the consistency loss function mines supervision signals in unlabeled data, and the aggregation weights are dynamically adjusted based on the learning performance evaluation value, thereby improving the robustness and generalization ability of the image classification model in heterogeneous data environments and enhancing image classification accuracy.

Description

Technical Field

[0001] This invention belongs to the field of image processing and relates to a federated learning training method and related apparatus for image classification models. Background Technology

[0002] Federated learning, as a distributed machine learning paradigm, can coordinate multiple participants to jointly train a global model without the data leaving the local machine, effectively solving the problems of data privacy protection and data silos. Therefore, it has received widespread attention in classification tasks involving sensitive image data, such as medical image analysis, autonomous driving data collaboration, and security monitoring. Traditional federated learning frameworks typically assume that all participating clients have a fully labeled image dataset. However, in practical applications, large-scale image data annotation requires significant human and time costs and deep involvement of domain experts, making it difficult for many clients to provide sufficient labeled data, severely limiting the large-scale deployment of federated learning in real-world scenarios. To address this challenge, federated semi-supervised learning has emerged, allowing some clients to hold the full amount of labeled image data, while others only hold unlabeled image data. By jointly utilizing information from both labeled and unlabeled data, a global image classification model can be collaboratively trained. In a typical federated semi-supervised learning scenario, labeled clients use cross-entropy loss for supervised training, while unlabeled clients typically employ semi-supervised techniques such as consistency regularization to generate pseudo-labels based on weakly and strongly augmented samples to guide model updates.

[0003] However, federated semi-supervised learning faces a unique adversarial interference problem caused by the extremely imbalanced distribution of globally labeled data when applied to image classification tasks. On the one hand, federated semi-supervised learning inherits the inherent non-independent and identically distributed (Non-IID) problem in federated learning, meaning that the class distribution of local image data from different clients varies significantly, causing the local model update direction to deviate from the global optimum. More seriously, due to the limited number of labeled clients and the diverse class distribution of their local image data, the global labeled data distribution formed after aggregating the local data from all labeled clients often still exhibits a severe class imbalance. This imbalance in the global labeled data distribution will cause the supervised learning process to favor the head classes with a larger sample size, resulting in a severe deficiency in the model's ability to learn images of tail classes. This, in turn, contaminates the quality of pseudo-labels generated by unlabeled clients, forming a chain of error accumulation and amplification from labeling bias and pseudo-label errors to further exacerbating model bias. Errors in pseudo-labels will further exacerbate the model's overfitting to head classes and underfitting to tail classes, ultimately severely impairing the convergence speed, classification accuracy, and generalization performance of the global image classification model.

[0004] Existing solutions to the Non-IID problem in federated semi-supervised learning, such as client-side consistency constraints or random sampling consensus aggregation, largely treat the Non-IID problem in federated semi-supervised learning as a simple extension of similar problems in traditional federated learning. They fail to fully recognize the unique adversarial interference effect introduced by the imbalanced distribution of globally labeled data. Specifically, existing methods lack a priori perception mechanism for the global labeled class distribution, making it impossible to perform targeted gradient compensation for tail classes during local supervised training. Furthermore, their aggregation strategies typically employ fixed weighted averages, failing to dynamically adjust the contribution ratio of supervised and unsupervised signals in the global model based on their real-time quality. Therefore, in image classification scenarios with severely biased labeled data, these methods exhibit significantly insufficient robustness, failing to effectively block the propagation chain of label bias to the pseudo-label generation stage, and unable to dynamically balance the contributions of supervised and unsupervised signals during aggregation. This leads to a sharp decline in model performance under extreme imbalance conditions, making it difficult to guarantee stable and efficient training of federated semi-supervised learning systems. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a federated learning training method and related apparatus for image classification models.

[0006] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, this invention provides a federated learning training method for an image classification model, comprising: distributing the image classification model to be trained to each training party in the federated learning; each training party performing local training of the image classification model based on local data; wherein, when the training party is a labeled training party, a label distribution-aware alignment loss function is used to calculate the loss value, and when the training party is an unlabeled training party, a consistency loss function is used to calculate the loss value; wherein, in the label distribution-aware alignment loss function, the category with fewer samples corresponds to a larger loss weight; the consistency loss function is constructed based on the prediction consistency of different augmented views of the same unlabeled sample; obtaining the model parameter update amount obtained by each training party based on local training, and aggregating it based on the labeled and unlabeled training parties to obtain labeled aggregation results and unlabeled aggregation results; obtaining and weighting the labeled aggregation results and unlabeled aggregation results according to the learning effect evaluation values ​​of the labeled and unlabeled training parties to obtain a global aggregation result, and updating the image classification model according to the global aggregation result to obtain the trained image classification model; wherein, the larger the learning effect evaluation value, the smaller the weighted aggregation weight.

[0007] Optionally, the label distribution-aware alignment loss function is:

[0008] in, For label distribution-aware alignment loss, For the first Model parameters of an image classification model trained by an labeled training method; For the first The total number of locally annotated image samples for each annotation training method; For the first The true category of each image sample The corresponding loss weights are as follows: the smaller the number of samples for a category, the greater the loss weight. The cross-entropy loss function; For the first The true category label of each image sample; For input The predicted output is a category. The probability of; For category index variables; For the first Image samples.

[0009] Optionally, the We obtain it from the following formula:

[0010] in, For globally labeled data The proportion of image samples, It is a preset minimum positive number.

[0011] Or it can be obtained through the following formula:

[0012] in, For globally labeled data The number of image samples, The total number of categories, For the first in the globally labeled data Number of image samples for each category.

[0013] Or it can be obtained through the following formula:

[0014] in, This is the preset first temperature coefficient.

[0015] Optionally, the consistency loss function is:

[0016] in, This represents the consistency loss value. For the first Model parameters of an image classification model with unlabeled training methods. For the first The total number of local unlabeled image samples for each unlabeled training method. For indicator functions, The value is 1 if the condition is met, and 0 otherwise. For image classification models The maximum value of the predicted probability distribution of the weakly enhanced samples. To preset the reliability threshold, Let cross-entropy be the loss function. for pseudo-tags For image classification models in The predicted output is a category. The probability, For the first Image samples, for Strongly enhanced image samples, This is a category index variable.

[0017] Among them, the Weakly enhanced samples by The weak enhancement operation is performed, which is a random horizontal flip and / or random cropping; Passing through The result is obtained by performing a strong enhancement operation, which is an automatic search enhancement strategy.

[0018] Optionally, obtaining the model parameter update amounts obtained by each training party based on local training, and aggregating them based on labeled and unlabeled training parties to obtain labeled aggregation results and unlabeled aggregation results includes: The annotation aggregation result is obtained using the following formula. :

[0019] in, For the first The model parameters of the image classification model after rounds of federated learning. To label the total number of training sides, For the first The total number of locally annotated image samples for each annotation training method. For the first The total number of locally annotated image samples for each annotation training method. For the first The number of model parameter updates for each labeled training method.

[0020] The unlabeled aggregation result is obtained using the following formula. :

[0021] in, This represents the total number of unlabeled training units. For the first The total number of local unlabeled image samples for each unlabeled training method. For the first The total number of local unlabeled image samples for each unlabeled training method. For the first The number of model parameter updates uploaded by each unlabeled training party.

[0022] Optionally, obtaining and evaluating the learning performance of labeled and unlabeled training participants includes: for each labeled training participant, calculating the average loss value based on the label distribution-aware alignment loss function before and after local training, and dividing the difference between the average loss value before and after local training by the average loss value before local training to obtain the learning performance evaluation value of the labeled training participant; or, taking an exponential negative mapping of the average loss value after local training to obtain the learning performance evaluation value of the labeled training participant; or, by statistically analyzing image samples in the locally labeled image samples where the prediction confidence of the image classification model after local training is greater than a preset confidence threshold. The proportion of the number of images in the local labeled image samples is used as the learning performance evaluation value for each labeled training agent. For each unlabeled training agent, the proportion of the number of images in the local unlabeled image samples whose prediction confidence based on the locally trained image classification model is greater than a preset confidence threshold is used as the learning performance evaluation value for each unlabeled training agent. The learning performance evaluation values ​​of each labeled training agent are weighted and averaged based on the number of image samples to obtain the learning performance evaluation value for the labeled training agent. The learning performance evaluation values ​​of each unlabeled training agent are weighted and averaged based on the number of image samples to obtain the learning performance evaluation value for the unlabeled training agent.

[0023] Optionally, the weighted aggregation of labeled and unlabeled aggregation results includes: The weighted aggregation results of labeled and unlabeled aggregation results are weighted and aggregated using the following formula:

[0024]

[0025]

[0026] in, For the first The model parameters of the image classification model after rounds of federated learning. To label the fusion weights of the aggregation results, To label the aggregation results, The fusion weights for unlabeled aggregation results. For unlabeled aggregation results, To label the learning performance evaluation values ​​of the trainer, The learning performance evaluation value for the unlabeled training method. This is the preset second temperature coefficient.

[0027] In a second aspect, the present invention provides a federated learning training system for an image classification model, comprising: a model distribution module for distributing the image classification model to be trained to each training party in the federated learning process; a local training module for each training party to perform local training of the image classification model based on local data; wherein, when the training party is a labeled training party, a label distribution-aware alignment loss function is used to calculate the loss value, and when the training party is an unlabeled training party, a consistency loss function is used to calculate the loss value; wherein, in the label distribution-aware alignment loss function, the category with fewer samples corresponds to a larger loss weight; the consistency loss function is constructed based on the prediction consistency of different augmented views of the same unlabeled sample; a local aggregation module for obtaining the model parameter update amount obtained by each training party based on local training, and aggregating it based on the labeled and unlabeled training parties to obtain labeled aggregation results and unlabeled aggregation results; and a global aggregation module for obtaining and weighting the labeled aggregation results and unlabeled aggregation results according to the learning effect evaluation values ​​of the labeled and unlabeled training parties to obtain a global aggregation result, and updating the image classification model according to the global aggregation result to obtain the trained image classification model; wherein, the larger the learning effect evaluation value, the smaller the weighted aggregation weight.

[0028] In a third aspect, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the federated learning training method for the image classification model.

[0029] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the federated learning training method for the image classification model.

[0030] Compared with the prior art, the present invention has the following beneficial effects: This invention presents a federated learning training method for image classification models. By distributing the image classification model to be trained to each training participant in the federated learning process, a distributed training framework is established. During local training based on local data, the labeled training participants use a label distribution-aware alignment loss function to calculate the loss value. In this loss function, the class with fewer samples corresponds to a larger loss weight, thus applying explicit adversarial compensation for class imbalance during the local supervised training phase. Global distribution priors are used to upsample and reweight the tail class losses, forcing the model to treat all classes fairly during optimization and cutting off the error propagation chain from label bias to the unsupervised pseudo-label generation stage. The unlabeled training participants use a consistency loss function to calculate the loss value. This loss function is constructed based on the prediction consistency of different augmented views of the same unlabeled sample, enabling the extraction of effective supervision signals from unlabeled data. By obtaining the model parameter update amounts obtained by each training participant based on local training, and aggregating the results from both labeled and unlabeled training participants to obtain labeled and unlabeled aggregation results respectively, the supervised and unsupervised signals are decoupled before aggregation, laying the foundation for subsequent dynamic fusion. By acquiring and weighting the learning performance evaluation values ​​of both labeled and unlabeled training participants, the method weights and aggregates the labeled and unlabeled results. The higher the learning performance evaluation value, the smaller the fusion weight, achieving dynamic weight allocation based on learning performance. This intelligently balances the contribution ratio of real and pseudo-labeled knowledge in the global model construction, avoiding the pollution of the global model by low-quality pseudo-labels under extreme imbalance conditions. It also prevents the limitation of limited labeled data on the model's generalization ability. Ultimately, this method effectively suppresses the adversarial interference caused by the imbalanced distribution of global labeled data in the training process of federated semi-supervised image classification models. It significantly improves the convergence performance, classification accuracy, and robustness of image classification models in heterogeneous data environments. It can be widely applied in distributed learning scenarios with stringent requirements for data privacy and model reliability, such as medical image analysis, autonomous driving data sharing, and cross-institutional financial risk control, achieving highly stable and accurate image classification. Attached Figure Description

[0031] Figure 1 This is a flowchart of the federated learning training method for the image classification model according to an embodiment of the present invention.

[0032] Figure 2 This is a schematic diagram illustrating the principle of the federated learning training method for the image classification model in an embodiment of the present invention.

[0033] Figure 3 This is a block diagram of the federated learning training system for the image classification model according to an embodiment of the present invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] The present invention will now be described in further detail with reference to the accompanying drawings: See Figure 1 In one embodiment of the present invention, a federated learning training method for an image classification model is provided, which helps to effectively resist adversarial region bias caused by the global imbalance of labeled data in a federated semi-supervised learning scenario, and significantly improves the classification accuracy and convergence stability of the image classification model.

[0037] Specifically, the federated learning training method for the image classification model of this invention includes the following steps: S1: Distribute the image classification model to be trained to each training party in the federated learning process.

[0038] Explaining this, this step establishes the initial state of the federated learning system. Federated learning is a distributed machine learning paradigm whose core characteristic is training a global model collaboratively by multiple participants without exchanging raw data. In practical deployment, a federated learning system consists of a central server and multiple clients participating in the training (i.e., trainers). The central server is responsible for initializing the global model parameters and distributing the image classification model to be trained to all trainers participating in this federated learning process. The image classification model can be any deep learning model suitable for image classification tasks, such as a convolutional neural network, residual network, or visual transformer. Based on this, each trainer obtains the initial model parameters, preparing for local training.

[0039] S2: Each training party trains the image classification model locally based on local data.

[0040] Specifically, when the trainer is a labeled trainer, the label distribution-aware alignment loss function is used to calculate the loss value; when the trainer is an unlabeled trainer, the consistency loss function is used to calculate the loss value. In the label distribution-aware alignment loss function, the category with fewer samples has a larger loss weight. The consistency loss function is constructed based on the prediction consistency of different augmented views of the same unlabeled sample.

[0041] Explained, this step is the core training stage of the method of this invention. In actual federated semi-supervised learning scenarios, the local data available to each training party differs. Some training parties possess a large number of manually labeled image samples and their corresponding class labels; these are called labeled training parties. Other training parties only possess raw, unlabeled image samples without corresponding class labels; these are called unlabeled training parties. To fully utilize the data resources of both types of training parties, this invention designs different local training loss functions for them.

[0042] Specifically, for the annotation training, this invention employs a label distribution-aware alignment loss function. Explain, traditional image classification model training typically uses the standard cross-entropy loss function, which assumes that all categories appear with equal frequency in the training data. However, in practical applications of federated semi-supervised learning, the difficulty of obtaining image samples from different categories varies significantly. For example, in medical image analysis tasks, image samples of common lesion types are readily available and plentiful, while image samples of rare lesion types are difficult to obtain and scarce. When training using the traditional cross-entropy loss function, the model tends to prioritize learning the head categories with a larger sample count, while severely underlearning the tail categories with fewer samples. To address this issue, the label distribution-aware alignment loss function proposed in this invention introduces a category weighting mechanism. Specifically, this loss function assigns differentiated loss weights to different categories based on the number of image samples for each category in the globally labeled data. Categories with fewer samples are assigned larger loss weights, resulting in a higher contribution proportion of samples from that category to the loss function. In this way, even if the number of samples in the tail category is small, the magnitude of its gradient update can be effectively enhanced, thereby forcing the model to treat all categories fairly during training and avoiding excessive bias of the model towards the head category.

[0043] Specifically, for unlabeled trainers, this invention employs a consistency loss function. Explain why unlabeled trainers lack real labels for supervised training; therefore, effective supervisory signals need to be mined from unlabeled image data. This invention constructs a consistency loss function based on the consistency regularization principle in semi-supervised learning. The core idea of ​​consistency regularization is that for a given unlabeled image sample, after different data augmentation operations, when input into the same model, the model should output consistent prediction results. Based on this principle, this invention generates weakly augmented and strongly augmented samples for each unlabeled image sample. Weakly augmented samples undergo simple image transformations, retaining most of the information of the original image, and the model's prediction results for them have high reliability; strongly augmented samples undergo more drastic image transformations, effectively expanding data diversity. The model's prediction results for weakly augmented samples are filtered through a confidence threshold to generate pseudo-labels, which are then used as supervisory signals to constrain the model's prediction output for strongly augmented samples. In this way, the model can learn effective feature representations from a large amount of unlabeled image data.

[0044] S3: Obtain the model parameter update amount obtained by each training party based on local training, and aggregate them based on labeled and unlabeled training parties to obtain labeled aggregation results and unlabeled aggregation results.

[0045] Explained, this step is used to perform hierarchical aggregation of model updates from different types of trainers on the server side. After each trainer completes local training, each trainer calculates a model parameter update, which represents the change in model parameters before and after local training. All trainers upload their respective model parameter update amounts to the central server. Upon receiving these update amounts, the server divides them into two independent sets according to the type of trainer (labeled trainer or unlabeled trainer).

[0046] Specifically, for the set of labeled training agents, supervised aggregation is performed. Interpretably, the purpose of supervised aggregation is to merge the knowledge learned by each labeled training agent based on their local labeled data, forming a unified supervised aggregation result. Since the number of locally labeled image samples possessed by different labeled training agents may vary, the model updates of training agents with more samples generally have higher statistical reliability. Therefore, during the aggregation process, the contribution of each labeled training agent is proportional to the number of locally labeled image samples; that is, training agents with more local samples have a larger weight in the aggregation process.

[0047] Specifically, for the set of unlabeled trainers, an unsupervised aggregation operation is performed. Interpretably, the purpose of unsupervised aggregation is to merge the knowledge learned by each unlabeled trainer based on its local unlabeled data, forming a unified unsupervised aggregation result. Similar to supervised aggregation, the aggregation weight of the unlabeled trainers is proportional to the number of their local unlabeled image samples.

[0048] By separately aggregating the model update amounts of labeled and unlabeled trainers, the supervised and unsupervised signals are decoupled before aggregation, laying the foundation for subsequent dynamic weighted fusion.

[0049] S4: Obtain and weight the aggregated results of labeled and unlabeled training agents based on their learning performance evaluation values ​​to obtain a global aggregated result. Update the image classification model based on the global aggregated result to obtain the trained image classification model. The larger the learning performance evaluation value, the smaller the corresponding weighted aggregation weight.

[0050] Explained, this step dynamically fuses supervised and unsupervised aggregation results based on the learning performance of each trainer, generating the final global model. In traditional federated averaging algorithms, the model parameter updates from all clients are directly weighted and averaged, ignoring the quality differences between different clients and different types of signals. However, in real-world federated semi-supervised learning scenarios, the learning performance of labeled and unlabeled trainers may dynamically fluctuate with training epochs. For example, in the early stages of training, with limited labeled data, the learning performance of labeled trainers may be unsatisfactory; while in later stages, the quality of pseudo-labels generated by unlabeled trainers may decline due to accumulated errors.

[0051] To adapt to this dynamic change, this invention introduces a learning performance evaluation value to quantify the maturity of each training party's model's fit to the data during local training. Based on this evaluation value, the weights of the weighted aggregation of labeled and unlabeled aggregation results are determined. The core design principle is that the higher the learning performance evaluation value, the smaller the fusion weight. This design is because when a subsystem's learning performance is good, it indicates that the subsystem has contributed sufficient knowledge in the current round, and its weight should be appropriately reduced to allow other subsystems more opportunities to contribute. Conversely, when a subsystem's learning performance is poor, it indicates that the subsystem has significant room for improvement and should be given a higher weight. This dynamic weighting mechanism effectively balances the contributions of supervised and unsupervised signals in global model construction, preventing one party from occupying excessive weight and polluting the global model when its quality is poor. Then, the supervised and unsupervised aggregation results are weighted and aggregated according to the calculated weights to obtain the global aggregation result for this round of communication, and this result is used to update the global parameters of the image classification model. The server then redistributes the updated global model parameters to each training party to enter the next training iteration. The above process is repeated until the performance metrics of the global model on the validation set converge or reach the preset communication round threshold, ultimately yielding the trained image classification model. Because this model fully considers the class imbalance problem and the dynamic balance between supervised and unsupervised signals during training, it exhibits significant robustness to heterogeneous data environments.

[0052] In summary, the federated learning training method for image classification models in this invention addresses the problem of adversarial region imbalance and non-independent identically distributed (ICD) issues that undermine model robustness in federated semi-supervised learning by employing refined supervised target correction and dynamic aggregation weight allocation strategies. This is achieved by decoupling supervised and unsupervised learning spaces and dynamically coordinating their contributions. Specifically, by introducing label distribution-aware alignment loss, the propagation chain of class bias to the pseudo-label generation stage is severed at the source of supervised training. Furthermore, by constructing a dynamic high-level aggregation mechanism, the weight ratio of real and pseudo-labeled knowledge in global model construction is intelligently balanced, preventing low-quality pseudo-labels from contaminating the global model under extreme imbalance conditions and avoiding the constraint of limited labeled data on the model's generalization ability. Therefore, the image classification model trained based on this method not only significantly outperforms traditional federated averaging and existing mainstream semi-supervised federated methods in terms of global accuracy when facing highly heterogeneous distributed data environments, but also exhibits minimal performance fluctuations, demonstrating extremely strong robustness and stability.

[0053] See Figure 2This paper illustrates the basic principle of the federated learning training method for the image classification model of the present invention, which mainly includes three parts: supervised learning, unsupervised learning, and dynamic high-level aggregation. In supervised learning, the server collects model updates uploaded by all labeled training participants. To address the problem of biased supervision targets caused by imbalanced global labeled data distribution, a label distribution-aware alignment loss is introduced locally by each labeled training participant. This loss function adaptively reweights the cross-entropy loss of each sample based on the global labeled class distribution statistics provided by the server, assigning higher loss weights to tail classes with fewer samples and lower loss weights to head classes with more samples, thus explicitly compensating for class imbalance during the local training phase. In unsupervised learning, the server collects model updates uploaded by all unlabeled training participants. Each unlabeled training participant processes its local unlabeled image data using a consistency regularization-based method, generating weakly augmented and strongly augmented views for each sample. Pseudo-labels are generated using the prediction results of the weakly augmented views, and the consistency loss between the prediction results of the strongly augmented views and the pseudo-labels is calculated, thereby extracting effective supervision signals from the unlabeled data. In the dynamic high-level aggregation mechanism, the server first performs weighted average aggregation on the update amounts of the two training parties to generate supervised aggregation results and unsupervised aggregation results. Then, based on the local learning effect evaluation values ​​uploaded by each training party, it calculates the learning effect evaluation values ​​of supervised learning and unsupervised learning respectively. Then, it dynamically calculates the fusion weight of supervised learning and unsupervised sub-learning, and performs weighted fusion on the supervised aggregation results and unsupervised aggregation results to generate a new round of global model.

[0054] In one possible implementation, the label distribution-aware alignment loss function is:

[0055] in, For label distribution-aware alignment loss, For the first Model parameters of an image classification model trained by an labeled training method; For the first The total number of locally annotated image samples for each annotation training method; For the first The true category of each image sample The corresponding loss weights are as follows: the smaller the number of samples for a category, the greater the loss weight. The cross-entropy loss function; For the first The true category label of each image sample; For input The predicted output is a category. The probability of; For category index variables; For the first Image samples.

[0056] Specifically, the construction of this loss function is based on the core idea of ​​label distribution awareness. Traditional cross-entropy loss... This loss measures the difference between the model's predicted probability distribution and the true labels; a higher value indicates a less accurate prediction. In standard image classification model training, this loss contributes equally to all samples. However, in cases of class imbalance, this equal treatment can lead to overfitting to the top class. To address this issue, this implementation multiplies the cross-entropy loss by a class-related weighting factor. The design principle of this weighting factor is: assign larger weights to categories with sparse sample sizes in the globally labeled data, and smaller weights to categories with sufficient sample sizes. By introducing such a weighting mechanism into the loss function, the loss contribution of tail category samples is amplified, allowing the model to obtain larger gradient updates from tail category samples during backpropagation. Thus, even though the absolute number of samples in the tail categories is small, their relative learning strength is effectively enhanced, thereby forcing the model to learn all categories fairly. The formula... This term is used to normalize the total number of locally labeled samples, so that the loss value is not affected by the number of local samples, which facilitates comparison between different training methods.

[0057] In one possible implementation, the We obtain it from the following formula:

[0058] in, For globally labeled data The proportion of image samples, It is a preset minimum positive number.

[0059] Or it can be obtained through the following formula:

[0060] in, For globally labeled data The number of image samples, The total number of categories, For the first in the globally labeled data Number of image samples for each category.

[0061] Or it can be obtained through the following formula:

[0062] in, This is the preset first temperature coefficient.

[0063] Specifically, this implementation provides three optional loss weight calculation formulas, each corresponding to a different class imbalance correction strategy. The first is based on the logarithmic reciprocal of the class sample size ratio. Explained, in this formula, Indicates category The percentage of a category in the globally labeled data, ranging from 0 to 1. When the percentage of samples from a certain category is very small, It is also correspondingly smaller, therefore its reciprocal The weight is relatively large, thus achieving the goal of assigning large weights to small sample categories. It is a preset, extremely small positive number used to prevent when The division error occurs when the denominator is zero as the value approaches zero.

[0064] The second method normalizes based on the reciprocal of the number of samples in each category. (Interpretive) This directly reflects the reciprocal of the number of samples in each category; the smaller the sample size, the larger the reciprocal. To keep the sum of the weights of all categories on a reasonable scale, this formula is further multiplied by... Normalization is performed to ensure that the sum of the loss weights for all classes equals the total number of classes. .

[0065] The third method uses exponential decay plus normalization. (Explanatory) middle, The preset first temperature coefficient is used to control the rate of exponential decay. The larger the value, the more sensitive the sample size is to the influence of the weights. This formula achieves a negative correlation between the weight factors and the sample size through an exponential function, and then normalizes the weight factors of all categories by summing the denominators, so that the sum of the loss weights of all categories is 1.

[0066] All three methods can effectively achieve weighted compensation for tail categories. In practical applications, the appropriate method can be selected based on the specific data distribution characteristics and training requirements.

[0067] In one possible implementation, the consistency loss function is:

[0068] in, This represents the consistency loss value. For the first Model parameters of an image classification model with unlabeled training methods. For the first The total number of local unlabeled image samples for each unlabeled training method. For indicator functions, The value is 1 if the condition is met, and 0 otherwise. For image classification models The maximum value of the predicted probability distribution of the weakly enhanced samples. To preset the reliability threshold, Let cross-entropy be the loss function. for pseudo-tags For image classification models in The predicted output is a category. The probability, For the first Image samples, for Strongly enhanced image samples, This is a category index variable.

[0069] Among them, the Weakly enhanced samples by The weak enhancement operation is performed, which is a random horizontal flip and / or random cropping; Passing through The result is obtained by performing a strong enhancement operation, which is an automatic search enhancement strategy.

[0070] Specifically, this implementation constructs a consistency loss function based on the FixMatch algorithm. The core idea of ​​this loss function is to generate high-confidence pseudo-labels using weakly augmented views, and then use these pseudo-labels to supervise the prediction of strongly augmented views. For each unlabeled image sample... First, weakly enhanced and strongly enhanced samples are generated. Weak enhancement operations include random horizontal flipping and random cropping. These operations cause minimal changes to the original image, preserving its original semantic information; therefore, the model's predictions for weakly enhanced samples have high reliability. Strong enhancement operations employ an automatic search enhancement strategy. This strategy uses reinforcement learning to automatically search for the optimal combination of enhancement strategies, applying more drastic transformations to the image (such as color perturbations and geometric transformations), thereby effectively expanding data diversity and improving the model's generalization ability.

[0071] Specifically, weakly augmented samples are input into the current local model to obtain the predicted probability distribution. Calculate the maximum value of this distribution. and compared with the preset confidence threshold Comparison. Only when... Only when the maximum value is reached is the model's prediction for that image sample considered reliable, and the category corresponding to that maximum value is then used as the pseudo-label for that sample. .if This indicates that the model's prediction confidence for this image sample is insufficient, and this image sample is ignored in this round of training and not included in the loss calculation. Optionally, a pre-set confidence threshold can be used. The value can be set to 0.9~0.99. This high threshold ensures that only image samples that the model is very confident in will be used for training, effectively filtering out low-quality pseudo-labels and avoiding misleading the model training with incorrect labels.

[0072] For image samples that meet the confidence criteria, the strongly enhanced samples will be... Input the model to obtain the predicted probability distribution Then calculate the distribution and the pseudo-label. The model in question uses cross-entropy loss. By minimizing this loss, the model is forced to produce consistent predictions for different augmented samples of the same original image. This consistency constraint enables the model to learn invariant feature representations from a large amount of unlabeled image data, thus fully utilizing the supervision information in the unlabeled data.

[0073] In one possible implementation, obtaining the model parameter update amounts obtained by each training party based on local training, and aggregating them based on labeled and unlabeled training parties to obtain labeled aggregation results and unlabeled aggregation results includes: The annotation aggregation result is obtained using the following formula. :

[0074] in, For the first The model parameters of the image classification model after rounds of federated learning. To label the total number of training sides, For the first The total number of locally annotated image samples for each annotation training method. For the first The total number of locally annotated image samples for each annotation training method. For the first The number of model parameter updates for each labeled training method.

[0075] The unlabeled aggregation result is obtained using the following formula. :

[0076] in, This represents the total number of unlabeled training units. For the first The total number of local unlabeled image samples for each unlabeled training method. For the first The total number of local unlabeled image samples for each unlabeled training method. For the first The number of model parameter updates uploaded by each unlabeled training party.

[0077] Specifically, this implementation provides the detailed calculation methods for labeled aggregation results and unsupervised aggregation results. Explained, in each round of communication in federated learning, the server first maintains a global model parameter set at the end of the previous round of communication. Each training party in Based on this, local training is performed, and the update amount of each model parameter is calculated. For the labeled training, the update amount is... Indicates the first The changes in model parameters after local supervised training for labeled trainees; the update amount for unlabeled trainees. Indicates the first The change in model parameters after local unsupervised training of an unlabeled training method.

[0078] Specifically, during the aggregation process, the server performs a weighted average of the model parameter updates for both types of training. Taking the aggregation of labeled training as an example, the weight coefficient is... , of which molecules It is the first The total number of locally labeled image samples for each annotation training method, denominator It is the sum of the total number of locally labeled image samples from all labeled training parties. Therefore, this weight coefficient actually represents the first... The proportion of local samples from each labeled training instance to the total number of samples from all labeled training instances. Training instances with more image samples have a greater weight in the aggregation of their model parameter updates. This is based on statistical reliability considerations; training instances with more samples typically provide more stable and reliable gradient estimates. The weighted updates from all weighted coefficients are summed and then added... This yields the results of this round of supervision aggregation. Unsupervised aggregation results The calculation method is similar, except that it uses the number of local unlabeled image samples from the unlabeled training side as the weighting criterion. Through this hierarchical aggregation mechanism, supervised and unsupervised signals are decoupled into two independent aggregation results, laying the foundation for subsequent dynamic weighted fusion.

[0079] In one possible implementation, obtaining and evaluating the learning performance of labeled and unlabeled training participants includes: for each labeled training participant, calculating the average loss value based on the label distribution-aware alignment loss function before and after local training, and dividing the difference between the average loss value before and after local training by the average loss value before local training to obtain the learning performance evaluation value of the labeled training participant; or, taking an exponential negative mapping of the average loss value after local training to obtain the learning performance evaluation value of the labeled training participant; or, by statistically analyzing the prediction confidence of the image classification model based on local training in locally labeled image samples that is greater than a preset confidence threshold. The proportion of image samples to the number of locally labeled image samples is used as the learning performance evaluation value for each labeled training agent. For each unlabeled training agent, the proportion of image samples in the local unlabeled image samples whose prediction confidence based on the locally trained image classification model is greater than a preset confidence threshold is used as the learning performance evaluation value for each unlabeled training agent. The learning performance evaluation values ​​of each labeled training agent are weighted and averaged based on the number of image samples to obtain the learning performance evaluation value for the labeled training agent. The learning performance evaluation values ​​of each unlabeled training agent are weighted and averaged based on the number of image samples to obtain the learning performance evaluation value for the unlabeled training agent.

[0080] Interpretive learning performance evaluation values ​​are used to quantify the learning effectiveness of each training participant in this round of training and are the core basis for subsequent dynamic weight calculation.

[0081] For example, this implementation provides three optional methods for calculating the learning performance evaluation value for the labeled training side. The first method is based on the magnitude of loss reduction. Specifically, before local training begins, the average loss value is first calculated on the local labeled data using the current local model. After several rounds of local iterative training, the updated local model is used again to calculate the average loss value on the same locally labeled data. Then calculate the magnitude of the loss decrease. This ratio reflects the relative decrease in the model's loss value due to this round of training. A larger decrease in loss indicates a more significant effect from this round of training, and a higher evaluation value for the learning effect. The second method is based on the loss value after training. Specifically, it directly uses the average loss value after local training. Through exponential negative mapping Calculate the evaluation value. Since the exponential function is a monotonically decreasing function, the smaller the loss value (indicating a better model fit), the closer the evaluation value is to 1; the larger the loss value, the closer the evaluation value is to 0. The third method is based on the proportion of high-confidence prediction samples. Specifically, for each image sample in the locally labeled data, the maximum Softmax probability value in the model's prediction results for that image sample is the prediction confidence. Count the number of image samples with prediction confidence greater than a preset confidence threshold, and calculate the proportion of this number to the total number of locally labeled image samples. The higher this proportion, the stronger the model's prediction certainty for the labeled data, and the higher the learning performance evaluation value.

[0082] For unlabeled training samples, since there are no real labels available for verification, this implementation method uses a method based on the proportion of high-confidence pseudo-label samples to calculate the learning performance evaluation value. Specifically, for each image sample in the local unlabeled data, the maximum value of the model's prediction probability distribution for its weakly augmented view is the prediction confidence. The number of image samples with prediction confidence greater than a preset confidence threshold is counted, and the proportion of this number to the total number of local unlabeled image samples is calculated. The higher this proportion, the stronger the model's prediction certainty for the unlabeled data, the higher the quality of the generated pseudo-labels, and the higher the learning performance evaluation value.

[0083] After obtaining the individual learning performance evaluation values ​​for each training participant, the server further calculates the learning performance evaluation values ​​for both types of training participants. Specifically, for the labeled training participants, the individual evaluation values ​​of each labeled training participant are weighted and averaged using the number of local image samples as the weight, to obtain the total learning performance evaluation value for the labeled training participants. For unlabeled trainees, the same weighted averaging method was used to obtain the learning performance evaluation value for unlabeled trainees. This weighted averaging method ensures that the training sample with a larger number of images contributes more to the evaluation value of the learning effect of the corresponding type of training sample, reflecting the principle of statistical reliability.

[0084] In one possible implementation, the weighted aggregation of labeled and unlabeled aggregation results includes: weighting the labeled and unlabeled aggregation results using the following formula:

[0085]

[0086]

[0087] in, For the first The model parameters of the image classification model after rounds of federated learning. To label the fusion weights of the aggregation results, To label the aggregation results, The fusion weights for unlabeled aggregation results. For unlabeled aggregation results, To label the learning performance evaluation values ​​of the trainer, The learning performance evaluation value for the unlabeled training method. This is the preset second temperature coefficient.

[0088] Interpretive, after obtaining the supervised aggregation results Unsupervised aggregation results And the learning performance evaluation values ​​of the two types of training methods. and Next, the server needs to merge the two aggregation results into a unified global model parameter. Specifically, the fusion weights are calculated using a Softmax function with a temperature coefficient to supervise the fusion weights of the aggregation results. For example, its calculation formula is: Explained, this formula maps two overall learning performance evaluation values ​​to the positive real number domain using an exponential function, then normalizes them so that the sum of the two weights equals 1. When the overall learning performance evaluation value of the supervised trainer... Much greater than the overall learning performance evaluation value of unlabeled training methods. hour, Much larger ,at this time When the value approaches 1, the global model tends to trust the results of supervised aggregation; conversely, when the overall learning performance evaluation value of the unlabeled trainers approaches 1, the model tends to trust the results of supervised aggregation. Much larger hour, As the value approaches 1, the global model tends to trust the results of unsupervised aggregation. Among these, the second temperature coefficient... Its function is to control the smoothness of the weight distribution. When When the value is large, the amplification effect of the exponential function on the difference in evaluation values ​​weakens, and the two weights tend to be equal (each 0.5); when When the value is small, the exponential function amplifies the difference in evaluation values, with the side with a higher evaluation value receiving a weight close to 1, and the side with a lower evaluation value receiving a weight close to 0. This can be mitigated by setting the value appropriately. The calculated fusion weights prevent weights from saturating too quickly due to fluctuations in a single training round, making the fusion process smoother and more stable. Finally, the supervised and unsupervised aggregation results are weighted and summed according to the calculated fusion weights to obtain the global model parameters for this communication round. After that, the server will The data was distributed to all training participants, and the next round of training iterations began.

[0089] Through this dynamic high-level aggregation mechanism, the present invention achieves an adaptive balance between supervised and unsupervised signals in global model construction, significantly enhancing the robustness of image classification models in heterogeneous data environments.

[0090] Experimental results demonstrate that, on multiple real-world image classification datasets such as CIFAR-10 and SVHN, under conditions simulating varying degrees of data heterogeneity and label imbalance using the Dirichlet distribution, the proposed method exhibits significantly superior robustness compared to existing mainstream methods. Specifically, under extreme conditions of high imbalance, the proposed method achieves a test accuracy improvement of over ten percentage points compared to the traditional federated average baseline method, with a significantly smaller decrease in accuracy as the imbalance increases. Furthermore, ablation experiments further demonstrate that both the label distribution-aware alignment loss and the dynamic high-level aggregation, the two core components, can independently positively improve baseline performance, achieving optimal results when working together.

[0091] The above experiments demonstrate that the present invention achieves the expected goals well through a refined supervised target correction and dynamic aggregation weight allocation strategy. Compared with defense methods that require complex global data resampling or adversarial example generation on the server side, the present invention has significant advantages in terms of computational overhead and communication efficiency, and is more suitable for the actual deployment of large-scale federated semi-supervised learning systems.

[0092] The following are embodiments of the apparatus of the present invention, which can be used to execute embodiments of the method of the present invention. For details not disclosed in the apparatus embodiments, please refer to the embodiments of the method of the present invention.

[0093] See Figure 3 In another embodiment of the present invention, a federated learning training system for an image classification model is provided to implement the above-described federated learning training method for an image classification model. Specifically, the federated learning training system for an image classification model includes a model distribution module, a local training module, a local aggregation module, and a global aggregation module.

[0094] The model distribution module distributes the image classification model to be trained to each training party in the federated learning process. The local training module allows each training party to train the image classification model locally using local data. When the training party is labeled, a label distribution-aware alignment loss function is used to calculate the loss value; when the training party is unlabeled, a consistency loss function is used. In the label distribution-aware alignment loss function, the category with fewer samples corresponds to a larger loss weight. The consistency loss function is constructed based on the prediction consistency of different augmented views of the same unlabeled sample. The local aggregation module obtains the model parameter update amounts obtained by each training party based on local training and aggregates them based on labeled and unlabeled training parties to obtain labeled and unlabeled aggregation results. The global aggregation module obtains and weights the labeled and unlabeled aggregation results based on the learning performance evaluation values ​​of labeled and unlabeled training parties to obtain a global aggregation result. The image classification model is then updated based on the global aggregation result to obtain the trained image classification model. A higher learning performance evaluation value corresponds to a smaller weighted aggregation weight.

[0095] In one possible implementation, the label distribution-aware alignment loss function is:

[0096] in, For label distribution-aware alignment loss, For the first Model parameters of an image classification model trained by an labeled training method; For the first The total number of locally annotated image samples for each annotation training method; For the first The true category of each image sample The corresponding loss weights are as follows: the smaller the number of samples for a category, the greater the loss weight. The cross-entropy loss function; For the first The true category label of each image sample; For input The predicted output is a category. The probability of; For category index variables; For the first Image samples.

[0097] In one possible implementation, the We obtain it from the following formula:

[0098] in, For globally labeled data The proportion of image samples, It is a preset minimum positive number.

[0099] Or it can be obtained through the following formula:

[0100] in, For globally labeled data The number of image samples, The total number of categories, For the first in the globally labeled data Number of image samples for each category.

[0101] Or it can be obtained through the following formula:

[0102] in, This is the preset first temperature coefficient.

[0103] In one possible implementation, the consistency loss function is:

[0104] in, This represents the consistency loss value. For the first Model parameters of an image classification model with unlabeled training methods. For the first The total number of local unlabeled image samples for each unlabeled training method. For indicator functions, The value is 1 if the condition is met, and 0 otherwise. For image classification models The maximum value of the predicted probability distribution of the weakly enhanced samples. To preset the reliability threshold, Let cross-entropy be the loss function. for pseudo-tags For image classification models in The predicted output is a category. The probability, For the first Image samples, for Strongly enhanced image samples, For category index variables; where, the Weakly enhanced samples by The weak enhancement operation is performed, which is a random horizontal flip and / or random cropping; Passing through The result is obtained by performing a strong enhancement operation, which is an automatic search enhancement strategy.

[0105] In one possible implementation, obtaining the model parameter update amounts obtained by each training party based on local training, and aggregating them based on labeled and unlabeled training parties to obtain labeled and unlabeled aggregation results includes: obtaining the labeled aggregation result using the following formula. :

[0106] in, For the first The model parameters of the image classification model after rounds of federated learning. To label the total number of training sides, For the first The total number of locally annotated image samples for each annotation training method. For the first The total number of locally annotated image samples for each annotation training method. For the first The number of model parameter updates for each labeled training method.

[0107] The unlabeled aggregation result is obtained using the following formula. :

[0108] in, This represents the total number of unlabeled training units. For the first The total number of local unlabeled image samples for each unlabeled training method. For the first The total number of local unlabeled image samples for each unlabeled training method. For the first The number of model parameter updates uploaded by each unlabeled training party.

[0109] In one possible implementation, obtaining and evaluating the learning performance of labeled and unlabeled training participants includes: for each labeled training participant, calculating the average loss value based on the label distribution-aware alignment loss function before and after local training, and dividing the difference between the average loss value before and after local training by the average loss value before local training to obtain the learning performance evaluation value of the labeled training participant; or, taking an exponential negative mapping of the average loss value after local training to obtain the learning performance evaluation value of the labeled training participant; or, by statistically analyzing the prediction confidence of the image classification model based on local training in locally labeled image samples that is greater than a preset confidence threshold. The proportion of image samples to the number of locally labeled image samples is used as the learning performance evaluation value for each labeled training agent. For each unlabeled training agent, the proportion of image samples in the local unlabeled image samples whose prediction confidence based on the locally trained image classification model is greater than a preset confidence threshold is used as the learning performance evaluation value for each unlabeled training agent. The learning performance evaluation values ​​of each labeled training agent are weighted and averaged based on the number of image samples to obtain the learning performance evaluation value for the labeled training agent. The learning performance evaluation values ​​of each unlabeled training agent are weighted and averaged based on the number of image samples to obtain the learning performance evaluation value for the unlabeled training agent.

[0110] In one possible implementation, the weighted aggregation of labeled and unlabeled aggregation results includes: weighting the labeled and unlabeled aggregation results using the following formula:

[0111]

[0112]

[0113] in, For the first The model parameters of the image classification model after rounds of federated learning. To label the fusion weights of the aggregation results, To label the aggregation results, The fusion weights for unlabeled aggregation results. For unlabeled aggregation results, To label the learning performance evaluation values ​​of the trainer, The learning performance evaluation value for the unlabeled training method. This is the preset second temperature coefficient.

[0114] All relevant content of each step involved in the aforementioned embodiments of the federated learning training method for image classification models can be referenced from the functional description of the corresponding functional module of the federated learning training system for image classification models in the embodiments of the present invention, and will not be repeated here.

[0115] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0116] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of a federated learning training method for image classification models.

[0117] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the federated learning training method for the image classification model in the above embodiments.

[0118] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A federated learning training method for an image classification model, characterized in that, include: The image classification model to be trained is distributed to each training party in the federated learning process; Each training party trains the image classification model locally based on local data. When the training party is labeled, the label distribution-aware alignment loss function is used to calculate the loss value; when the training party is unlabeled, the consistency loss function is used to calculate the loss value. In the label distribution-aware alignment loss function, the category with fewer samples has a larger loss weight. The consistency loss function is constructed based on the prediction consistency of different augmented views of the same unlabeled sample. Obtain the model parameter update amount obtained by each training party based on local training, and aggregate them based on labeled and unlabeled training parties to obtain labeled aggregation results and unlabeled aggregation results; The learning performance evaluation values ​​of labeled and unlabeled trainees are obtained and weighted to obtain a global aggregation result. The image classification model is then updated based on the global aggregation result to obtain the trained image classification model. The larger the learning performance evaluation value, the smaller the corresponding weighted aggregation weight.

2. The federated learning training method for image classification models according to claim 1, characterized in that, The label distribution-aware alignment loss function is: in, For label distribution-aware alignment loss, For the first Model parameters of an image classification model trained by an labeled training method; For the first The total number of locally annotated image samples for each annotation training method; For the first The true category of each image sample The corresponding loss weights are as follows: the smaller the number of samples for a category, the greater the loss weight. The cross-entropy loss function; For the first The true category label of each image sample; For input The predicted output is a category. The probability of; For category index variables; For the first Image samples.

3. The federated learning training method for image classification models according to claim 2, characterized in that, The We obtain it from the following formula: in, For globally labeled data The proportion of image samples, The minimum positive number is preset; Or it can be obtained through the following formula: in, For globally labeled data The number of image samples, The total number of categories, For the first in the globally labeled data Number of image samples in each category; Or it can be obtained through the following formula: in, This is the preset first temperature coefficient.

4. The federated learning training method for image classification models according to claim 1, characterized in that, The consistency loss function is: in, This represents the consistency loss value. For the first Model parameters of an image classification model with unlabeled training methods. For the first The total number of local unlabeled image samples for each unlabeled training method. For indicator functions, The value is 1 if the condition is met, and 0 otherwise. For image classification models The maximum value of the predicted probability distribution of the weakly enhanced samples. To preset the reliability threshold, Let cross-entropy be the loss function. for pseudo-tags For image classification models in The predicted output is a category. The probability, For the first Image samples, for Strongly enhanced image samples, For category index variables; Among them, the Weakly enhanced samples by The weak enhancement operation is performed, which is a random horizontal flip and / or random cropping; Passing through The result is obtained by performing a strong enhancement operation, which is an automatic search enhancement strategy.

5. The federated learning training method for image classification models according to claim 1, characterized in that, The step of obtaining the model parameter update amounts obtained by each training party based on local training, and aggregating them based on labeled and unlabeled training parties to obtain labeled aggregation results and unlabeled aggregation results includes: The annotation aggregation result is obtained using the following formula. : in, For the first The model parameters of the image classification model after rounds of federated learning. To label the total number of training sides, For the first The total number of locally annotated image samples for each annotation training method. For the first The total number of locally annotated image samples for each annotation training method. For the first The amount of model parameter updates for each labeled training method; The unlabeled aggregation result is obtained using the following formula. : in, This represents the total number of unlabeled training units. For the first The total number of local unlabeled image samples for each unlabeled training method. For the first The total number of local unlabeled image samples for each unlabeled training method. For the first The number of model parameter updates uploaded by each unlabeled training party.

6. The federated learning training method for image classification models according to claim 1, characterized in that, The acquisition and evaluation of learning performance based on labeled and unlabeled training agents includes: For each annotation training agent, calculate the average loss value based on the label distribution-aware alignment loss function before and after local training. Divide the difference between the average loss value before and after local training by the average loss value before local training to obtain the learning performance evaluation value of the annotation training agent; or, take the exponential negative mapping of the average loss value after local training to obtain the learning performance evaluation value of the annotation training agent; or, use the proportion of image samples in the local annotated image samples whose prediction confidence based on the image classification model after local training is greater than a preset confidence threshold to the total number of local annotated image samples as the learning performance evaluation value of each annotation training agent. For each unlabeled training agent, the learning performance evaluation value is calculated by the proportion of image samples in the local unlabeled image samples whose prediction confidence based on the locally trained image classification model is greater than a preset confidence threshold to the total number of local unlabeled image samples. The learning performance evaluation values ​​of each labeled training agent are weighted and averaged based on the number of image samples to obtain the learning performance evaluation value of the labeled training agent; and the learning performance evaluation values ​​of each unlabeled training agent are weighted and averaged based on the number of image samples to obtain the learning performance evaluation value of the unlabeled training agent.

7. The federated learning training method for image classification models according to claim 1, characterized in that, The weighted aggregation of labeled and unlabeled aggregation results includes: The weighted aggregation results of labeled and unlabeled aggregation results are weighted and aggregated using the following formula: in, For the first The model parameters of the image classification model after rounds of federated learning. To label the fusion weights of the aggregation results, To label the aggregation results, The fusion weights for unlabeled aggregation results. For unlabeled aggregation results, To label the learning performance evaluation values ​​of the trainer, The learning performance evaluation value for the unlabeled training method. This is the preset second temperature coefficient.

8. A federated learning training system for an image classification model, characterized in that, include: The model distribution module is used to distribute the image classification model to be trained to the various training parties in federated learning. The local training module is used by each training party to train the image classification model locally based on local data. When the training party is labeled, the label distribution-aware alignment loss function is used to calculate the loss value; when the training party is unlabeled, the consistency loss function is used. In the label distribution-aware alignment loss function, the category with fewer samples corresponds to a larger loss weight. The consistency loss function is constructed based on the prediction consistency of different augmented views of the same unlabeled sample. The local aggregation module is used to obtain the model parameter update amount obtained by each training party based on local training, and to aggregate based on labeled and unlabeled training parties to obtain labeled aggregation results and unlabeled aggregation results. The global aggregation module is used to obtain and weight the aggregated results of labeled and unlabeled training participants based on their learning performance evaluation values ​​to obtain a global aggregation result. The image classification model is then updated based on the global aggregation result to obtain the trained image classification model. The larger the learning performance evaluation value, the smaller the corresponding weighted aggregation weight.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the federated learning training method for the image classification model as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the federated learning training method for the image classification model as described in any one of claims 1 to 7.

Images