A longitudinal federated learning model optimization method based on semi-aligned data

Abstract

This application discloses a method for optimizing a longitudinal federated learning model based on semi-aligned data. By introducing a longitudinal federated learning framework with semi-aligned samples, this method significantly improves the model's training efficiency and accuracy while protecting data privacy. This effectively addresses the problem of large amounts of misaligned data being wasted in traditional longitudinal federated learning due to low sample alignment rates. While ensuring data privacy and security, this method achieves efficient utilization of misaligned samples, addressing the problem of limited model performance in low-sample scenarios.

Description

Technical Field

[0001] The present application relates to the technical field of vertical federated learning, and in particular to a method for optimizing a vertical federated learning model based on semi-aligned data. Background Art

[0002] Federated learning is a new machine learning approach proposed by Google, designed to enable multiple clients to collaboratively build machine learning models without sharing data. Under this framework, clients can train models locally, keeping sensitive data local. By uploading trained models or parameters, model training can be continued without compromising privacy. Federated learning effectively addresses the "data silo" problem.

[0003] Although federated learning can integrate data from multiple different participants for model training, it still faces some challenges in the vertical federated learning framework.

[0004] In vertical federated learning, a key prerequisite for model training is finding shared training sample individuals. However, as the number of participating clients increases, the number of shared client individuals decreases significantly. This phenomenon results in a decrease in the number of available samples despite the increase in the number of participants.

[0005] Furthermore, since each participant possesses unique attributes and the model trainer cannot obtain specific information about the participants before training begins, balancing the sample size and number of features in the training data becomes extremely difficult during longitudinal model training. Consequently, a large number of samples remain unused in the longitudinal federated learning framework.

[0006] In vertical federated learning, due to the small number of aligned samples, a large number of unaligned samples are underutilized. This problem is exacerbated when vertical federated learning is combined with other few-shot learning methods. Each participant's local model typically has few sample features, and most participants also face a serious lack of labels. At the same time, in the process of vertical federated learning, special attention must be paid to user privacy protection. Enhancing the model's ability to utilize unused samples to improve global model accuracy without infringing on the original privacy protection budget has become a major challenge in current research. Summary of the Invention

[0007] The embodiment of the present application provides a longitudinal federated learning model optimization method based on semi-aligned data, which can expand the sample data set for the longitudinal federated learning model, thereby improving the accuracy of model training.

[0008] In order to achieve the above-mentioned purpose, the technical solution of the embodiment of the present invention is:

[0009] An embodiment of the present invention provides a longitudinal federated learning model optimization method based on semi-aligned data, comprising: based on aligned samples between each participant, using a local feature extractor of each participant to obtain intermediate features of the aligned samples; uploading the intermediate features to a server, and the server jointly training a feature transformer based on the intermediate features to learn the correlation between the intermediate features of different participants; the server fusing the intermediate features of all participants to generate a global feature representation, and optimizing the inter-class discrimination through supervised comparative learning, and jointly updating the local models of each participant; extracting intermediate features of unaligned samples through the feature extractor, and iteratively generating pseudo labels for the unaligned samples through a semi-supervised clustering method based on the feature similarity between the unaligned samples and the aligned samples; obtaining proxy features of the labeled participants through the feature transformer, calculating the feature similarity between the proxy features and the unaligned samples, and selecting the unaligned samples with the highest similarity as semi-aligned supplementary samples; setting a distance threshold based on the quartile method, eliminating low-confidence samples, and retaining high-quality semi-aligned samples; adding the screened high-quality semi-aligned samples to the training data set, and iteratively optimizing the longitudinal federated learning model.

[0010] In some possible implementations, the aligned samples between the participants are obtained through a private intersection method.

[0011] In some possible implementations, the feature transformer is represented as:

[0012] ;

[0013] in, represents the feature change loss, Indicates assignment, Indicates the task party, represents the loss function, Indicates the samples, Indicates auxiliary party, Indicates auxiliary samples, It represents the result of local feature extraction of the task side sample. Indicates the result of local feature extraction of auxiliary samples.

[0014] In some possible implementations, the server fuses the intermediate features of all participants to generate a global feature representation, optimizes inter-class discrimination through supervised contrastive learning, and jointly updates the local models of each participant, including:

[0015] The server concatenates the intermediate features of each participant, expressed as:

[0016] ;

[0017] in, Represents the features after splicing, Indicates the task party, represents the aligned samples, Indicates auxiliary party, It represents the result of local feature extraction of the task side’s aligned samples. Represents the result of local feature extraction of auxiliary alignment samples;

[0018] Map the overall features after splicing and express it as:

[0019] ;

[0020] in, represents the mapping function, Represents the mapped features;

[0021] The mapped features are used as the input of supervised contrastive learning to enhance the distinction between classes, which can be expressed as:

[0022] ;

[0023] in, represents all positive samples in the same category, Represents negative samples in different classes, represents the temperature coefficient, represents the total number of all positive samples, represents the contrast loss, represents a positive sample, An index set representing a sample;

[0024] Combining the feature transformer with the loss function in supervised contrastive learning is expressed as:

[0025] ;

[0026] in, represents the hyperparameter, Indicates overall loss;

[0027] The gradient of each client is calculated according to the loss and returned to the corresponding participant, expressed as:

[0028] ;

[0029] in, represents the gradient of the auxiliary side, represents the gradient calculation, Indicates auxiliary party embedding;

[0030] Each participant updates the local model weights using the obtained gradient, expressed as:

[0031] ;

[0032] in, represents the model parameters of the i-th participant, Represents the learning rate.

[0033] In some possible implementations, extracting intermediate features of the misaligned samples by the feature transformer, and iteratively generating pseudo labels for the misaligned samples by a semi-supervised clustering method based on feature similarity between the misaligned samples and the aligned samples, including:

[0034] The feature extractor extracts the intermediate features of the unaligned samples of all unlabeled participants, which are expressed as:

[0035] ;

[0036] in, represents the intermediate embedding of unaligned samples, represents misaligned samples, Indicates misalignment;

[0037] According to the distance between the features of the unaligned samples and the existing aligned samples, the closest distance is iteratively calculated to generate a pseudo label for it. In the iterative process, the labels of the existing aligned samples are kept unchanged, and the samples with missing labels are continuously iteratively transformed according to the distance to the category center, which can be expressed as:

[0038] ;

[0039] in, Representation sample Go to Category Center distance, Indicates the embedded eigenvalues, Indicates the Category centers, Represents the eigenvalue.

[0040] In some possible implementations, obtaining the proxy features of the labeled participants through the feature transformer, calculating the feature similarity between the proxy features and the unaligned samples, and selecting the unaligned sample with the highest similarity as the semi-aligned supplementary sample includes:

[0041] The required supplemented labeled samples are passed through the transformer to obtain the agent features of the labeled participants, which are expressed as:

[0042] ;

[0043] in, Indicates the The proxy embedding obtained by transforming samples is represents the converter parameters, Indicates samples that the task party needs to supplement;

[0044] Compare the features of the unaligned samples with the proxy features to obtain a set of supplementary sample indexes, and calculate their distance to obtain the unaligned sample with the highest similarity as the semi-aligned supplementary sample, which is expressed as:

[0045] ;

[0046] in, Indicates a converter.

[0047] In some possible implementations, a distance threshold is set based on the quartile method to remove low-confidence samples and retain high-quality semi-aligned samples, which can be expressed as:

[0048] ;

[0049] ;

[0050] ;

[0051] ;

[0052] in, represents the upper quartile, Indicates percentage, represents the minimum distance set, represents the lower quartile, represents the interquartile range, represents the threshold value, represents a hyperparameter.

[0053] One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:

[0054] In this embodiment of the present invention, by introducing a longitudinal federated learning framework with semi-aligned samples, the model's training efficiency and accuracy are significantly improved while protecting data privacy. This effectively addresses the problem of large amounts of misaligned data wasted in traditional longitudinal federated learning due to low sample alignment rates. This method achieves efficient utilization of misaligned samples while ensuring data privacy and security, addressing the problem of limited model performance in low-sample scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 A schematic flow chart of an embodiment of a method for optimizing a longitudinal federated learning model based on semi-aligned data provided for the implementation of the present invention;

[0057] Figure 2 The following is a comparison chart of the accuracy of the present invention and other methods on the MNIST dataset;

[0058] Figure 3 The following figure shows the accuracy comparison between the present invention and other methods on the NUS-WIDE dataset;

[0059] Figure 4 The following is a comparison chart of the accuracy of the present invention and other methods on the CIFAR10 dataset;

[0060] Figure 5 This is a comparison chart of the accuracy of the present invention and other methods under the MNIST dataset and different imbalance ratios;

[0061] Figure 6 This is a comparison chart of the accuracy of the present invention and other methods under the NUS-WIDE dataset and different imbalance ratios;

[0062] Figure 7 This is a comparison chart of the accuracy of the present invention and other methods under the CIFAR10 dataset and different imbalance ratios;

[0063] Figure 8 The accuracy diagram of the present invention under different data sets and different thresholds;

[0064] Figure 9 The accuracy graph of the present invention under different data sets and different ratios of the number of misaligned samples;

[0065] Figure 10 The accuracy diagram of the present invention under different data sets and the ratio of the number of misaligned samples. DETAILED DESCRIPTION

[0066] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0067] In the relevant description of this embodiment, the terms "including, containing, having" and the like are open terms and are generally understood to include but not be limited to; the term "at least one" is generally understood to mean one or more, where "plurality" refers to two or more; the term "at least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items, for example, "at least one of a, b or c", or "at least one of a, b and c", can all represent: a, b, c, ab (i.e., a and b), ac, bc, or abc, where a, b, c can be single or multiple respectively; the symbol "A / B" is used to describe the selection relationship of associated objects, generally indicating an "or" relationship before and after.

[0068] In the following description of the present embodiment, the terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a", "an" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0069] Those skilled in the art should understand that in the following description of the embodiments of the present application, the order of serial numbers does not mean the order of execution, some or all of the steps can be executed in parallel or sequentially, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0070] It will be understood by those skilled in the art that the numerical ranges in the examples of the present application are to be understood as also specifically disclosing each intermediate value between the upper and lower limits of the ranges. Each smaller range between the intermediate value in any stated value or stated range and any other stated value or intermediate value in the range is also included in the present invention. The upper and lower limits of these smaller ranges may be independently included or excluded in the scope.

[0071] Unless otherwise indicated, the technical / scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which this application belongs. Although this application describes only preferred methods and materials, any methods and materials similar or equivalent to those herein may also be used in the implementation or testing of this application. All documents mentioned in this specification are incorporated by reference to disclose and describe the methods and / or materials related to the documents. In the event of any conflict with any incorporated document, the content of this specification shall prevail.

[0072] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0073] Federated learning is a new machine learning approach proposed by Google, designed to enable multiple clients to collaboratively build machine learning models without sharing data. Under this framework, clients can train models locally, keeping sensitive data local. By uploading trained models or parameters, model training can be continued without compromising privacy. Federated learning effectively addresses the "data silo" problem.

[0074] Although federated learning can integrate data from multiple different participants for model training, it still faces some challenges in the vertical federated learning framework.

[0075] In vertical federated learning, a key prerequisite for model training is finding shared training sample individuals. However, as the number of participating clients increases, the number of shared client individuals decreases significantly. This phenomenon results in a decrease in the number of available samples despite the increase in the number of participants.

[0076] Furthermore, since each participant possesses unique attributes and the model trainer cannot obtain specific information about the participants before training begins, balancing the sample size and number of features in the training data becomes extremely difficult during longitudinal model training. Consequently, a large number of unused samples exist in the longitudinal federated learning framework.

[0077] In vertical federated learning, due to the small number of aligned samples, a large number of unaligned samples are underutilized. This problem is exacerbated when vertical federated learning is combined with other few-shot learning methods. Each participant's local model typically has few sample features, and most participants also face a serious lack of labels. At the same time, in the process of vertical federated learning, special attention must be paid to user privacy protection. Enhancing the model's ability to utilize unused samples to improve global model accuracy without infringing on the original privacy protection budget has become a major challenge in current research.

[0078] Based on this, an embodiment of the present invention provides a longitudinal federated learning model optimization method based on semi-aligned data, which can expand the sample data set for the longitudinal federated learning model to improve the accuracy of model training.

[0079] Figure 1 A flow chart of an embodiment of a longitudinal federated learning model optimization method based on semi-aligned data provided for the implementation of the present invention is shown in FIG. Figure 1 As shown, the above method may include:

[0080] S101, based on the aligned samples between the participants, using the local feature extractor of each participant to obtain the intermediate features of the aligned samples;

[0081] It should be noted that the embodiment of the present invention is mainly used in the vertical federated learning framework. Specifically, in this case, there are clients, and each client holds a local dataset ,in Representing each specific participant The participants with labels are identified as , the remaining participants are identified as There is an aligned sample dataset in the vertical federated learning framework , unaligned sample dataset , there are a total of r rounds of iterations.

[0082] In some embodiments, the aligned samples between the participants are obtained by the privacy intersection method. Each participant locally trains the feature extractor to obtain the local model of each round, which is expressed as The intermediate features of the corresponding aligned samples are obtained through the local model of each participant.

[0083] S102, uploading the intermediate features to the server, and the server jointly training the feature transformer based on the intermediate features to learn the correlation between the intermediate features of different participants;

[0084] After receiving the intermediate features of each participant, the server jointly trains a feature transformer between the intermediate feature results of different participants. For example, the feature transformer is represented as:

[0085] ;

[0086] in, represents the feature change loss, Indicates assignment, Indicates the task party, represents the loss function, Indicates the samples, Indicates auxiliary party, Indicates auxiliary samples, It represents the result of local feature extraction of the task side sample. Indicates the result of local feature extraction of auxiliary samples.

[0087] S103: The server fuses the intermediate features of all participants to generate a global feature representation, optimizes the inter-class discrimination through supervised contrastive learning, and jointly updates the local models of each participant.

[0088] In some embodiments, the above step S103 specifically includes:

[0089] S1031, the server stitches together the intermediate features of each participant, expressed as:

[0090] ;

[0091] in, Represents the features after splicing, Indicates the task party, represents the aligned samples, Indicates auxiliary party, It represents the result of local feature extraction of the task side’s aligned samples. Represents the result of local feature extraction of auxiliary alignment samples;

[0092] S1032, mapping the overall features after splicing, expressed as:

[0093] ;

[0094] in, represents the mapping function, Represents the mapped features;

[0095] S1033: The mapped features are used as the input of supervised contrastive learning to enhance the distinction between classes, which is expressed as:

[0096] ;

[0097] in, represents all positive samples in the same category, Represents negative samples in different classes, represents the temperature coefficient, represents the total number of all positive samples, represents the contrast loss, represents a positive sample, An index set representing a sample;

[0098] S1034 combines the feature transformer with the loss function in supervised contrastive learning, expressed as:

[0099] ;

[0100] in, represents the hyperparameter, Indicates overall loss;

[0101] S1035, calculate the gradient of each client according to the loss and return it to the corresponding participant, expressed as:

[0102] ;

[0103] in, represents the gradient of the auxiliary side, represents the gradient calculation, Indicates auxiliary party embedding;

[0104] In step S1036, each participant updates the local model weights using the obtained gradient, which is expressed as:

[0105] ;

[0106] in, represents the model parameters of the i-th participant, Represents the learning rate.

[0107] S104, extracting intermediate features of the misaligned samples through a feature extractor, and iteratively generating pseudo labels for the misaligned samples through a semi-supervised clustering method based on feature similarity between the misaligned samples and the aligned samples;

[0108] Since the data samples of the participants lack labels, pseudo labels are added to the unaligned samples of the participants through semi-supervised clustering. In some embodiments, the above step S104 includes:

[0109] S1041: The feature extractor extracts intermediate features of all unaligned samples of participants without labels, expressed as:

[0110] ;

[0111] in, represents the intermediate embedding of unaligned samples, represents misaligned samples, Indicates misalignment;

[0112] S1042: Based on the distance between the features of the unaligned samples and the existing aligned samples, the closest distance is iteratively calculated to generate a pseudo label for it. During the iteration, the labels of the existing aligned samples are kept unchanged, and the samples with missing labels are continuously iteratively transformed according to the distance to the class center, which can be expressed as:

[0113] ;

[0114] in, Representation sample Go to Category Center distance, Indicates the embedded eigenvalues, Indicates the Category centers, Represents the eigenvalue.

[0115] S105, obtaining the proxy features of the labeled participants through the feature transformer, calculating the feature similarity between the proxy features and the unaligned samples, and selecting the unaligned sample with the highest similarity as the semi-aligned supplementary sample;

[0116] In some embodiments, the above step S105 includes:

[0117] S1051: The labeled samples to be supplemented are passed through the transformer to obtain the agent features of the labeled participants, which are expressed as:

[0118] ;

[0119] in, Indicates the The proxy embedding obtained by transforming samples is represents the converter parameters, Indicates samples that the task party needs to supplement;

[0120] S1052: Compare the features of the unaligned samples with the proxy features to obtain a set of supplementary sample indexes, calculate their distances, and obtain the unaligned samples with the highest similarity as semi-aligned supplementary samples, which can be expressed as:

[0121] ;

[0122] in, Represents a converter.

[0123] S106: Setting a distance threshold based on the quartile method to remove low-confidence samples and retain high-quality semi-aligned samples;

[0124] It is understandable that since the supplementation of semi-aligned samples is based on pseudo-labels, it is necessary to further improve the performance of the model by screening out incorrect paired samples. In some embodiments, a distance threshold is set based on the quartile method to eliminate low-confidence samples and retain high-quality semi-aligned samples, which is expressed as:

[0125] ;

[0126] ;

[0127] ;

[0128] ;

[0129] in, represents the upper quartile, Indicates percentage, represents the minimum distance set, represents the lower quartile, represents the interquartile range, represents the threshold value, represents a hyperparameter.

[0130] In the embodiment of the present invention, the upper and lower quartiles are found by the distance between the features of the unaligned samples obtained above and the proxy features, and then the supplementary sample indexes outside the distance threshold are excluded, thereby obtaining high-quality semi-aligned samples.

[0131] S107, add the screened high-quality semi-aligned samples to the training dataset and iteratively optimize the longitudinal federated learning model.

[0132] In an embodiment of the present invention, the screened high-quality semi-aligned samples are dynamically integrated into the training data set through a vertical federated learning framework. Each participant generates intermediate features based on the locally updated feature extractor and uploads them to the server. The server optimizes the inter-class discrimination through global feature fusion and supervised comparative learning. At the same time, it combines the high-confidence semi-aligned samples screened out by proxy feature comparison and the IQR quartile method to gradually expand the diversity and coverage of the training data. Through multiple rounds of iterative optimization, the model continuously adjusts the feature representation and classification boundaries, significantly improving the classification accuracy and generalization ability while protecting data privacy, effectively solving the pain points of low utilization of unaligned samples and limited model performance in traditional vertical federated learning, and providing an efficient and secure model optimization path for cross-institutional data collaboration scenarios.

[0133] The effects of the present invention are verified below with specific examples:

[0134] In an embodiment of the present invention, a real learning scenario in federated learning is simulated on the distributed library of PyTorch. The experiment uses the MNIST, NUS-WIDE, and CIFAR-10 datasets. In each dataset, the data is divided into two parts: the task party and the auxiliary party. The task party holds the data and its true label, while the auxiliary party only holds the data itself, without label information. Specifically, each participant only holds a part of the attributes of the data. The CNN model architecture is used for the MNIST dataset, the MLP model is used for the NUS-WIDE dataset, and the ResNet18 model architecture is used for the CIFAR-10 dataset. And the unaligned sample dataset of each party is divided into twice the aligned sample dataset. The optimizer uses the adam method, the batch size is set to 128, and the learning rate is set to 1e-4.

[0135] Figure 2 、 Figure 3 、 Figure 4 A schematic diagram of the model accuracy under different numbers of aligned samples is shown, and it is compared with existing methods and traditional methods. Figure 2 The following is a comparison chart of the accuracy of the present invention and other methods on the MNIST dataset. Figure 3The following is a comparison chart of the accuracy of the present invention and other methods on the NUS-WIDE dataset. Figure 4 The following figure compares the accuracy of our method with other methods on the CIFAR10 dataset. The horizontal axis represents the number of aligned samples, and the vertical axis represents accuracy. SplitNN, FedHSSL-SimSiam, FedHSSL-Moco, and Nun-S are existing methods, while Nun represents our method. A simplified version of our method is also proposed (in which pseudo-labels are generated from unaligned samples using only aligned samples, i.e., the aligned true labels are used instead of pseudo-labels). As the number of aligned samples increases, the accuracy of traditional vertical federated learning gradually improves, and our method also exhibits a similar trend. However, compared to traditional, unoptimized vertical federated learning, all methods achieve significant performance improvements. The orange and cyan curves in the accuracy graph represent the two methods proposed by our method, respectively. The blue and green curves represent the accuracy comparison with the FedHSSL method. The gray curve represents traditional vertical federated learning. Experimental results show that the two methods proposed by our method outperform traditional vertical federated learning in terms of accuracy across all numbers of aligned samples, consistently maintaining optimal performance. These experimental results demonstrate that our method consistently significantly improves model performance across different numbers of aligned samples, validating its effectiveness in vertical federated learning.

[0136] Understandably, in the case of vertical federated learning in reality, the data distribution is often heterogeneous. To this end, this paper further studies the performance of data under different long-tail distributions. The imbalance ratio is defined as:

[0137] ;

[0138] in, Indicates the number of categories with the most samples, Indicates the number of categories with the least number of samples. Figure 5 , Figure 6 , Figure 7 The results show that the performance of the method is evaluated under different values, and multiple imbalance ratios such as 1 / 2, 1 / 4, 1 / 6, 1 / 8 and 1 / 10 are set to meet different data distribution conditions by setting them from more to less. Size represents the imbalance ratio, and the ordinate represents the accuracy; Figure 5 This is a comparison chart of the accuracy of the present invention and other methods under the MNIST dataset and different imbalance ratios. Figure 6 This is a comparison chart of the accuracy of the present invention and other methods under the NUS-WIDE dataset and different imbalance ratios. Figure 7The following chart compares the accuracy of our proposed method with other methods under different imbalance ratios on the CIFAR-10 dataset. In these experiments, the aligned samples account for 0.1%, 0.1%, and 1% of the total number of samples in the MNIST, NUS-WIDE, and CIFAR-10 datasets, respectively. The figure also compares the accuracy achieved by the proposed baseline method under imbalanced data conditions.

[0139] While the accuracy of our method exhibits a similar trend to other baseline methods, gradually decreasing with decreasing imbalance ratio, the magnitude of the decrease is significantly smaller than that of the other baseline methods. This demonstrates that our method consistently finds appropriate supplementary samples for the model. These experimental results strongly demonstrate that our method maintains robustness even under adverse data distribution conditions, effectively utilizing misaligned data and improving overall model performance.

[0140] In this invention, we can also experimentally adjust the range of hyperparameters (i.e., the distance threshold set by the quartile method) to further explore their impact on the overall model performance. Specifically, when the hyperparameter value is small, the model will be more rigorous in matching supplementary samples, selecting a narrower range of pairs and only selecting supplementary samples that are highly similar to the target sample, thereby ensuring matching accuracy.

[0141] like Figure 8 As shown in the figure, the present invention studies the effects of four different parameter values on the model accuracy to comprehensively evaluate the impact of semi-aligned samples on the model effect. Figure 8 Middle top parameter Indicates the threshold size, the vertical axis represents the accuracy, and the horizontal axis is the MNIST, NUS-WIDE and CIFAR10 data sets.

[0142] In the experiments, the model's accuracy was evaluated using parameter values of -1, -0.5, 0.5, and 1.5 on three datasets. Negative values indicate that the number of semi-aligned samples is less than the tertiary level. The model's accuracy was calculated using training with approximately 2%, 2%, and 10% aligned samples, respectively. Therefore, choosing this parameter appropriately can further improve model accuracy.

[0143] like Figure 9 and Figure 10 As shown in Figure 2, we conducted in-depth research on different misaligned samples and compared the impact of the ratio of the total number of samples to the number of aligned samples. Figure 9 The horizontal axis represents the ratio of the number of different misaligned samples, and the vertical axis represents the accuracy. The three curves correspond to the MNIST, NUS-WIDE and CIFAR10 datasets respectively. Figure 10The horizontal axis represents the ratio of the number of aligned samples, the vertical axis represents the accuracy, and the three curves correspond to the MNIST, NUS-WIDE and CIFAR10 data sets respectively.

[0144] Figure 10 Ratio and Figure 9 The difference is that this ratio is different from the misaligned sample ratio discussed previously - the ratio here represents the ratio of misaligned samples to aligned samples of a single participant, while the previous ratio refers to the ratio of the number of misaligned samples between different participants.

[0145] Figure 9 The experimental results for three datasets with different misaligned sample ratios are shown. The three experimental results in the figure were obtained under the conditions of 0.1%, 1%, and 1% aligned samples in the corresponding datasets. By analyzing the experimental data in the figure, it can be clearly seen that all the change curves are relatively smooth and stable, indicating that the method of the present invention can still show good performance even when the number of misaligned samples is insufficient. At the same time, as the proportion of misaligned samples in the auxiliary side slowly increases, the model performance shows a gradual improvement trend.

[0146] like Figure 10 As shown in the experimental configuration, the three experimental results in the figure were obtained under the conditions of 0.1%, 0.1%, and 1% aligned samples on the corresponding dataset. By analyzing the experimental data in the figure, it can be clearly seen that the proposed method consistently outperforms the baseline method, showing better performance regardless of the number of misaligned samples.

[0147] In the embodiments of the present invention, experiments validated the effectiveness of semi-aligned samples, demonstrating their potential to improve performance in longitudinal federated learning. Based on this, the present invention proposes two versions of the method that effectively utilize both aligned and unaligned samples to improve model performance. Furthermore, supervised contrastive learning and semi-supervised clustering methods are used to generate pseudo-labels for unlabeled data. By employing these techniques, the present invention provides a novel, efficient, and robust high-performance longitudinal federated learning solution while ensuring data privacy. This solution addresses the underutilization of unaligned samples in traditional longitudinal federated learning frameworks. A comprehensive evaluation on three public datasets (MNIST, NUS-WIDE, and CIFAR10) validates the effectiveness of the algorithm. Experimental results demonstrate that the present invention significantly outperforms three state-of-the-art baseline methods. Specifically, the present invention achieves model accuracy improvements ranging from 1.68% to 13.51% with varying numbers of aligned samples. It demonstrates strong robustness across a wide range of data distributions, achieving an 8.99% performance improvement even in highly imbalanced data scenarios.

[0148] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from other embodiments.

[0149] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some or all of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present application.

Claims

1. A longitudinal federated learning model optimization method based on semi-aligned data, characterized in that: include: Based on the aligned samples between the participants, the intermediate features of the aligned samples are obtained using the local feature extractors of the participants; Uploading the intermediate features to a server, wherein the server jointly trains a feature transformer based on the intermediate features to learn the correlation between the intermediate features of different participants; The server fuses the intermediate features of all participants to generate a global feature representation, optimizes the inter-class discrimination through supervised contrastive learning, and jointly updates the local models of each participant; Extracting intermediate features of the misaligned samples by the feature extractor, and iteratively generating pseudo labels for the misaligned samples by a semi-supervised clustering method based on feature similarity between the misaligned samples and the aligned samples; Obtaining the proxy features of the participants with labels through the feature transformer, calculating the feature similarity between the proxy features and the unaligned samples, and selecting the unaligned samples with the highest similarity as semi-aligned supplementary samples; The distance threshold is set based on the quartile method to remove low-confidence samples and retain high-quality semi-aligned samples; The screened high-quality semi-aligned samples are added to the training dataset, and the longitudinal federated learning model is iteratively optimized.

2. The method according to claim 1, characterized in that The alignment samples between the participants are obtained through a private intersection method.

3. The method according to claim 2, characterized in that The feature transformer is expressed as: ； in, represents the feature change loss, Indicates assignment, Indicates the task party, represents the loss function, Indicates the samples, Indicates auxiliary party, Indicates auxiliary samples, It represents the result of local feature extraction of the task side sample. Indicates the result of local feature extraction of auxiliary samples.

4. The method according to claim 3, characterized in that The server fuses the intermediate features of all participants to generate a global feature representation, optimizes inter-class discrimination through supervised contrastive learning, and jointly updates the local models of each participant, including: The server concatenates the intermediate features of each participant, expressed as: ； in, Represents the features after splicing, Indicates the task party, represents the aligned samples, Indicates auxiliary party, It represents the result of local feature extraction of the task side’s aligned samples. Represents the result of local feature extraction of auxiliary alignment samples; Map the overall features after splicing and express it as: ； in, represents the mapping function, Represents the features after mapping; The mapped features are used as the input of supervised contrastive learning to enhance the distinction between classes, which can be expressed as: ； in, represents all positive samples in the same category, Represents negative samples in different classes, represents the temperature coefficient, represents the total number of all positive samples, represents the contrast loss, represents a positive sample, An index set representing a sample; Combining the feature transformer with the loss function in supervised contrastive learning is expressed as: ； in, represents the hyperparameter, Indicates overall loss; The gradient of each client is calculated according to the loss and returned to the corresponding participant, expressed as: ； in, represents the gradient of the auxiliary side, represents the gradient calculation, Indicates auxiliary party embedding; Each participant updates the local model weights using the obtained gradient, expressed as: ； in, represents the model parameters of the i-th participant, Represents the learning rate.

5. The method according to claim 4, characterized in that The method of extracting intermediate features of the misaligned samples by the feature extractor and iteratively generating pseudo labels for the misaligned samples by a semi-supervised clustering method based on feature similarity between the misaligned samples and the aligned samples comprises: The feature extractor extracts the intermediate features of the unaligned samples of all unlabeled participants, which are expressed as: ； in, represents the intermediate embedding of unaligned samples, represents misaligned samples, Indicates misalignment; According to the distance between the features of the unaligned samples and the existing aligned samples, the closest distance is iteratively calculated to generate a pseudo label for it. In the iterative process, the labels of the existing aligned samples are kept unchanged, and the samples with missing labels are continuously iteratively transformed according to the distance to the category center, which can be expressed as: ； in, Represents a sample Go to Category Center distance, Indicates the embedded eigenvalues, Indicates the Category centers, Represents the eigenvalue.

6. The method according to claim 5, characterized in that The method of obtaining the proxy features of the participants with labels through the feature transformer, calculating the feature similarity between the proxy features and the unaligned samples, and selecting the unaligned samples with the highest similarity as the semi-aligned supplementary samples includes: The required supplemented labeled samples are passed through the transformer to obtain the agent features of the labeled participants, which are expressed as: ； in, Indicates the The proxy embedding obtained by transforming samples is represents the converter parameters, Indicates samples that the task party needs to supplement; Compare the features of the unaligned samples with the proxy features to obtain a set of supplementary sample indexes, and calculate their distance to obtain the unaligned sample with the highest similarity as the semi-aligned supplementary sample, which is expressed as: ； in, Represents a converter.

7. The method according to claim 6, characterized in that The distance threshold is set based on the quartile method, low-confidence samples are eliminated, and high-quality semi-aligned samples are retained, which is expressed as: ； ； ； ； in, represents the upper quartile, Indicates percentage, represents the minimum distance set, represents the lower quartile, represents the interquartile range, represents the threshold value, represents a hyperparameter.

Images