Federal collaborative learning method and device for multi-dimensional heterogeneous data

Abstract

The invention provides a federal collaborative learning method and device for multi-dimensional heterogeneous data, and relates to the technical field of federal learning. The method comprises the following steps: for any client in a plurality of clients with non-shared local samples, mapping the local samples of the client to a public space through a global feature anchor mapping model to obtain mapping samples; inputting the hard tag and the mapping sample into a global feature anchor point mapping model to obtain a generation sample consistent with global anchor point distribution; and inputting the hard tag, the mapping sample and the generated sample into a prediction model based on regression knowledge distillation to obtain a prediction result. According to the method, under the condition that original data of all participants are not shared and only model parameters and statistics are exchanged, accurate training and robust aggregation of a global model are achieved, and the accuracy and generalization ability of the model in a real Non-IID scene are improved.

Description

Technical Field

[0001] This invention relates to the fields of distributed machine learning and federated learning technologies, and in particular to a federated collaborative learning method and apparatus for multidimensional heterogeneous data. Background Technology

[0002] In practical applications of cross-organizational collaborative computing and joint modeling, efficient and reliable model training while protecting data privacy is crucial. These scenarios typically involve multiple institutions or devices (such as enterprises, hospitals, factory production lines, and IoT edge nodes), where sharing raw data is difficult due to compliance and security concerns. Federated Learning (FL) provides a feasible path to address these needs: each participant trains the model locally, exchanging only parameters or gradients to achieve global aggregation, thus completing collaborative modeling while protecting sensitive data.

[0003] However, current mainstream federated collaborative learning (FL) research largely relies on public datasets and idealized settings for validation, primarily focusing on classification tasks. This research fails to adequately consider the non-independent and identically distributed (Non-IID) characteristics of real-world business scenarios, leading to decreased model accuracy and generalization ability during deployment. Furthermore, the lack of systematic experimental validation with real-world complex data and long-cycle production environments leaves the feasibility and stability of existing methods uncertain in industrial, medical-engineering, and urban computing scenarios. Therefore, there is an urgent need for federated collaborative learning solutions that strike a balance between privacy protection, robustness, and efficiency to meet the demands of large-scale, long-chain, and highly constrained practical applications.

[0004] Based on this, it is important to emphasize that these limitations of "setting and evaluation" are not the only bottleneck; in real deployments, the more fundamental difficulty comes from the structural differences in the data generation process itself—that is, multidimensional heterogeneity—which directly undermines the comparability and transferability of federated aggregation, and is the core source of unstable training, slow convergence and performance degradation.

[0005] Feature space heterogeneity: Inconsistencies in feature sets, dimensions, scales, and semantics among different participants (including missing modalities and device / protocol differences) undermine the premise of "comparable gradients," causing semantic misalignment of cross-end parameters / gradients. Direct aggregation is equivalent to forcibly averaging updates from different representation subspaces, leading to distortion of the global update direction and slow convergence. Normalized statistics (such as mean / covariance, BN / LayerNorm estimates) cannot be reused across ends, resulting in domain differences between training and inference. Shared representations are prone to representation collapse or catastrophic forgetting during multiple rounds of "pulling," and if key discriminative features exist only on a few ends, they will create inherent information gaps, lowering the global optimal upper bound.

[0006] Imbalanced sample size: When there are significant differences in sample size, proportion of hard cases, and coverage area among clients, aggregation weighted by sample size will cause the gradient of "large clients" to dominate in the long term, while the gradient of "small clients" and long-tail patterns will be systematically diluted, resulting in unfair fitting and persistent underfitting to sparse regions; coverage differences lead to high variance and reversal of gradient direction between clients, causing training curve oscillation, slow convergence, or even divergence; at the same time, the coexistence of "overfitting of large clients and underfitting of small clients" often occurs in the global model, ultimately forming a "mediocre but ineffective model for key tail indicators", and the recall rate of extremely rare patterns is difficult to recover.

[0007] Label distribution discrepancies: When the marginal / conditional distributions of the target variable at each end exhibit mean drift, variance differences, and varying noise levels (especially in regression), the aggregated model often shows optimality for the "average label distribution," but exhibits systematic bias and inaccurate calibration at any end; inconsistent optimal points of the lost landscape at each end lead to severe client-side bias, resulting in a lack of consistent convergence direction across rounds of updates; high-noise or dimensionally inconsistent ends contribute "harmful gradients," causing negative migration and pushing the global solution away from the target domain; macro-average indicators appear stable, but there is a significant mismatch at key quantiles and tail risks, exposing potential accuracy and reliability vulnerabilities during the deployment phase.

[0008] In summary, the multidimensional heterogeneity of data reduces the comparability and aggregability of client updates, leading to problems such as unstable global model training, slow convergence, and performance degradation. It may also amplify systematic biases and weaken the robustness and effectiveness of the model in real-world environments. Summary of the Invention

[0009] To address the technical problems of convergence instability, ineffective aggregation, and insufficient generalization in existing federated learning methods for cross-organizational collaborative computing caused by multidimensional heterogeneity (including feature space heterogeneity, sample imbalance, and differences in label distribution), this invention provides a federated collaborative learning method and apparatus for multidimensional heterogeneous data. The technical solution is as follows: On the one hand, a federated collaborative learning method for multidimensional heterogeneous data is provided. This method is implemented by a federated collaborative learning device for multidimensional heterogeneous data, and includes: S1. For any client among multiple clients that do not share local samples, the client's local samples are mapped to the public space through the global feature anchor mapping model to obtain the mapped samples; wherein, the public space is obtained by the server performing federated aggregation of the local anchors of all clients to obtain the global anchor distribution, according to the definition of the global anchor distribution.

[0010] S2. Input the hard labels and mapped samples of the local samples into the global feature anchor mapping model to obtain generated samples that are consistent with the global anchor distribution; wherein, the parameters of the generator and discriminator in the global feature anchor mapping model are optimized by the server federating the parameters of all clients.

[0011] S3. Input the hard labels, mapped samples, and generated samples of the local samples into the prediction model based on regression knowledge distillation to obtain the prediction results; wherein, the training process of the prediction model is guided by the global soft label distribution obtained by the server aggregating the soft labels of all clients.

[0012] Optionally, the feature spaces of multiple clients are heterogeneous, which can be quantified by defining a composite feature heterogeneity index.

[0013] The imbalance of samples across multiple clients is quantified by defining a sample imbalance coefficient.

[0014] The differences in tag distribution across multiple clients are determined by defining... The overall difference in label distribution among the datasets is quantified using an indicator.

[0015] Optionally, in S1, the client's local samples are mapped to the public space using a global feature anchor mapping model to obtain mapped samples, including: The client's local samples are mapped to the initial public space through a local mapping network using a global feature anchor mapping model, resulting in mapped samples.

[0016] Calculate the statistics of the mapped samples to obtain the local anchor points.

[0017] The server performs federated aggregation of local anchor points from all clients to obtain a global anchor point distribution. Based on the global anchor point distribution and the federated round cycle, the common space is adaptively updated, and the local samples of the clients are mapped according to the updated common space.

[0018] Optionally, the global feature anchor mapping model in S1 employs a multilayer perceptron.

[0019] The optimization objective of the global feature anchor mapping model includes: jointly minimizing the reconstruction and prediction consistency term and the distribution alignment term; wherein, the reconstruction and prediction consistency term adopts the mean squared error, and the distribution alignment term adopts the closed form of the second-order Wasserstein distance, the closed form of the second-order Wasserstein distance including the Bures distance.

[0020] The optimization objective of the global feature anchor mapping model is shown in equation (1) below: (1) in, (2) (3) (4) In the formula, This represents the objective function of the global feature anchor mapping model. Indicates alignment loss. Indicates the reconstruction loss weights. Indicates the reconstruction loss. This represents the global anchor point loss weight. Indicates global anchor point loss. Indicates the number of samples. Indicates the first The original input feature vector of each sample, Indicates the first Sample features after mapping of each sample This represents the weighting coefficient based on the proportion of features. Indicates the first The number of features for each sample This represents the total number of features across all datasets. This represents the Wasserstein distance operator. Indicates in The local feature distribution obtained under the mapping Indicates the first The feature vector of each sample Represents the global feature distribution. This represents the total number of samples after mapping. This represents the local label prediction function. Indicates the first Features of the mapped samples The parameters representing the local model, Indicates the first The label value of each sample, The mean vector representing the global feature distribution. Indicates the first The mean vector of a local feature distribution. Indicates the Bures distance. The covariance matrix representing the global feature distribution. Indicates the first The covariance matrix of the local feature distribution.

[0021] Optionally, in S2, the hard labels and mapped samples of the local samples are input into the global feature anchor mapping model to obtain generated samples consistent with the global anchor distribution, including: The hard labels of local samples and global anchors are input into the generator of the global feature anchor mapping model to obtain generated samples that are consistent with the global anchor distribution.

[0022] The mapped samples and generated samples are input into the discriminator of the global feature anchor mapping model for discrimination training, and the parameters of the trained generator and discriminator are obtained. The parameters of the trained generator and discriminator are periodically uploaded to the server.

[0023] Optionally, in S3, the hard labels of the local samples, the mapped samples, and the generated samples are input into a prediction model based on regression knowledge distillation to obtain prediction results, including: The mapped samples are input into a student model of a prediction model based on regression knowledge distillation to obtain soft predictions.

[0024] Hard predictions are obtained based on hard labels.

[0025] The generated samples are input into the teacher model of the prediction model based on regression knowledge distillation to obtain soft labels.

[0026] Hard label supervision is constructed based on hard labels and hard predictions, and distillation supervision is constructed based on soft predictions and soft labels. The student model is then trained to obtain a well-trained student model, which is then used for prediction.

[0027] On the other hand, a federated collaborative learning device for multidimensional heterogeneous data is provided. This device is applied to a federated collaborative learning method for multidimensional heterogeneous data, and the device includes: The global feature anchor mapping module is used to map the local samples of any client to a common space through a global feature anchor mapping model, which is used to obtain mapped samples. The common space is obtained by the server performing federated aggregation of the local anchors of all clients to obtain the global anchor distribution, which is defined according to the global anchor distribution.

[0028] The global feature anchor mapping module is used to input the hard labels and mapped samples of local samples into the global feature anchor mapping model to obtain generated samples that are consistent with the global anchor distribution. The parameters of the generator and discriminator in the global feature anchor mapping model are optimized by the server federating the parameters of all clients.

[0029] The prediction module based on regression knowledge distillation is used to input the hard labels of local samples, mapped samples, and generated samples into the prediction model based on regression knowledge distillation to obtain the prediction results. The training process of the prediction model is guided by the global soft label distribution obtained by the server aggregating the soft labels of all clients.

[0030] Optionally, the feature spaces of multiple clients are heterogeneous, which can be quantified by defining a composite feature heterogeneity index.

[0031] The imbalance of samples across multiple clients is quantified by defining a sample imbalance coefficient.

[0032] The differences in tag distribution across multiple clients are determined by defining... The overall difference in label distribution among the datasets is quantified using an indicator.

[0033] Optionally, the global feature anchor mapping module is further used for: The client's local samples are mapped to the initial public space through a local mapping network using a global feature anchor mapping model, resulting in mapped samples.

[0034] Calculate the statistics of the mapped samples to obtain the local anchor points.

[0035] The server performs federated aggregation of local anchor points from all clients to obtain a global anchor point distribution. Based on the global anchor point distribution and the federated round cycle, the common space is adaptively updated, and the local samples of the clients are mapped according to the updated common space.

[0036] Optionally, the global feature anchor mapping model employs a multilayer perceptron.

[0037] The optimization objective of the global feature anchor mapping model includes: jointly minimizing the reconstruction and prediction consistency term and the distribution alignment term; wherein, the reconstruction and prediction consistency term adopts the mean squared error, and the distribution alignment term adopts the closed form of the second-order Wasserstein distance, the closed form of the second-order Wasserstein distance including the Bures distance.

[0038] The optimization objective of the global feature anchor mapping model is shown in equation (1) below: (1) in, (2) (3) (4) In the formula, This represents the objective function of the global feature anchor mapping model. Indicates alignment loss. Indicates the reconstruction loss weights. Indicates the reconstruction loss. This represents the global anchor point loss weight. Indicates global anchor point loss. Indicates the number of samples. Indicates the first The original input feature vector of each sample, Indicates the first Sample features after mapping of each sample This represents the weighting coefficient based on the proportion of features. Indicates the first The number of features for each sample This represents the total number of features across all datasets. This represents the Wasserstein distance operator. Indicates in The local feature distribution obtained under the mapping Indicates the first The feature vector of each sample Represents the global feature distribution. This represents the total number of samples after mapping. This represents the local label prediction function. Indicates the first Features of the mapped samples The parameters representing the local model, Indicates the first The label value of each sample, The mean vector representing the global feature distribution. Indicates the first The mean vector of a local feature distribution. Indicates the Bures distance. The covariance matrix representing the global feature distribution. Indicates the first The covariance matrix of the local feature distribution.

[0039] Optionally, the global feature anchor mapping module is further used for: The hard labels of local samples and global anchors are input into the generator of the global feature anchor mapping model to obtain generated samples that are consistent with the global anchor distribution.

[0040] The mapped samples and generated samples are input into the discriminator of the global feature anchor mapping model for discrimination training, and the parameters of the trained generator and discriminator are obtained. The parameters of the trained generator and discriminator are periodically uploaded to the server.

[0041] Optionally, the prediction module based on regression knowledge distillation is further used for: The mapped samples are input into a student model of a prediction model based on regression knowledge distillation to obtain soft predictions.

[0042] Hard predictions are obtained based on hard labels.

[0043] The generated samples are input into the teacher model of the prediction model based on regression knowledge distillation to obtain soft labels.

[0044] Hard label supervision is constructed based on hard labels and hard predictions, and distillation supervision is constructed based on soft predictions and soft labels. The student model is then trained to obtain a well-trained student model, which is then used for prediction.

[0045] On the other hand, a federated collaborative learning device for multidimensional heterogeneous data is provided. The federated collaborative learning device for multidimensional heterogeneous data includes: a processor; a memory, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, any one of the methods described above for federated collaborative learning of multidimensional heterogeneous data is implemented.

[0046] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described federated collaborative learning methods for multidimensional heterogeneous data.

[0047] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this invention, given that the local data of each participant cannot go out of the domain and that there are basic prior statistics and training interfaces, an integrated framework of collaborative feature alignment, sample balancing and label alignment is adopted to complete the iterative optimization of the global model with considerable computational and communication overhead. The proposed method comprehensively considers the impact of feature space inconsistency on parameter aggregation, gradient variance amplification and training oscillation caused by sample imbalance, and systematic bias and client offset caused by label distribution differences. It proposes a federated collaborative algorithm consisting of a global feature anchor mapping model, a conditional generative adversarial model based on global prediction distribution, and a prediction model based on regression knowledge distillation. The global feature anchor mapping model maps features from each endpoint to a common space and aligns statistical differences using distance metrics. It also generates supplementary samples to cover long-tail and sparse regions without exposing the original data. The prediction model based on regression knowledge distillation constrains predictions from each endpoint with a global soft label distribution, achieving label distribution consistency and stable convergence. On the server side, global anchor parameter aggregation, generator / discriminator parameter aggregation, and soft label distribution aggregation are executed respectively, and updates are sent to the client for continued local training, thereby obtaining a robust and accurate global model in heterogeneous scenarios. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart of a federated collaborative learning method for multidimensional heterogeneous data provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a federated collaborative learning method for multidimensional heterogeneous data provided in an embodiment of the present invention; Figure 3 This is a block diagram of a federated collaborative learning device for multidimensional heterogeneous data provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a federated collaborative learning device for multidimensional heterogeneous data provided in an embodiment of the present invention. Detailed Implementation

[0050] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0051] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0052] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent.

[0053] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0054] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0055] This invention provides a federated collaborative learning method for multidimensional heterogeneous data. This method can be implemented by a federated collaborative learning device for multidimensional heterogeneous data, which can be a terminal or a server. Figure 1 The flowchart shown is for a federated collaborative learning method for multidimensional heterogeneous data. The processing flow of this method may include the following steps: S1. For any client among multiple clients that do not share local samples, the client's local samples are mapped to the public space through the global feature anchor mapping model to obtain the mapped samples.

[0056] In the public space, the server performs federated aggregation of the local anchor points of all clients to obtain the global anchor point distribution, which is defined according to the global anchor point distribution.

[0057] Optionally, the feature spaces of multiple clients are heterogeneous, which can be quantified by defining a composite feature heterogeneity index.

[0058] The imbalance of samples across multiple clients is quantified by defining a sample imbalance coefficient.

[0059] The differences in tag distribution across multiple clients are determined by defining... The overall difference in label distribution among the datasets is quantified using an indicator.

[0060] In one feasible implementation, to improve the accuracy and generalization ability of federated learning in real-world scenarios, this invention addresses multidimensional heterogeneity as a core issue and performs quantitative modeling. Due to variations in data acquisition conditions, measurement methods, and processing procedures across institutions and devices, data exhibits significant differences across multiple dimensions. This invention focuses on three key types of heterogeneity: feature space differences, sample imbalance, and label distribution differences, and provides actionable metrics to guide the optimization of federated training and aggregation strategies.

[0061] Furthermore, this invention formally models multidimensional heterogeneity as: a) Feature space heterogeneity: different clients have inconsistent feature attributes and feature numbers, denoted by the composite feature heterogeneity index as... b) Imbalanced sample size: The sample size differs significantly between the two ends, leading to client bias. This invention defines a sample imbalance coefficient. Quantifying the degree of imbalance; c) Label distribution differences: The label distribution among clients typically exhibits non-independent and identically distributed characteristics. This difference will further exacerbate prediction bias. The overall difference index of label distribution among the datasets is .

[0062] Specifically, feature space differences: different participants may have different indicator systems and feature dimensions (including missing modalities, inconsistencies in units and semantics). To quantify feature heterogeneity across clients, a composite indicator is defined: (1) (2) (3) In the formula, A comprehensive metric representing the differences in the feature space. This indicates the number of clients participating in federated learning. Indicates the first Index for each client, Indicates the first Index for each client, Indicates the client With the client Jaccard similarity between the feature attributes. Indicates client The set of characteristic attributes it possesses Indicates client The set of characteristic attributes it possesses Indicates client With the client Similarity in feature dimensions between them Indicates client Feature dimensions in the local dataset Indicates client Feature dimensions in the local dataset. The value comprehensively describes "which features (similar attributes)" and "how many dimensions (similar dimensions)". The lower the value, the stronger the heterogeneity of the feature space, indicating the need for a more proactive feature alignment / mapping and normalized statistical domain division strategy.

[0063] Imbalanced sample size: Significant differences in client sample size and coverage can lead to large clients dominating aggregation and long-tail samples being diluted. Define the imbalance coefficient: (4) In the formula, Indicates the sample imbalance coefficient. Indicates the first Number of samples per client Indicates the number of clients. The larger the value, the higher the risk of gradient variance and convergence oscillation during training. Mechanisms such as reweighting / resampling / hard example mining and adaptive aggregation weights need to be introduced to ensure that small sample ends and sparse regions are fully learned.

[0064] Label distribution discrepancies: The marginal / conditional distributions of the target variable exhibit differences in mean, variance, or noise levels across different participants (especially noticeable in regression tasks). The average KL divergence is used to measure cross-end label distribution inconsistency. (5) In the formula, This represents the average KL divergence metric. Indicates the number of clients. Indicates client Label distribution Indicates client The distribution of labels. The larger the value, the more pronounced the systematic bias and client offset become. It is necessary to use methods such as soft labeling / distributed distillation, quantile / segmentation loss, and calibration regularization to perform label alignment and robust convergence control.

[0065] Optionally, step S1 above may include: The client's local samples are mapped to the initial public space through a local mapping network using a global feature anchor mapping model, resulting in mapped samples.

[0066] Calculate the statistics of the mapped samples to obtain the local anchor points.

[0067] The server performs federated aggregation of local anchor points from all clients to obtain a global anchor point distribution. Based on the global anchor point distribution and the federated round cycle, the common space is adaptively updated, and the local samples of the clients are mapped according to the updated common space.

[0068] In one feasible implementation, the federated collaborative learning method for multidimensional heterogeneous data aims to address the challenge of multidimensional heterogeneity in federated learning through collaborative learning. Its overall framework is as follows: Figure 2 As shown. The framework mainly includes two core stages: (1) co-training local models and (2) federated aggregation. Among them, the three local models—the global feature anchor mapping model, the conditional generative adversarial model based on the global prediction distribution, and the prediction model based on regression knowledge distillation—work together to align the feature space, balance the data distribution, and optimize the label representation, respectively, thereby mitigating the impact of feature space differences, sample imbalance, and label distribution differences on the federated learning model. The federated aggregation mechanism further ensures the consistency of the global model and improves the overall prediction performance.

[0069] A global feature anchor mapping model is used to align client-side local features with server-side global feature representations. During the prediction update phase, a mapping network is learned based on local data. By projecting high-dimensional features from each client onto a common representation space defined by a unified global anchor distribution, the comparability and compatibility of feature spaces across different clients are ensured. In the global aggregation phase, a Gaussian family of anchor distributions (with mean and covariance weighted and aggregated from statistics from each client, including a second-order correction term) is used as a unified representation. The model approximates the consistency between each client and the global anchor distribution by jointly minimizing the Wasserstein-Bures distribution distance and the reconstruction / task loss, thereby improving the comparability and aggregability of cross-client parameters and gradients. Specifically, the global feature anchors are federated and adaptively updated by the server based on statistics / parameters uploaded by each client, improving feature alignment quality under Non-IID conditions and enhancing the generalization ability and robustness of the global model.

[0070] Specifically, the model consists of four components: local samples, local model, mapped samples, and local anchors. The client takes "local samples" as input, processes them through the "local model" to obtain "mapped samples," and forms "local anchors" (mean / covariance, etc.) based on their statistics. The server performs federated aggregation of the "local anchors" from each client to obtain a "global anchor distribution," which is then distributed to guide the next round of mapping. This ensures that different clients maintain comparability and compatibility within the common representation space defined by the global anchors, thereby improving feature alignment quality and the generalization and robustness of the global model under Non-IID conditions.

[0071] The optimization objectives of the global feature anchor mapping model include: global feature anchor mapping model The high-dimensional features of each client are projected into a unified low-dimensional space, and joint optimization using "reconstruction and prediction consistency + distribution alignment" is employed to improve cross-platform comparability and global generalization ability; its comprehensive objective (including weights) Alignment weights The definition is as follows: (6) in, (7) The consistency term for reconstruction and prediction uses mean squared error: (8) The distribution alignment term is expressed in closed form using the second-order Wasserstein distance (Wasserstein-2): (9) Bures distance: (10) In the formula, This represents the objective function of the global feature anchor mapping model. Indicates alignment loss. Indicates the reconstruction loss weights. Indicates the reconstruction loss. This represents the global anchor point loss weight. Indicates global anchor point loss. Indicates the number of samples. Indicates the first The original input feature vector of each sample, Indicates the first Sample features after mapping of each sample This represents the weighting coefficient based on the proportion of features. Indicates the first The number of features for each sample This represents the total number of features across all datasets. This represents the Wasserstein distance operator. Indicates in The local feature distribution obtained under the mapping Indicates the first The feature vector of each sample Represents the global feature distribution. This represents the total number of samples after mapping. This represents the local label prediction function. Indicates the first Features of the mapped samples The parameters representing the local model, Indicates the first The label value of each sample, The mean vector representing the global feature distribution. Indicates the first The mean vector of a local feature distribution. Represents Bures distance, explicitly aligned mean and covariance statistics. The covariance matrix representing the global feature distribution. Indicates the first The covariance matrix of each local feature distribution. To evaluate the effectiveness of the anchor point for the task, Global anchor point sampling Calculate the prediction error to improve the discriminability and usability of anchor samples.

[0072] Global feature anchor mapping model training: It employs an MLP (Multilayer Perceptron), implemented using fully connected layers (FC) and activations, to transform local features from dimensionality... Mapped to a common low-dimensional space Minimize during training The FC layer can be followed by ReLU (Rectified Linear Unit) or Tanh activation to enhance nonlinear expression, while also... The computed Wasserstein alignment term continuously compresses the statistical difference between the client and the global anchor; the process stably produces comparable common representations under Non-IID conditions and avoids interference from one-end bias on global aggregation.

[0073] Local anchor aggregation and update: Global anchors are distributed in a Gaussian manner. Initialize and update according to the federated round cycle: Each client reports its local statistics. The server is based on sample / feature weighting. Aggregation yields: (11) In the formula, This indicates the number of clients.

[0074] Subsequently issued The client updates the next round of mapping using SGD (Stochastic Gradient Descent) or its adaptive variant. To minimize the local Wasserstein loss, a closed loop of "statistical aggregation - local optimization" is formed, enabling feature alignment and task prediction to converge synchronously.

[0075] S2. Input the hard labels and mapped samples of the local samples into the global feature anchor mapping model to obtain generated samples that are consistent with the global anchor distribution.

[0076] In the global feature anchor mapping model, the parameters of the generator and discriminator are optimized by server-wide aggregation of parameters from all clients.

[0077] Optionally, step S2 above may include: The hard labels of local samples and global anchors are input into the generator of the global feature anchor mapping model to obtain generated samples that are consistent with the global anchor distribution.

[0078] The mapped samples and generated samples are input into the discriminator of the global feature anchor mapping model for discrimination training, and the parameters of the trained generator and discriminator are obtained. The parameters of the trained generator and discriminator are periodically uploaded to the server.

[0079] In one feasible implementation, the conditional adversarial generative model based on global prediction distribution employs a generative adversarial network (GAN) mechanism. Hard labels and samples mapped by a global feature anchor mapping model are used as conditional inputs. The generator produces synthetic samples that conform to the global feature distribution to balance the data distribution across clients. A discriminator distinguishes between real and synthetic samples. During the update phase, local generators and discriminators are trained using noise and task labels sampled from the global anchor distribution as conditions to generate supplementary samples to cover long-tail and sparse regions. Each client periodically reports generator / discriminator parameters. The server uses a weighted or adaptive federated aggregation strategy to aggregate and optimize the generator / discriminator parameters before distributing them. This mitigates the increased gradient variance and training oscillations caused by sample imbalance between clients without leaking the original data, enabling the generator to obtain stable, high-quality supplementary data generation capabilities, thereby alleviating sample imbalance and promoting global data distribution consistency.

[0080] Specifically, the model includes components such as hard labels, global anchors, a generator, mapped samples, real and fake generated samples, and a discriminator. The client drives the generator with "hard labels" and "global anchors" as conditions to synthesize "generated samples" consistent with the global feature distribution. Simultaneously, the "mapped samples" and "real and fake generated samples" are fed into the discriminator for training. Each client periodically uploads generator / discriminator parameters, which are then distributed by the server after federated aggregation. This ensures the generator consistently produces high-quality supplementary data to cover long-tail and sparse regions, balance data distribution across clients, and promote global data distribution consistency.

[0081] The optimization objective of the conditional generative adversarial model based on global prediction distribution is: the conditional generative adversarial model based on global prediction distribution uses global anchor sampling. Drive the generator with hard tags as conditions It produces synthetic samples that fit the global feature distribution, while the discriminator... Discrimination and The authenticity of the information; its target is: (12)

[0082] In the formula, This represents the loss of a conditional generative adversarial model based on a global prediction distribution. The operator representing the mathematical expectation of the corresponding random variable. Indicates the first The first client's The true feature values ​​of each sample Indicates the first The true label value of each sample This represents the prior distribution of the label values. Indicates the first The first client's The label values ​​generated for each sample This represents the global feature distribution obtained from global feature statistics. Minimize the generation loss to approximate the true distribution. Maximize the discrimination loss to improve the ability to distinguish; this mechanism expands the coverage of long-tail and sparse regions without exposing the original data, thus alleviating sample imbalance.

[0083] Training of a conditional generative adversarial model based on a globally predicted distribution: At each client, a generator and discriminator are locally maintained and optimized alternately. The generator uses an MLP (Functionally Connected + Batch Normalization + ReLU) to receive input. generate The discriminator uses an MLP (FC+BN+LeakyReLU, where LeakyReLU is an activation with leakage linearity) receiver. or To improve the robustness and generalization of the discrimination; by minimizing / maximizing the above adversarial objectives, the synthetic data is made consistent with the global distribution in terms of feature space and label conditions, thereby reducing the inter-end gradient variance and training oscillations.

[0084] Parameter aggregation and updating of conditional generative adversarial models based on global prediction distribution: Under the federated learning framework, each client periodically reports... Parameters; the server performs a weighted average to obtain global parameters. And distribute synchronously: This aggregation achieves cross-platform knowledge transfer and sample balancing while ensuring privacy, complementing the feature alignment of the global feature anchor mapping model, thereby significantly improving the robustness and generalization performance of the global model in Non-IID scenarios.

[0085] S3. Input the hard labels of the local samples, the mapped samples, and the generated samples into the prediction model based on regression knowledge distillation to obtain the prediction results.

[0086] The training process of the prediction model is guided by the global soft label distribution obtained by the server aggregating the soft labels of all clients.

[0087] Optionally, step S3 above may include: The mapped samples are input into a student model of a prediction model based on regression knowledge distillation to obtain soft predictions.

[0088] Hard predictions are obtained based on hard labels.

[0089] The generated samples are input into the teacher model of the prediction model based on regression knowledge distillation to obtain soft labels.

[0090] Hard label supervision is constructed based on hard labels and hard predictions, and distillation supervision is constructed based on soft predictions and soft labels. The student model is then trained to obtain a well-trained student model, which is then used for prediction.

[0091] In one feasible implementation, the prediction model based on regression knowledge distillation is optimized using a teacher-student structure based on the knowledge distillation paradigm: the teacher model transmits global knowledge through soft labels, the student model makes predictions and learns based on mapped local samples, and the client performs joint optimization using mean squared error + teacher constraints + distribution-level KL regularization (a temperature parameter can be set to adjust the smoothness of the soft labels), and synchronously sends back local prediction distribution statistics to update the global soft labels, thereby suppressing systematic bias and client offset caused by label distribution differences, and improving the consistency and reliability of the model on different sites; the server aggregates the local prediction distribution parameters of each client to form a global soft label distribution, and guides the client training accordingly, so as to maintain high prediction accuracy and generalization ability under the conditions of unbalanced data distribution and non-independent and identically distributed data.

[0092] Specifically, the model consists of components such as hard labels, mapped samples, student model, hard prediction, soft prediction, generated samples, teacher model, soft labels, and loss. The client inputs "mapped samples" into the "student model" to obtain "soft predictions" and "hard labels" to obtain "hard predictions"; simultaneously, it inputs "generated samples" into the "teacher model" to obtain "soft labels." During training, the "loss" term jointly constrains: hard label supervision (hard prediction and hard label) and distillation supervision (soft prediction and soft label). The server aggregates the "soft labels" from each client to form a global soft label distribution, which is then fed back to guide client updates. This mechanism improves the learning ability of the client-side and the accuracy and generalization ability of the global model under conditions of imbalanced data distribution and Non-IID.

[0093] The goal of the regression-based knowledge distillation-based prediction model is to map samples locally and generated samples using a student model. The core of the prediction is based on the above, and the label distribution differences are calibrated through the global soft label provided by the teacher model; its comprehensive objective is: (13) In the formula, The mean squared error (MSE) is... The student model is supervised by globally generated samples and soft labels (soft labels are reparameterized from the global distribution). get), Align global and local prediction distributions using Kullback–Leibler (KL) divergence. For the global feature anchor mapping model alignment term, the weights are... , , A trade-off between local accuracy and global consistency.

[0094] Training of prediction models based on regression knowledge distillation: Each client implements a local model using a multilayer perceptron (MLP). In synthetic datasets The above-mentioned comprehensive loss is minimized to simultaneously absorb local supervision, teacher soft labels, and distribution alignment signals; after training, the local prediction distribution is approximated as a Gaussian distribution. Used for subsequent cross-end calibration, among which Obtained through local prediction statistics, ensuring that label-level distillation alignment and robust learning are completed without sharing the original data.

[0095] Soft label aggregation and updating: Each client reports its local predicted distribution parameters. Server aggregation forms a global soft tag distribution. ,in: (14) In the formula, This represents the mean of the global feature distribution. Indicates the number of clients. Indicates the first The mean of the feature distribution of each client, This represents the global characteristic distribution covariance. Indicates the first The feature distribution covariance of each client.

[0096] And issued To drive the next round of distillation; the client then continues to minimize accordingly. (especially) and This allows for global calibration and stable convergence of the tag distribution under Non-IID conditions, thus enabling the achievement of stable convergence.

[0097] This invention addresses the multidimensional heterogeneity issues in federated environments, such as feature space differences, sample imbalance, and label distribution differences, without sharing the original data of each participant. It constructs a collaborative local training framework consisting of a global feature anchor mapping model, a conditional adversarial generative model based on global prediction distribution, and a prediction model based on regression knowledge distillation. Each client trains locally and reports statistics / parameters. Global consistency optimization is achieved through Gaussian anchor aggregation, adversarial generator / discriminator parameter aggregation, soft label distribution aggregation, and KL regularization alignment. The federated training and communication mechanism supports global feature anchor mapping within the same federated round. The model, a conditional generative adversarial model based on global prediction distribution, and a prediction model based on regression knowledge distillation are alternately or in parallel executed in a "three-way aggregation" process (global anchor aggregation, generator / discriminator parameter aggregation, and soft label distribution aggregation) and updated accordingly. This process iterates until convergence to obtain a global regression model with higher accuracy and robustness in heterogeneous scenarios. The aggregation frequency, learning rate, and loss weights can be adaptively adjusted based on validation error and stability metrics. In bandwidth or privacy-constrained scenarios, differential privacy techniques such as statistical compression, L2 pruning, and Gaussian noise mechanisms can be employed, combined with secure aggregation / encrypted transmission to ensure the privacy and communication security of participating parties. This method utilizes Wasserstein-Bures distance-aligned feature distribution, alleviates sample imbalance based on conditional generative adversarial mechanisms, and reduces the impact of label distribution differences on the accuracy of the prediction model through teacher-student distillation, thus achieving a balance of accuracy, robustness, and scalability in multi-institutional, multi-dimensional heterogeneous data scenarios.

[0098] This invention addresses the problem of regression or continuous value prediction tasks in real-world Non-IID scenarios, and is suitable for collaborative modeling applications involving cross-organizational, cross-device, or multimodal data. Each participant's local samples are data records with continuous labels collected and stored locally during their business operations. Each sample consists of a set of feature data describing the object's state or attributes (e.g., material composition and process parameters, statistics of equipment sensor time-series signals, user behavior and profile features, or feature vectors extracted from multimodal data such as images, text, and time-series signals) and a corresponding target continuous value (e.g., material performance indicators, equipment health or remaining life assessment values, risk scores, production or energy efficiency indicators, etc.). Each participant preprocesses the original data locally, performing missing value imputation, outlier removal, normalization / standardization, and multimodal feature encoding and alignment. The processed features are then input into the collaborative training framework of this invention. The trained global prediction model outputs the prediction results of the aforementioned target continuous values, supporting cross-organizational continuous value estimation and decision-making without sharing the original data.

[0099] In this embodiment of the invention, given that the local data of each participant cannot go out of the domain and that there are basic prior statistics and training interfaces, an integrated framework of collaborative feature alignment, sample balancing and label alignment is adopted to complete the iterative optimization of the global model with considerable computational and communication overhead. The proposed method comprehensively considers the impact of feature space inconsistency on parameter aggregation, gradient variance amplification and training oscillation caused by sample imbalance, and systematic bias and client offset caused by label distribution differences. It proposes a federated collaborative algorithm consisting of a global feature anchor mapping model, a conditional generative adversarial model based on global prediction distribution, and a prediction model based on regression knowledge distillation. The global feature anchor mapping model maps features from each endpoint to a common space and aligns statistical differences using distance metrics. It also generates supplementary samples to cover long-tail and sparse regions without exposing the original data. The prediction model based on regression knowledge distillation constrains predictions from each endpoint with a global soft label distribution, achieving label distribution consistency and stable convergence. On the server side, global anchor parameter aggregation, generator / discriminator parameter aggregation, and soft label distribution aggregation are executed respectively, and updates are sent to the client for continued local training, thereby obtaining a robust and accurate global model in heterogeneous scenarios.

[0100] Figure 3 This is a block diagram of a federated collaborative learning device for multidimensional heterogeneous data, illustrated according to an exemplary embodiment. The device is used in a federated collaborative learning method for multidimensional heterogeneous data. (Refer to...) Figure 3 The device includes a global feature anchor mapping module 310, a global feature anchor mapping module 320, and a prediction module 330 based on regression knowledge distillation. Wherein: The global feature anchor mapping module 310 is used to map the client's local samples to a public space for any client among multiple clients that do not share local samples, thereby obtaining mapped samples. The public space is obtained by the server performing federated aggregation of the local anchors of all clients to obtain the global anchor distribution, which is defined according to the global anchor distribution.

[0101] The global feature anchor mapping module 320 is used to input the hard labels and mapped samples of local samples into the global feature anchor mapping model to obtain generated samples that are consistent with the global anchor distribution; wherein, the parameters of the generator and discriminator in the global feature anchor mapping model are optimized by the server federating the parameters of all clients.

[0102] The prediction module 330 based on regression knowledge distillation is used to input the hard labels of local samples, mapped samples, and generated samples into the prediction model based on regression knowledge distillation to obtain the prediction results; wherein, the training process of the prediction model is guided by the global soft label distribution obtained by the server aggregating the soft labels of all clients.

[0103] In this embodiment of the invention, given that the local data of each participant cannot go out of the domain and that there are basic prior statistics and training interfaces, an integrated framework of collaborative feature alignment, sample balancing and label alignment is adopted to complete the iterative optimization of the global model with considerable computational and communication overhead. The proposed method comprehensively considers the impact of feature space inconsistency on parameter aggregation, gradient variance amplification and training oscillation caused by sample imbalance, and systematic bias and client offset caused by label distribution differences. It proposes a federated collaborative algorithm consisting of a global feature anchor mapping model, a conditional generative adversarial model based on global prediction distribution, and a prediction model based on regression knowledge distillation. The global feature anchor mapping model maps features from each endpoint to a common space and aligns statistical differences using distance metrics. It also generates supplementary samples to cover long-tail and sparse regions without exposing the original data. The prediction model based on regression knowledge distillation constrains predictions from each endpoint with a global soft label distribution, achieving label distribution consistency and stable convergence. On the server side, global anchor parameter aggregation, generator / discriminator parameter aggregation, and soft label distribution aggregation are executed respectively, and updates are sent to the client for continued local training, thereby obtaining a robust and accurate global model in heterogeneous scenarios.

[0104] Figure 4 This is a schematic diagram of the structure of a federated collaborative learning device for multidimensional heterogeneous data provided in an embodiment of the present invention, as shown below. Figure 4 As shown, federated collaborative learning devices for multidimensional heterogeneous data may include the above-mentioned Figure 3 The illustrated federated collaborative learning device for multidimensional heterogeneous data. Optionally, the federated collaborative learning device 410 for multidimensional heterogeneous data may include a first processor 2001.

[0105] Optionally, the federated collaborative learning device 410 for multidimensional heterogeneous data may also include a memory 2002 and a transceiver 2003.

[0106] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0107] The following is combined Figure 4 A detailed introduction to the various components of the federated collaborative learning device 410 for multidimensional heterogeneous data is provided below: The first processor 2001 is the control center of the federated collaborative learning device 410 for multidimensional heterogeneous data. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0108] Optionally, the first processor 2001 can perform various functions of the federated collaborative learning device 410 for multidimensional heterogeneous data by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0109] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0110] In a specific implementation, as one example, the federated collaborative learning device 410 for multidimensional heterogeneous data may also include multiple processors, for example... Figure 4 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0111] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0112] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be accessed through the interface circuit of the federated collaborative learning device 410 for multidimensional heterogeneous data. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0113] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0114] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0115] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected via the interface circuit of the federated collaborative learning device 410 for multidimensional heterogeneous data. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0116] It should be noted that, Figure 4 The structure of the federated collaborative learning device 410 for multidimensional heterogeneous data shown in the figure does not constitute a limitation on the router. Actual knowledge structure identification devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0117] Furthermore, the technical effects of the federated collaborative learning device 410 for multidimensional heterogeneous data can be referred to the technical effects of the federated collaborative learning method for multidimensional heterogeneous data described in the above method embodiments, and will not be repeated here.

[0118] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0119] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0120] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0121] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0122] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0123] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0124] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0125] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0126] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0127] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0128] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0129] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A federated collaborative learning method for multidimensional heterogeneous data, characterized in that, The method includes: S1. For any client among multiple clients that do not share local samples, the client's local samples are mapped to the public space through the global feature anchor mapping model to obtain the mapped samples; wherein, the public space is obtained by the server performing federated aggregation of the local anchors of all clients to obtain the global anchor distribution, according to the definition of the global anchor distribution; S2. Input the hard labels and mapped samples of the local samples into the global feature anchor mapping model to obtain generated samples that are consistent with the global anchor distribution; wherein, the parameters of the generator and discriminator in the global feature anchor mapping model are optimized by the server federating the parameters of all clients. S3. Input the hard labels, mapped samples, and generated samples of the local samples into the prediction model based on regression knowledge distillation to obtain the prediction results; wherein, the training process of the prediction model is guided by the global soft label distribution obtained by the server aggregating the soft labels of all clients.

2. The federated collaborative learning method for multidimensional heterogeneous data according to claim 1, characterized in that, The heterogeneity of the feature spaces of the multiple clients is quantified by defining a composite feature heterogeneity index. Imbalanced samples from multiple clients are quantified by defining a sample imbalance coefficient. The differences in tag distribution across multiple clients are determined by defining... The overall difference in label distribution among the datasets is quantified using an indicator.

3. The federated collaborative learning method for multidimensional heterogeneous data according to claim 1, characterized in that, The S1 step, which maps the client's local samples to the public space using a global feature anchor mapping model to obtain mapped samples, includes: The client's local samples are mapped to the initial public space through a local mapping network using a global feature anchor mapping model, resulting in mapped samples. Calculate the statistics of the mapped samples to obtain the local anchor points; The server performs federated aggregation of local anchor points from all clients to obtain a global anchor point distribution. Based on the global anchor point distribution and the federated round cycle, the common space is adaptively updated, and the local samples of the clients are mapped according to the updated common space.

4. The federated collaborative learning method for multidimensional heterogeneous data according to claim 1, characterized in that, The global feature anchor mapping model in S1 uses a multilayer perceptron. The optimization objective of the global feature anchor mapping model includes: the joint minimization of the reconstruction and prediction consistency term and the distribution alignment term; wherein the reconstruction and prediction consistency term adopts the mean squared error, and the distribution alignment term adopts the closed form of the second-order Wasserstein distance, the closed form of the second-order Wasserstein distance including the Bures distance. The optimization objective of the global feature anchor mapping model is shown in equation (1): （1） in, （2） （3） （4） In the formula, This represents the objective function of the global feature anchor mapping model. Indicates alignment loss. Indicates the reconstruction loss weights. Indicates the reconstruction loss. This represents the global anchor point loss weight. Indicates global anchor point loss. Indicates the number of samples. Indicates the first The original input feature vector of each sample, Indicates the first Sample features after mapping of each sample This represents the weighting coefficient based on the proportion of features. Indicates the first The number of features for each sample This represents the total number of features across all datasets. This represents the Wasserstein distance operator. Indicates in The local feature distribution obtained under the mapping Indicates the first The feature vector of each sample Represents the global feature distribution. This represents the total number of samples after mapping. This represents the local label prediction function. Indicates the first Features of the mapped samples The parameters representing the local model, Indicates the first The label value of each sample, The mean vector representing the global feature distribution. Indicates the first The mean vector of a local feature distribution. Indicates the Bures distance. The covariance matrix representing the global feature distribution. Indicates the first The covariance matrix of the local feature distribution.

5. The federated collaborative learning method for multidimensional heterogeneous data according to claim 1, characterized in that, The step S2, which involves inputting the hard labels and mapped samples of local samples into a global feature anchor mapping model to obtain generated samples consistent with the global anchor distribution, includes: The hard labels of local samples and global anchors are input into the generator of the global feature anchor mapping model to obtain generated samples that are consistent with the global anchor distribution. The mapped samples and generated samples are input into the discriminator of the global feature anchor mapping model for discrimination training, and the parameters of the trained generator and discriminator are obtained. The parameters of the trained generator and discriminator are periodically uploaded to the server.

6. The federated collaborative learning method for multidimensional heterogeneous data according to claim 1, characterized in that, In step S3, the hard labels of the local samples, the mapped samples, and the generated samples are input into a prediction model based on regression knowledge distillation to obtain prediction results, including: The mapped samples are input into a student model based on a prediction model derived from regression knowledge distillation to obtain a soft prediction. Hard predictions are obtained based on hard labels; The generated samples are input into a teacher model based on a regression knowledge distillation prediction model to obtain soft labels; Hard label supervision is constructed based on hard labels and hard predictions, and distillation supervision is constructed based on soft predictions and soft labels. The student model is then trained to obtain a well-trained student model, which is then used for prediction.

7. A federated collaborative learning device for multidimensional heterogeneous data, wherein the federated collaborative learning device for multidimensional heterogeneous data is used to implement the federated collaborative learning method for multidimensional heterogeneous data as described in any one of claims 1-6, characterized in that, The device includes: The global feature anchor mapping module is used to map the local samples of any client to a common space through the global feature anchor mapping model, which is used to obtain mapped samples. The common space is obtained by the server performing federated aggregation of the local anchors of all clients to obtain the global anchor distribution, which is defined according to the global anchor distribution. The global feature anchor mapping module is used to input the hard labels and mapped samples of local samples into the global feature anchor mapping model to obtain generated samples that are consistent with the global anchor distribution. The parameters of the generator and discriminator in the global feature anchor mapping model are optimized by the server federating the parameters of all clients. The prediction module based on regression knowledge distillation is used to input the hard labels of local samples, mapped samples, and generated samples into the prediction model based on regression knowledge distillation to obtain the prediction results. The training process of the prediction model is guided by the global soft label distribution obtained by the server aggregating the soft labels of all clients.

8. The federated collaborative learning device for multidimensional heterogeneous data according to claim 7, characterized in that, The heterogeneity of the feature spaces of the multiple clients is quantified by defining a composite feature heterogeneity index. Imbalanced samples from multiple clients are quantified by defining a sample imbalance coefficient. The differences in tag distribution across multiple clients are determined by defining... The overall difference in label distribution among the datasets is quantified using an indicator.

9. A federated collaborative learning device for multidimensional heterogeneous data, characterized in that, The federated collaborative learning device for multidimensional heterogeneous data includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 6.

Images