Federal learning structure adaptive method in multi-domain heterogeneous scene

Abstract

The invention provides a federal learning structure self-adaption method in a multi-domain heterogeneous scene. The method comprises the following steps: calculating the importance degree of a channel based on a plurality of data domains contained in each client; aggregating the importance of each client through the server and generating a global pruning structure; the client reactivates a part of channels based on the local importance, and constructs a final channel structure of the client; the server updates a global structure prototype according to the structure vectors constructed by the plurality of clients in the local training process; the server executes structure-aware parameter aggregation on the shared channel subset and updates a global model; according to the method, the uniform sparse structure is constructed at the server side, so that the structure consistency between the clients is ensured; and meanwhile, part of channels are selectively reactivated at the client according to the importance degree of the local region, so that coordination and unification of a unified structure and a local personalized structure are realized, and the stability and the representation capability during cross-client aggregation are improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and federated learning technology, and in particular to an adaptive method for federated learning structures in multi-domain heterogeneous scenarios. Background Technology

[0002] With increasing demands for data privacy and compliance, Federated Learning (FL) is widely used for collaborative training of data across devices and institutions. However, in real-world applications, significant data heterogeneity often exists both between and within clients. In particular, multi-domain skewed data from different sources, styles, and acquisition methods severely restricts the generalization performance and convergence stability of the global model. Furthermore, research shows that domain shift leads to inconsistent distributions between the feature space and label space, making it difficult for the model to learn robust representations shared across domains.

[0003] While existing technologies have made some progress in representation alignment, model personalization, and feature constraints, they generally overlook a crucial fact: multi-domain hybrid scenarios in federated learning not only require alignment in the feature space but also the achievement of fine-grained "channel-level structural consistency" at the network structure level. Especially when multiple domains coexist within the client, different domains often have completely different channel importance distributions. Existing federated methods cannot simultaneously identify domain-specific structures, shared structures, and multi-domain conflict structures, making it difficult for the global model to aggregate stably.

[0004] Therefore, in a multi-domain heterogeneous federated learning environment, there is an urgent need to propose a new structure adaptation method that can simultaneously model heterogeneity at the channel, structure, and domain levels, thereby improving the model's stability, generalization ability, and cross-domain robustness. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a federated learning structure adaptation method in multi-domain heterogeneous scenarios. This invention can ensure that different clients maintain structural consistency while still having the ability to process local specific domain information.

[0006] The technical solution of this invention is: an adaptive method for federated learning structures in multi-domain heterogeneous scenarios, comprising the following steps: S1) Calculate the importance of the channel based on the multiple data fields contained in each client; S2) Aggregate the importance of each client through the server and generate a global pruning structure; S3) The client reactivates some channels based on local importance and constructs the client's final channel structure; S4) During local training, the client constructs a structure vector based on its activated structure and local domain data and uploads it to the server. The server updates the global structure prototype based on the structure vectors of multiple clients. S5) The server performs structure-aware parameter aggregation on a subset of shared channels and updates the global model; S6) Determine whether the global model has reached the preset convergence target. If yes, the iteration ends; otherwise, return to step S1.

[0007] Preferably, in step S1), a gradient-based approach is used to construct a domain-level channel importance representation using the sensitivity of task loss to channels.

[0008] Preferably, in step S2), after the client calculates the channel importance, it uploads the client's importance to the server. The server then performs weighted aggregation based on the data scale of each client to form a global channel importance.

[0009] Preferably, in step S2), the server generates a global pruning mask based on the target sparsity threshold and the channel importance at the global scale. The global mask defines the structural backbone that all clients need to follow. In this way, the server explicitly requires clients to adhere to the same structural framework during training, ensuring the consistency of the cross-client structure.

[0010] Preferably, in step S3), the final channel structure of the client is represented as follows: ; (4) In the formula, Indicates the first The first client's The final structure of each channel; Indicates the first Global mask for each channel; Indicates the first The client recovered the first [database name] based on domain characteristics. A set of channels.

[0011] Preferably, in step S3), the client constructs a structure vector based on the activated structure and local domain data during the local training process.

[0012] Secondly, the present invention provides a federated learning structure adaptive system for multi-domain heterogeneous scenarios, including multiple clients and servers; the clients include an importance calculation module and a channel reactivation module; the servers include a global pruning module, a global structure prototype update module, and a global model update module. The importance calculation module allows each client to calculate the importance of a channel for each of its multiple data fields and upload the results to the server. The server uses a global pruning module to aggregate the importance of each client and generate a global pruning structure. The channel reactivation module of the client reactivates some channels based on local importance to construct the final channel structure of the client. The global structure prototype update module is used in the following ways: during local training, the client constructs a structure vector based on the final channel structure it activates and the local domain data, and uploads it to the server; the server's global structure prototype update module updates the global structure prototype based on the structure vectors of multiple clients. The global model update module of the server performs structure-aware parameter aggregation on a subset of shared channels and updates the global model.

[0013] The beneficial effects of this invention are as follows: 1. This invention obtains domain-level structural information by calculating the channel importance of each domain on the client side, enabling the server to identify structural differences across clients and reducing the problem of accidental deletion caused by uniform pruning from the source; 2. This invention ensures structural consistency among clients by constructing a unified sparse structure on the server side; at the same time, it selectively reactivates some channels on the client side according to the importance of the local domain, thereby achieving coordination and unification between the unified structure and the local personalized structure, and improving the stability and representation capability when aggregating across clients. 3. This invention enables the server to form a stable multi-domain structure prototype by constructing a structure vector on the client and uploading it to the server. Based on this prototype, the server performs structure-aware parameter aggregation and merges and updates only on a subset of shared channels, thereby further improving the model's generalization ability and training stability in a multi-domain environment. Attached Figure Description

[0014] Figure 1 This is a schematic flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the framework of the method of the present invention; Figure 3 This is a comparison chart of the performance and efficiency of the present invention under different sparsity settings; Figure 4 This is a schematic diagram comparing the performance of the present invention under different hyperparameter settings. Detailed Implementation

[0015] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings: Example 1 like Figure 1and 2 As shown, this embodiment provides an adaptive method for federated learning structures in multi-domain heterogeneous scenarios, including the following steps: S1) Each client calculates the importance of each domain channel based on its multi-domain data; In this embodiment, each client calculates the importance of channels for each of its multiple data domains to reflect the differences in the contribution of each domain to the network channels during training.

[0016] This embodiment employs a gradient-based approach, utilizing the sensitivity of task loss to channels to construct a domain-level channel importance representation, namely: ; (1) In the formula, Indicates client In the domain Next Sensitivity of each channel; Represents a random variable; Indicates client In the domain The amount of data on it; Indicates client In the domain Data samples; Represents the model parameter vector; Indicates the relationship with the first Scalar parameters related to each channel; This indicates that the loss function is applied to the parameters. The first-order partial derivative.

[0017] This embodiment enables the client to identify the importance patterns of different domains at the structural level through the above operations, providing accurate domain-level information for the server to build a unified sparse structure.

[0018] S2) Aggregate the importance of each client through the server and generate a global pruning structure; In this embodiment, after the client calculates the channel importance, it uploads the client's importance to the server. The server then performs weighted aggregation based on the data scale of each client to form a global-scale channel importance, i.e.: ; (2) In the formula, Indicates the first The global importance of each channel; Indicates the number of clients participating in federated training; Indicates the client For the Normalized sensitivity calculated for each channel; Indicates the client The size of the local dataset is used as a weighting factor to balance the contributions of different clients; The server generates a global pruning mask based on the target sparsity threshold and the channel importance at a global scale, i.e.: ; (3) In the formula, Indicates the first Global mask for each channel; Indicates the target sparsity threshold; Among them, the global mask The server defines a common structural framework that all clients must follow. In this way, the server can explicitly require clients to adhere to the same structural framework during training, fundamentally ensuring cross-client structural consistency.

[0019] S3) The client reactivates some channels based on local importance and constructs the client's final channel structure; In this embodiment, although the global structure ensures consistency across clients, the channel importance distribution across different domains is not uniform. If only a global mask is used, some channels that are crucial to a specific domain may be mistakenly deleted. Therefore, in this embodiment, the client reactivates some channels based on its own domain differences.

[0020] The channel structure ultimately used by the client is represented as follows: ; (4) In the formula, Indicates the first The first client's The final structure of each channel; This indicates the first [unit / item] determined by the server at the global level. The global structure selection results for each channel; Indicates the first The client recovered the first [database name] based on domain characteristics. A set of channels.

[0021] Through this mechanism, the client can flexibly restore the channels required locally, thereby significantly improving the expressive power in the local domain while still maintaining alignment with the global model in the core structure.

[0022] S4) The client constructs a structure vector during local training, and the server updates the global structure prototype based on the structure vectors of multiple clients. In this embodiment, to enable the server to understand the differences in multi-domain structures, the client constructs a structure vector based on the final channel structure it activates and the local domain data during local training. Then the client will send the structure vector. Upload to the server; that is: ; (5) in, Indicates client Based on local domain The original structure vector is constructed from the activation structure and channel importance. Indicates client In the domain The amount of data on it; Indicates belonging to the domain A collection of clients; For the client The weighted structure vector uploaded to the server.

[0023] The server is based on the structure vectors of multiple clients. Continuously update the global structure prototype ; (6) in, Indicates the domain maintained by the server The global structure prototype; For domain A collection of clients; The structure vector uploaded by the client.

[0024] By updating the global structure prototype to reflect the overall structural trend of multiple domains, the global structure prototype plays an important role in subsequent aggregation. It not only enables the server to further optimize the aggregation strategy according to the domain characteristics, but also enhances the robustness to domain offsets at the structural level.

[0025] S5) The server performs structure-aware parameter aggregation on a subset of shared channels and updates the global model; specifically as follows: Let the global pruning mask be: ; Shared channel subset is defined as , in, This represents a set of shared channel indexes that are retained by all clients. Let the client be in the t-th round of communication. In the The local parameters on each channel are , For the client For local datasets of a certain size, the server performs structure-aware weighted aggregation on a subset of shared channels and updates the global model parameters. : In the formula, Indicates the number of clients participating in federated training; For channels that do not belong to a subset of shared channels, the server maintains the global parameters from the previous round unchanged, that is:

[0026] In this embodiment, the server performs parameter aggregation only on a subset of channels that are activated by all clients, thereby avoiding aggregation misalignment caused by the final channel structure reactivated by each client.

[0027] S6) Determine whether the global model has reached the preset convergence target. If yes, the iteration ends; otherwise, return to step S1.

[0028] This embodiment verifies the existence and effectiveness of multi-domain channel importance modeling, global pruning structure, domain-sensitive structure prototype, and structure-aware aggregation through ablation experiments. The results are shown in Tables 1, 2, and 3. When any module such as domain-sensitive pruning, domain structure prototype, or structure-aware aggregation is removed, the accuracy of the global model decreases, especially on datasets with significant domain offsets. This indicates that each step of the present invention plays an irreplaceable role in the overall structural adaptability and plays a key role in improving structural consistency and domain-specific expressive power.

[0029] Table 1. The role of each step on the Digits dataset. Note: DSFP stands for Domain Feature Aware Pruning; DSR stands for Domain Feature Recovery; SA stands for Structure Aware Aggregation.

[0030] Table 2. Impact of Module Combinations on the Performance of the Office-Caltech Dataset Table 3. Impact of Module Combinations on PACS Dataset Classification Performance On multi-domain datasets (Digits, Office-Caltech, and PACS), the method in this embodiment consistently improves the accuracy of the global model under mixed multi-domain distributions and significantly reduces the number of communication rounds required for convergence. The results are shown in Table 4. On multiple multi-domain datasets, the method in this embodiment achieves the highest average accuracy while maintaining the fewest communication rounds and low model complexity. This demonstrates that this embodiment achieves a good balance between structural consistency and domain-specific expression, significantly improving the generalization ability of the global model.

[0031] Table 4. Comparison of classification performance of various methods on the Digits, Office-Caltech, and PACS datasets. This embodiment achieves model structure compression through a global sparse structure and combines it with a client-side reactivation strategy for personalized supplementation, enabling the model to significantly reduce parameter size and communication costs while maintaining high accuracy. Figure 3 As shown, this embodiment of FedFIP can simultaneously achieve high accuracy and significant model compression on different datasets. Figure 3 (a) in the figure represents the improvement in model accuracy on the Digits dataset, as well as the ratio of the number of parameters (Para) and the computational cost (FLOPs) to the original model, which shows that FedFIP can reduce the number of parameters and computational cost while maintaining higher accuracy. Figure 3 (b) in the table represents the model accuracy, parameter percentage, and FLOPs percentage on the Office-Caltech dataset, demonstrating the efficiency and performance advantages of FedFIP on this dataset.

[0032] Figure 3 This demonstrates that the method in this embodiment maintains high accuracy under different sparsity settings, and exhibits good performance stability even at high compression rates, highlighting the advantages of this embodiment in structural compression and performance preservation.

[0033] This embodiment improves structural consistency across different domains by constructing a domain-sensitive structure prototype, thereby enhancing the model's generalization ability across cross-domain data. Figure 4 The experiment demonstrates systematic experiments with four hyperparameters under different settings to verify structural stability and the accuracy of the final model. Each subplot illustrates the following: Global Sparsity Ratio Impact on structural consistency and classification performance The smaller the size, the denser the global structure and the more shared structures. It performs best when it is moderate. If the sparsity is too large, too many channels will be cut off, resulting in performance degradation. This verifies that the global structure of FedFIP can still maintain stable performance under different sparsity settings. The impact of the local importance threshold θ on the reactivation range and the ability to preserve domain features: as the local importance threshold θ increases, the local reactivation mechanism changes its ability to adapt to multi-domain features. The impact of the prototype update coefficient μ on the stability of cross-domain structural aggregation under different update rates is shown, demonstrating the performance changes of global prototypes under different update rates μ. By comparing the impact of the ontrastive regularization coefficient λ on overall performance when adjusting the inter-domain structural discriminativeness, we can verify the importance of "cross-domain structural contrast" in FedFIP for stability and generalization performance.

[0034] As can be seen from the figure, the performance remains stable within a reasonable range for each hyperparameter, further verifying the robustness and effectiveness of the proposed structure-aware mechanism. With adjustments to the reactivation threshold, sparsity rate, or structural prototype update coefficients, the method in this embodiment exhibits a stable trend, indicating that the structure adaptation mechanism of this embodiment has strong robustness and is insensitive to hyperparameters.

[0035] Furthermore, this embodiment demonstrates stable performance improvements across backbone networks of varying sizes (such as ResNet-10, ResNet-18, and ResNet-50), indicating the invention's excellent scalability. Table 5 shows a performance comparison under different model capacities. As network capacity increases, the method in this embodiment continues to improve model performance while maintaining training stability, demonstrating its adaptability to various deep network structures and broad practical application potential.

[0036] Table 5. Comparison of classification performance of different backbone networks on multiple datasets. Example 2 This embodiment provides a federated learning structure adaptive system for multi-domain heterogeneous scenarios, including multiple clients and servers; the clients include an importance calculation module and a channel reactivation module; the servers include a global pruning module, a global structure prototype update module, and a global model update module. The importance calculation module allows each client to calculate the importance of a channel for each of its multiple data fields and upload the results to the server. This embodiment employs a gradient-based approach, utilizing the sensitivity of task loss to channels to construct a domain-level channel importance representation, namely: ; (1) In the formula, Indicates client In the domain Next Sensitivity of each channel; Represents a random variable; Indicates client In the domain The amount of data on it; Indicates client In the domain Data samples; Represents the model parameter vector; Indicates the relationship with the first Scalar parameters related to each channel; This indicates that the loss function is applied to the parameters. The first-order partial derivative.

[0037] The server uses a global pruning module to aggregate the importance of each client and generate a global pruning structure. In this embodiment, after the client calculates the channel importance, it uploads the client's importance to the server. The server then performs weighted aggregation based on the data scale of each client to form a global-scale channel importance, i.e.: ; (2) In the formula, Indicates the first The global importance of each channel; Indicates the number of clients participating in federated training; Indicates the client For the Normalized sensitivity calculated for each channel; Indicates the client The size of the local dataset is used as a weighting factor to balance the contributions of different clients; The server generates a global pruning mask based on the target sparsity threshold and the channel importance at a global scale, i.e.: ; (3) In the formula, Indicates the first Global mask for each channel; Indicates the target sparsity threshold; Among them, the global mask The server defines a common structural framework that all clients must follow. In this way, the server can explicitly require clients to adhere to the same structural framework during training, fundamentally ensuring cross-client structural consistency.

[0038] The channel reactivation module of the client reactivates some channels based on local importance to construct the final channel structure of the client. The channel structure ultimately used by the client is represented as follows: ; (4) In the formula, Indicates the first The first client's The final structure of each channel; This indicates the first [unit / item] determined by the server at the global level. The global structure selection results for each channel; Indicates the first The client recovered the first [database name] based on domain characteristics. A set of channels.

[0039] The global structure prototype update module allows the client to construct a structure vector based on the structure of its activated final channel and local domain data during local training. And uploaded to the server, the server's global structure prototype update module updates the structure vectors of multiple clients. Update global structure prototype ;Right now: ; (5) ; (6) in, Indicates client Based on local domain The original structure vector is constructed from the activation structure and channel importance. Indicates client In the domain The amount of data on it; Indicates belonging to the domain A collection of clients; For the client The weighted structure vector uploaded to the server; Indicates the domain maintained by the server The global structure prototype; The global model update module of the server performs structure-aware parameter aggregation on a subset of shared channels and updates the global model. Let the global pruning mask be: ; Shared channel subset is defined as , in, This represents a set of shared channel indexes that are retained by all clients. Let the client be in the t-th round of communication. In the The local parameters on each channel are , For the client For local datasets of a certain size, the server performs structure-aware weighted aggregation on a subset of shared channels and updates the global model parameters. : In the formula, Indicates the number of clients participating in federated training; For channels that do not belong to a subset of shared channels, the server maintains the global parameters from the previous round unchanged, that is: In this embodiment, the server performs parameter aggregation only on a subset of channels that are activated by all clients, thereby avoiding aggregation misalignment caused by the final channel structure reactivated by each client.

[0040] The embodiments and descriptions above are merely illustrative of the principles and preferred embodiments of the present invention. Various changes and modifications may be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed.

Claims

1. A federated learning structure adaptation method in multi-domain heterogeneous scenarios, characterized in that, Includes the following steps: S1) Calculate the importance of the channel based on the multiple data fields contained in each client; S2) Aggregate the importance of each client through the server and generate a global pruning structure; S3) The client reactivates some channels based on local importance and constructs the client's final channel structure; S4) During local training, the client constructs a structure vector based on its activated structure and local domain data and uploads it to the server. The server updates the global structure prototype based on the structure vectors of multiple clients. S5) The server performs structure-aware parameter aggregation on a subset of shared channels and updates the global model; S6) Determine whether the global model has reached the preset convergence target. If yes, the iteration ends; otherwise, return to step S1.

2. The adaptive federated learning structure method in a multi-domain heterogeneous scenario according to claim 1, characterized in that: In step S1), a gradient-based approach is used to construct a domain-level channel importance representation based on the sensitivity of the task loss to the channel, i.e.: ； （1） In the formula, Indicates client In the domain Next Sensitivity of each channel; Represents a random variable; Indicates client In the domain The amount of data on it; Indicates client In the domain Data samples; Represents the model parameter vector; Indicates the relationship with the first Scalar parameters related to each channel; This indicates that the loss function is applied to the parameters. The first-order partial derivative.

3. The adaptive federated learning structure method in a multi-domain heterogeneous scenario according to claim 2, characterized in that: In step S2), after the client calculates the channel importance, it uploads the client's importance to the server. The server then performs weighted aggregation based on the data size of each client to form a global-scale channel importance, i.e.: ； （2） In the formula, Indicates the first The global importance of each channel; Indicates the number of clients participating in federated training; Indicates the client For the Normalized sensitivity calculated for each channel; Indicates the client The size of the local dataset is used as a weighting factor to balance the contributions of different clients.

4. The adaptive federated learning structure method in a multi-domain heterogeneous scenario according to claim 3, characterized in that: In step S2), the server generates a global pruning mask based on the target sparsity threshold and the channel importance at the global scale, i.e.: ； （3） In the formula, Indicates the first Global mask for each channel; This represents the target sparsity threshold.

5. The adaptive federated learning structure method in a multi-domain heterogeneous scenario according to claim 4, characterized in that: In step S3), the final channel structure of the client is represented as follows: ； （4） In the formula, Indicates the first The first client's The final structure of each channel; Indicates the first Global mask for each channel; Indicates the first The client recovered the first [database name] based on domain characteristics. A set of channels.

6. The adaptive federated learning structure method in a multi-domain heterogeneous scenario according to claim 5, characterized in that: In step S4), the client constructs a structure vector based on the final channel structure it activates and the local domain data during local training. Specifically: ； （5） in, Indicates client Based on local domain The original structure vector is constructed from the activation structure and channel importance. Indicates client In the domain The amount of data on it; Indicates belonging to the domain A collection of clients; For the client The weighted structure vector uploaded to the server.

7. The adaptive federated learning structure method in a multi-domain heterogeneous scenario according to claim 6, characterized in that: In step S4), the server determines the structure vectors of multiple clients. Continuously update the global structure prototype Specifically: ； （6） in, Indicates the domain maintained by the server The global structure prototype; For domain A collection of clients; The structure vector uploaded by the client.

8. The adaptive federated learning structure method in a multi-domain heterogeneous scenario according to claim 7, characterized in that: In step S5), the server performs structure-aware parameter aggregation on a subset of shared channels and updates the global model; specifically as follows: Let the global pruning mask be: ;in, Indicates the number of channels; superscript Indicates the global scope; Shared channel subset is defined as , This represents a set of shared channel indexes that are retained by all clients. Let the client be in the t-th round of communication. In the The local parameters on each channel are , For the client For local datasets of a certain size, the server performs structure-aware weighted aggregation on a subset of shared channels and updates the global model parameters. : In the formula, This indicates the number of clients participating in the federated training.

9. The adaptive federated learning structure method in a multi-domain heterogeneous scenario according to claim 8, characterized in that: In step S5), for channels that do not belong to a subset of shared channels, the server maintains the global parameters from the previous round unchanged, that is: The server performs parameter aggregation only on a subset of channels that are commonly activated by all clients.

10. A federated learning structure adaptive system for multi-domain heterogeneous scenarios, characterized in that, The system described herein utilizes the method described in any one of claims 1-9 to achieve structure-adaptive federated learning. The system includes multiple clients and servers. The clients include an importance calculation module and a channel reactivation module. The servers include a global pruning module, a global structure prototype update module, and a global model update module. The importance calculation module allows each client to calculate the importance of a channel for each of its multiple data fields and upload the results to the server. The server uses a global pruning module to aggregate the importance of each client and generate a global pruning structure. The channel reactivation module of the client reactivates some channels based on local importance to construct the final channel structure of the client. The global structure prototype update module is used in the following ways: during local training, the client constructs a structure vector based on the final channel structure it activates and the local domain data, and uploads it to the server; the server's global structure prototype update module updates the global structure prototype based on the structure vectors of multiple clients. The global model update module of the server performs structure-aware parameter aggregation on a subset of shared channels and updates the global model.

Images