A model dynamic pruning and alignment method based on federated learning

Abstract

The application belongs to the technical field of federated learning and model optimization, and discloses a model dynamic pruning and alignment method based on federated learning. The application can autonomously adjust the local model pruning degree according to the resource state and training performance feedback of each client within the range of terminal hardware resource constraints, realize adaptive optimization of model complexity, and generate a pruning mask by fusing the global consensus of the federation and the personalized needs of the terminal when constructing the local model, so as to construct a lightweight subnetwork with global generalization ability and local task adaptation characteristics. In the local training stage, a global knowledge alignment constraint mechanism is introduced to realize the unity of local model optimization and federated collaborative consistency. The application can effectively avoid the resource and performance imbalance problem caused by the fixed pruning or unified pruning rate strategy, and simultaneously consider global generalization and local adaptation, thereby significantly improving the practicability and reliability of the federated learning system in the heterogeneous terminal scene.

Description

Technical Field

[0001] This application belongs to the field of federated learning and model optimization technology, specifically relating to a method for dynamic pruning and alignment of models based on federated learning. Background Technology

[0002] In edge deployment scenarios for artificial intelligence models, terminal devices often face stringent requirements for data privacy protection. Simultaneously, the data types, data distributions, and task objectives processed by different terminals vary significantly, necessitating the construction of personalized models tailored to their specific data characteristics. Furthermore, terminal devices exhibit high heterogeneity in hardware resource configurations such as computing power, memory capacity, and communication bandwidth. To meet the requirements of real-time inference and low latency, models need to be lightweighted to adapt to resource-constrained deployment environments. To address the collaborative challenges of privacy protection, personalized modeling, and efficient resource utilization, federated learning, as a distributed machine learning paradigm, has been introduced. It achieves multi-terminal collaborative optimization while protecting data privacy by training models locally on the terminal and only uploading model parameters or parameter updates to the server for aggregation.

[0003] Current methods for lightweighting and federated training models for edge heterogeneous environments still have significant shortcomings, mainly manifested in the following aspects: 1) The model structure is rigid and cannot adapt to the real-time state of terminal model training. When resource-constrained terminals run large models, although the performance is high, resources are scarce and inference is sluggish; when high-performance terminals use small models, although the resource consumption is low, the performance is insufficient and computing power is wasted, making it difficult to achieve an effective balance between resource constraints and model performance. Even when some methods introduce pruning strategies, they often use a uniform pruning rate or have the pruning configuration centrally set by the server, which is difficult to reflect the differences in hardware resources, training performance, and local data distribution among different clients. 2) Due to the lack of global consensus in local model construction, the underlying architecture of the model has inherent generalization defects. Terminals only rely on local data to build the topology and parameter system, without integrating multi-terminal consensus generalization knowledge, resulting in an inherent lack of cross-domain generalization support at the structural level. When facing unknown distributed data or cross-domain tasks, the generalization performance is significantly limited. 3) Local model training does not achieve knowledge alignment with the global model, limiting performance improvement. The terminal optimizes the model based solely on local data without establishing a knowledge collaboration mechanism with the global model. This causes the model to be overly biased towards local characteristics, making it unable to leverage global collaborative training to improve performance or approach the core performance level of the global model, ultimately hindering the overall collaborative effect of the federated system. Summary of the Invention

[0004] To address the aforementioned technical issues, this application provides a model dynamic pruning and alignment method based on federated learning. This method can effectively avoid resource and performance imbalances caused by fixed pruning or uniform pruning rate strategies, while taking into account both global generalization and local adaptation, significantly improving the practicality and reliability of federated learning systems in heterogeneous terminal scenarios.

[0005] To achieve the above objectives, this application employs the following technical solution:

[0006] This application presents a model dynamic pruning and alignment method based on federated learning, which specifically includes the following steps:

[0007] Step 1: Data Preparation and System Initialization: The server establishes a public proxy dataset and constructs an initial global dense model. The client loads the initial global dense model. When a client first connects to the federated system, a lightweight resource probe is automatically run. This probe assesses the client's hardware capabilities using three-dimensional resource metrics. Based on these metrics, the client calculates a fusion resource score and maps the overall resource score to a lower limit for the pruning rate. ;

[0008] Step 2, Adaptive Adjustment of Client-Side Pruning Rate: Calculate performance changes based on historical client training performance, and adjust based on comprehensive resource scores. A resource adjustment factor is constructed, and this factor, along with performance changes, is introduced into the pruning rate update process to obtain the adjusted pruning rate for the next cycle. ;

[0009] Step 3: Generate client mask and construct client subnetwork: The client is based on the adjusted next-cycle pruning rate. Get the local initial mask The client will use the local initial mask. The comprehensive perturbation retention score for each parameter unit and the pruning rate for the next cycle. Uploaded to the server; the server receives the local initial mask uploaded by each client. Maintain score with integrated disturbances Next, a client consensus mask is generated. After receiving the consensus mask distributed by the server, the client completes mask merging, pruning rate verification and updating to obtain the final mask. The client then uses the final mask as the filtering criterion to select from the global dense model distributed by the server. Extract the parameter units marked as 1 and construct the client subnetwork. ;

[0010] Step 4: The client performs multiple rounds of local training on the client subnetwork constructed in Step 3 and evaluates the performance of the trained subnetwork. After the client completes local training, it calculates the parameter increment only for the parameter units marked as 1 in the final mask and uploads the data packet. The server receives the data packets uploaded by each client and aggregates the global update of the parameter units to obtain the updated global model.

[0011] Step 5, the client uses The training round serves as a testing cycle, with the client maintaining the current final mask. Continue training using the current client subnetwork, unchanged; when completed... After each training round, the performance changes during the detection period are used to determine whether it is necessary to regenerate the mask and reconstruct the subnetwork.

[0012] Step 6: Repeat steps 2-5 to determine the final sub-network model that is adapted to local hardware resources and meets the task performance requirements.

[0013] A further improvement in this application is that step 1 specifically includes the following steps:

[0014] Step 1.1: Set up the initial global dense model and basic environment on the server side: The server first builds the initial global dense model. Meanwhile, the server establishes a public agent dataset and sets global training hyperparameters, including temperature coefficient and dual-objective loss weight coefficient.

[0015] Step 1.2, Probe Client Hardware Resources: When a client first connects to the federated system, a lightweight resource probe is automatically run. This lightweight resource probe evaluates the client's hardware capabilities using three-dimensional resource metrics, including: computing power. Memory capacity and communication bandwidth Based on three-dimensional resource indicators, the client calculates a comprehensive resource score through weighted fusion. :

[0016]

[0017] in, The normalization function maps the metrics of each dimension to... interval, , , These are the weighting coefficients;

[0018] Step 1.3: Set upper and lower limits for pruning rate: based on comprehensive resource score. Set a lower limit for the pruning rate and use the Sigmoid function to calculate the overall resource score. Mapped to the lower limit of pruning rate :

[0019]

[0020] in, To adjust the coefficient, As the offset coefficient, ensure At the same time, set an upper limit on the pruning rate. .

[0021] A further improvement in this application is that step 2 specifically includes the following steps:

[0022] Step 2.1: Calculate the performance change: Use the performance at the time of the last effective adjustment of the pruning rate as the initial performance baseline. To trigger client subnet updates The training rounds are defined as statistical intervals, and the performance value of the last round within that interval is taken. Calculate the change in performance :

[0023]

[0024] in, As an initial performance benchmark, if the performance evaluation metric used to quantify task completion capability during client subnetwork training is accuracy or F1-score: This indicates that the performance is better than the initial performance benchmark. , This indicates that the performance is inferior to the initial performance baseline. If the indicator is a loss value: This indicates that the performance is better than the initial performance benchmark. , This indicates that the performance is inferior to the initial performance baseline. ;

[0025] Step 2.2: Calculate the resource adjustment factor based on the comprehensive resource score. Construct resource adjustment factors and define clients. Resource adjustment factors :

[0026] ;

[0027] Step 2.3, Adjust the pruning rate: Adjust the resource adjustment factor. With performance change A joint pruning rate update process is introduced to obtain the adjusted pruning rate for the next cycle:

[0028]

[0029] in, For the client Current pruning rate, Adjust the step size based on the base. It is the hyperbolic tangent function. , The pruning rate has been increasing continuously. , The pruning rate decreased continuously. This is the boundary constraint operator for the pruning rate.

[0030] A further improvement in this application is that, in step 3, the client bases the adjusted pruning rate for the next cycle. Get the local initial mask Specifically, it includes the following steps:

[0031] Step 3.1.1: The client bases the adjusted pruning rate for the next cycle. The initial global dense model issued by the server Perform parameter unit-level importance assessment: The client will use the initial global dense model. Load it locally and freeze its parameters without training or updating; then apply the initial globally dense model. Perform forward reasoning to obtain the complete globally dense model. Output:

[0032] ,

[0033] in, For classification tasks, a complete globally dense model In the sample Output distribution on Represents a globally dense model For input samples The output, Temperature coefficient;

[0034] Step 3.1.2, for any parameter unit The client-side construction only masks this parameter unit. The perturbation model:

[0035] ,

[0036] in, For parameter unit Corresponding shielding indicator matrix, parameter unit The corresponding position is assigned a value of 1, and the remaining positions are assigned a value of 0. Represents element-wise multiplication; perturbation model In the sample The output distribution on is represented as:

[0037] ,

[0038] The client is based on the complete global density model. Output distribution With perturbation model Output distribution Differences between them, calculation parameter unit Maintain sensitivity to local output :

[0039]

[0040] in, Indicates KL divergence;

[0041] Step 3.1.3, Client-side parameter calculation unit Local task loss perturbation :

[0042]

[0043] in, This represents the local task loss function;

[0044] Step 3.1.4: Based on the local task loss perturbation amount and parameter unit Maintain sensitivity of local output Client-side parameter calculation unit The overall perturbation maintenance score :

[0045]

[0046] in, Units for comprehensive characterization parameters The higher the score, the greater the contribution of the parameter unit to preserving global output knowledge and local task performance. The stronger the disturbance caused by the behavior, the more it should be preserved;

[0047] Step 3.1.5: Set up a global dense model The total number of intermediate parameter units is Client Based on the adjusted pruning rate for the next cycle Determine the number of parameter units to retain. The client maintains a score for all parameter units based on the overall disturbance. Sort by high to low, select the top One parameter unit is marked as 1, and the remaining parameter units are marked as 0, thus obtaining the local initial mask. The client will use the local initial mask. The corresponding comprehensive disturbance preservation score And the adjusted pruning rate for the next cycle Uploaded to the server.

[0048] A further improvement of this application is that, in step 3, the server receives the local initial mask uploaded by each client. Maintain score with integrated disturbances Next, a client consensus mask is generated, which includes the following steps:

[0049] Step 3.2.1: Set the parameter unit The overall perturbation maintenance score Introducing a consensus generation process, for parameter units Server calculation parameter unit consensus score :

[0050]

[0051] in, Indicates client Parameter unit in local initial mask The state of preservation; This indicates the number of clients participating in the current round of federated training;

[0052] Step 3.2.2: The server, based on the client... Next cycle pruning rate Determine the number of consensus parameter units retained by the server. The server scores according to consensus. Sort by high to low, select the top One parameter unit is marked as 1, and the remaining parameter units are marked as 0, thus obtaining the client... consensus mask The server will use the client consensus mask. Distribute to the corresponding clients for use in the subsequent final mask generation.

[0053] A further improvement in this application is that, in step 3, after the client receives the consensus mask distributed by the server, it completes mask merging, pruning rate verification and updating to obtain the final mask, specifically including the following steps:

[0054] Step 3.3.1: Execute the client consensus mask With local initial mask The union operation yields the candidate union mask. ,statistics The number of 1s The current pruning rate is calculated based on the number of parameter units retained in the candidate union mask. Verify whether the current pruning rate meets the set lower limit for pruning rate:

[0055] like This indicates that the number of parameter units retained by the candidate union mask does not exceed the capacity of the client hardware, so there is no need to adjust the parameter units, and the candidate union mask can be directly used as the final mask. The current pruning rate will be used as the actual pruning rate for the client. ;

[0056] like This indicates that the candidate union mask retains too many parameter units, and it is necessary to remove some parameter units marked as 1 to improve the pruning rate. Calculate the number of units to be removed. For parameter units that need to be removed, priority should be given to selecting them from the parameter units newly added in the consensus mask. The client should then score these parameter units according to consensus criteria. Sort by score from lowest to highest, and select the lowest score. Each parameter unit, in The flag in the image is changed from 1 to 0 to obtain the final mask. ; Obtain the final mask Then, calculate the final mask. The final number of 1s Calculate the updated actual pruning rate ;

[0057] Step 3.3.2, the client uses the final mask. The global dense model is used as the selection criterion. Extract the parameter units marked as 1 and construct the client subnetwork. .

[0058] A further improvement of this application is that step 4 specifically includes the following steps:

[0059] Step 4.1: The client constructs the resulting client subnetwork. Multiple rounds of local training were conducted, and during the training process, the final mask was determined. Keeping the parameters constant, the client only updates the parameter units marked as 1 by the final mask. Parameter units not retained do not participate in forward propagation, backward propagation, or parameter updates. The client uses a dual-objective loss function that balances local data fitting and global knowledge alignment for training, as shown in the formula:

[0060]

[0061] in, These are the weighting coefficients. , For local tasks, the loss is used; for classification tasks, cross-entropy loss is used. The regression task uses mean squared error loss. , Loss for global knowledge alignment:

[0062]

[0063] in, Logits output for the client subnet. for The output Logits, The temperature coefficient and To control the smoothness of the distribution;

[0064] Step 4.2: After the client completes multiple rounds of local training, the performance of the trained client subnetwork is evaluated using a local validation set independent of the training set to obtain the client performance score. ;

[0065] Step 4.3: After the client completes local training, only the final mask is processed. The parameter increment is calculated for the parameter cell marked as 1, for any retained parameter cell. Its parameter unit increment is expressed as:

[0066]

[0067] in, For the client Subnetwork parameters before training These are the parameters of the trained subnetwork. This indicates that the parameter unit is preserved in the final mask and participates in local training; client The overall parameter increment of the upload is denoted as The client uploads data packets to the server. These data packets include... ;

[0068] Step 4.4: After receiving the data packets uploaded by each client, the server determines the final mask for each client. Determine which parameter units were updated for a globally dense model. Any parameter unit The server first determines the set of clients that will participate in updating the parameter unit. The global update value of this parameter unit:

[0069]

[0070] in, Represents the parameter unit obtained from server aggregation Global update volume Indicates client Uploaded parameter unit The incremental change; the server applies the aggregated global update to the global dense model. Corresponding parameter unit: After aggregating all parameter units, the server obtains the updated global model. .

[0071] A further improvement in this application is that step 5 specifically includes the following steps:

[0072] Step 5.1: Set the detection cycle and performance index collection: The client follows the preset... Each training round is a testing cycle. In each testing cycle, the client trains the current client subnetwork according to the training process in step 4, and records the performance score on the local validation set after each training round.

[0073] Step 5.2, End-of-cycle performance status determination: When completed... After each training round, a performance check is automatically triggered. Based on the performance score within that period, it is determined whether the client sub-network structure needs to be adjusted. Specifically:

[0074] Performance Stagnation Detection: Calculation The performance improvement rate per round of training is calculated using the following formula: ;like All wheel boost rates are below the threshold for performance stagnation. This is determined to be a performance stagnation;

[0075] Performance degradation determination: If If a performance shows a decreasing trend for two or more consecutive rounds, it is considered a performance degradation.

[0076] If either performance stagnation or performance degradation is met, the current client subnetwork structure is determined to need updating; if neither condition is met, the current client subnetwork structure is determined not to need updating.

[0077] Step 5.3: Based on the performance status assessment results in Step 5.2, update the client subnetwork or continue training.

[0078] If the performance status indicates that the client sub-network structure needs adjustment, repeat steps 2-3 to update the final mask. After the mask update is complete, reconstruct the client sub-network based on the new mask and continue to the next step. Rotational training;

[0079] If the performance status indicates that no adjustment to the client subnet structure is needed, the pruning rate adjustment and mask generation process will not be triggered. In this case, the server will no longer distribute the complete global model, and the server will use the final mask recorded by the client. From the latest global model The latest subnet is extracted and distributed to the client. After receiving the client's subnet, the client continues to execute the next one. The training cycle continues until the next testing cycle is completed, at which point the performance evaluation process is repeated.

[0080] The beneficial effects of this application are:

[0081] This application achieves a dynamic balance between model lightweighting and performance optimization: Through a pruning rate adaptive adjustment mechanism jointly driven by performance feedback and resource status, this application enables each terminal client to autonomously determine and dynamically adjust its local model pruning rate based on its own hardware resource constraints and actual training performance changes, rather than using a fixed pruning rate or a server-uniformly distributed pruning configuration. This approach not only overcomes the static limitations of fixed pruning strategies in adapting to changes in resource availability and task performance, but also allows clients under different resource conditions to form differentiated model structural complexities. While meeting the requirements for lightweight deployment, it maintains a reasonable match between model expressive power, inference efficiency, and resource consumption, improving the environmental adaptability and resource utilization of the terminal model.

[0082] This application enhances the local adaptability and global generalization capability of the client subnetwork: By integrating a personalized mask generation method with global consensus, this application embeds common knowledge from multiple terminals and key features of local tasks during the terminal model construction stage. This enables the model to not only adapt to local tasks but also form a general understanding of diverse data distributions, effectively enhancing its robustness and generalization capability across scenarios and in unseen data scenarios. Simultaneously, the integration of global consensus provides each terminal model with a collaborative cognitive foundation, reducing output fluctuations caused by model differences in federated collaboration and ensuring the collaborative reliability and result stability of the federated system.

[0083] This application optimizes federated training efficiency and reduces communication overhead: During the client-side training phase, global knowledge alignment constraints guide the local update direction, ensuring that the client's sub-network training results align with the global model's collaborative optimization objective, thus reducing update conflicts and performance fluctuations during federated aggregation. Simultaneously, the client only uploads the increments of parameter units retained in the final mask, and the server only performs performance-weighted aggregation on the actually updated parameter units, avoiding the transmission of complete model parameters and invalid parameter aggregation. This maintains model performance while reducing communication load, improving the training efficiency and practical deployment feasibility of the federated learning system. Attached Figure Description

[0084] Figure 1 This is a framework diagram of the dynamic pruning and alignment method of the model in this application.

[0085] Figure 2 This is a flowchart illustrating the process of this application.

[0086] Figure 3 This is a schematic diagram illustrating the adaptive adjustment of the client-side pruning rate in this application.

[0087] Figure 4 This is a schematic diagram of client mask generation and subnetwork construction in this application.

[0088] Figure 5 This is a schematic diagram of the client subnetwork being trained in this application.

[0089] Figure 6 This is a schematic diagram illustrating the dynamic control of client subnetwork updates in this application. Detailed Implementation

[0090] The embodiments of the present invention will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the present invention. That is, in some embodiments of the present invention, these practical details are not essential. In addition, for the sake of simplicity, some conventional structures and components will be shown in the drawings in a simple schematic manner.

[0091] like Figure 1 and Figure 2 As shown, this application presents a model dynamic pruning and alignment method based on federated learning. The specific steps of the model dynamic pruning and alignment method are as follows:

[0092] Step 1: Data Preparation and System Initialization: The server establishes a public proxy dataset and constructs an initial global dense model. The client loads the initial global dense model. When a client first connects to the federated system, a lightweight resource probe is automatically run. This probe assesses the client's hardware capabilities using three-dimensional resource metrics. Based on these metrics, the client calculates a fusion resource score and maps the overall resource score to a lower limit for the pruning rate. Specifically, the steps include the following:

[0093] Step 1.1: Set up the initial global dense model and basic environment on the server side: The server first builds the initial global dense model. The model structure needs to be determined based on the task type. Simultaneously, the server establishes a public proxy dataset to provide global knowledge alignment during knowledge distillation. Furthermore, the server needs to set global training hyperparameters, including temperature coefficients, bi-objective loss weight coefficients, etc.

[0094] Step 1.2, Detecting Client Hardware Resources: This invention achieves quantitative characterization of client hardware capabilities through hardware resource detection, providing a hardware adaptation basis for determining the subsequent pruning rate lower limit. When the client first connects to the federated system, a lightweight resource probe is automatically run. This probe does not require excessive computing resources. The lightweight resource probe evaluates the client's hardware capabilities through three-dimensional resource indicators, including: computing power... Memory capacity and communication bandwidth Based on three-dimensional resource indicators, the client calculates a comprehensive resource score through weighted fusion. :

[0095]

[0096] in, The normalization function maps the metrics of each dimension to... interval, , , These are weighting coefficients; they need to be adjusted based on the task's sensitivity to resources; among them, computing power... : Detects CPU / NPU FLOPs or core count to quantify the upper limit of computing power; memory capacity. : Detects peak available runtime memory, reflecting parameter storage capacity; communication bandwidth Determine the efficiency of incremental parameter upload by measuring the uplink / downlink rates within 10 seconds;

[0097] Step 1.3: Set upper and lower limits for pruning rate: based on comprehensive resource score. Set a lower limit for the pruning rate; the more abundant the resources, the lower the pruning rate, thus retaining more parameters; the more scarce the resources, the higher the pruning rate, thus retaining fewer parameters. Simultaneously, set an upper limit for the pruning rate. (Can be set according to actual needs) to avoid excessive pruning that could cause the model's expressive power to collapse. The Sigmoid function is used to integrate the resource scores. Mapped to the lower limit of pruning rate :

[0098]

[0099] in, To adjust the coefficient, Controlling the impact of resource score on pruning rate This is the offset coefficient. Adjust the base pruning rate range to ensure .

[0100] Step 2, Adaptive Adjustment of Client Pruning Rate (Low-Frequency Exploration Cycle): This application proposes an adaptive pruning rate adjustment strategy. At the start of each exploration cycle, the client adaptively adjusts the pruning rate for the next cycle based on its historical training performance and hardware resource status, achieving automated and personalized management of the client's subnetwork complexity. The performance change is calculated based on the client's historical training performance, and the overall resource score is used to determine the pruning rate. A resource adjustment factor is constructed, and this factor, along with performance changes, is introduced into the pruning rate update process to obtain the adjusted pruning rate for the next cycle. ;like Figure 3 As shown. Step 2 specifically includes the following steps:

[0101] Step 2.1: Calculate the performance change: Use the performance at the time of the last effective adjustment of the pruning rate as the initial performance baseline. To trigger client subnet updates The training rounds are defined as statistical intervals, and the performance value of the last round within that interval is taken. Right now Calculate the performance change in the final round of training. :

[0102]

[0103] in, As an initial performance benchmark, if the performance evaluation metric used to quantify task completion capability during client subnetwork training is accuracy or F1-score: This indicates that the performance is better than the initial performance benchmark. , This indicates that the performance is inferior to the initial performance baseline. If the indicator is a loss value: This indicates that the performance is better than the initial performance benchmark. , This indicates that the performance is inferior to the initial performance baseline. ;

[0104] Step 2.2, Calculate the resource adjustment factor: To adapt to the resource heterogeneity among different client terminal devices in the federated learning system, this application is based on the comprehensive resource score. Construct resource adjustment factors to match the pruning rate adjustment range with the client's hardware capacity, and define the client. Resource adjustment factors :

[0105]

[0106] When the comprehensive resource score At higher levels, A larger value indicates that the client has strong computing, storage, and communication capabilities, allowing for greater structural adjustments; when the client resource score... At lower levels, A smaller value indicates that the client's resources are limited, and the pruning rate adjustment process should be more stable and conservative.

[0107] Step 2.3, Adjust the pruning rate: Adjust the resource adjustment factor. With performance change A joint pruning rate update process is introduced to obtain the adjusted pruning rate for the next cycle:

[0108]

[0109] in, For the client Current pruning rate, Adjust the step size based on the base. It is the hyperbolic tangent function. , The pruning rate is continuously increased to further compress the size of the client subnetwork and reduce computational and communication overhead. , The pruning rate is continuously reduced to preserve more client subnetwork parameters and restore the expressive power of the client subnetwork; The pruning rate boundary constraint operator is defined as follows: This is used to ensure that the updated pruning rate always falls between the lower and upper limits of the pruning rate.

[0110] Step 3: Generate client mask and construct client subnetwork: The client is based on the adjusted next-cycle pruning rate. Get the local initial mask The client will use the local initial mask. The comprehensive perturbation retention score for each parameter unit and the pruning rate for the next cycle. Uploaded to the server; this provides a basis for subsequently generating the client consensus mask. The server receives the local initial mask uploaded by each client. Maintain score with integrated disturbances Next, a client consensus mask is generated. After receiving the consensus mask distributed by the server, the client completes mask merging, pruning rate verification and updating to obtain the final mask. The client uses the final mask as the filtering criterion to select from the global dense model distributed by the server. Extract the parameter units marked as 1 and construct the client subnetwork. This step is the innovation of this application. This application proposes a method for fusing local mask generation based on parameter unit shielding perturbation with federated perturbation consensus. The client, at the adjusted pruning rate... Under constraints, parameter units that can be preserved or pruned as a whole are taken as the basic objects. The comprehensive importance of a parameter unit for both the client's local task and global knowledge preservation is determined by evaluating the degree of perturbation to the model output distribution and local task loss caused by masking. Subsequently, the server generates a client consensus mask based on the local initial mask and perturbation preservation score uploaded by each client. The client then fuses the local initial mask with the consensus mask to obtain the final mask and constructs a local sub-network based on the final mask. In this application, a parameter unit refers to a set of model parameters in the global model that can be preserved or pruned as a whole, including but not limited to single weight parameters, convolutional kernels, convolutional channels, fully connected neurons, attention heads, normalized layer parameter groups, or continuous parameter blocks. Each parameter unit contains one or more model parameters, and the model parameters within the same parameter unit have the same preservation or pruning state during mask generation, perturbation masking, sub-network construction, and incremental parameter upload. Figure 4 As shown.

[0111] In step 3, the client bases the adjusted pruning rate for the next cycle. Get the local initial mask Specifically, it includes the following steps:

[0112] Step 3.1.1: The client bases the adjusted pruning rate for the next cycle. The initial global dense model issued by the server Perform parameter unit-level importance assessment: The client will use the initial global dense model. Load the data locally and freeze its parameters without performing training updates; then use 100-500 pieces of local task-related data. For the initial global dense model Perform forward reasoning to obtain the complete globally dense model. Output:

[0113] ,

[0114] in, For classification tasks, a complete globally dense model In the sample Output distribution on Represents a globally dense model For input samples The output, Temperature coefficient;

[0115] Step 3.1.2, for any parameter unit The client-side construction only masks this parameter unit. The perturbation model:

[0116] ,

[0117] in, For parameter unit Corresponding shielding indicator matrix, parameter unit The corresponding position is assigned a value of 1, and the remaining positions are assigned a value of 0. This represents element-wise multiplication; using the above method, only the parameter units are... The corresponding set of parameters is temporarily masked, while the remaining parameters remain unchanged, perturbation model In the sample The output distribution on is represented as:

[0118] ,

[0119] The client is based on the complete global density model. Output distribution With perturbation model Output distribution Differences between them, calculation parameter unit Maintain sensitivity to local output :

[0120]

[0121] in, Denotes KL divergence, The larger the value, the more likely it is to be a masking parameter unit. The more pronounced the change in the output distribution of the subsequent model, the more effective the parameter unit is in maintaining a globally dense model. The more important the knowledge output on the client's local data;

[0122] Step 3.1.3, Client-side parameter calculation unit Local task loss perturbation :

[0123]

[0124] in, This represents the local task loss function. The larger the value, the more likely it is to be a masking parameter unit. The more significant the change in the loss of local tasks, the more important this parameter unit is to the performance of client-side local tasks;

[0125] Step 3.1.4: Based on the local task loss perturbation amount and parameter unit Maintain sensitivity to local output Client-side parameter calculation unit The overall perturbation maintenance score :

[0126]

[0127] in, Used for comprehensive characterization parameter unit The higher the score, the greater the contribution of the parameter unit to preserving global output knowledge and local task performance. The stronger the disturbance caused by the behavior, the more it should be preserved;

[0128] Step 3.1.5: Set up a global dense model The total number of intermediate parameter units is Client Based on the adjusted pruning rate for the next cycle Determine the number of parameter units to be retained. The client maintains a score for all parameter units based on the overall disturbance. Sort by high to low, select the top One parameter unit is marked as 1, and the remaining parameter units are marked as 0, thus obtaining the local initial mask. The client will use the local initial mask. The corresponding comprehensive disturbance preservation score And the adjusted pruning rate for the next cycle Uploaded to the server, providing a basis for subsequent generation of client consensus masks.

[0129] In step 3, the server receives the local initial mask uploaded by each client. Maintain score with integrated disturbances Next, a client consensus mask is generated, which includes the following steps:

[0130] Step 3.2.1, this application will define the parameter unit. The overall perturbation maintenance score Introducing a consensus generation process enables the server to generate consensus based on parameter units. Dense model of the whole world after being masked The consensus value is determined by the impact of output distribution and local task loss, for parameter units. Server calculation parameter unit consensus score :

[0131]

[0132] in, Indicates client Parameter unit in local initial mask The state of preservation; This indicates the number of clients participating in the current round of federated training;

[0133] Step 3.2.2: The server, based on the client... Next cycle pruning rate Determine the number of consensus parameter units that the server should retain. The server scores according to consensus. Sort by high to low, select the top One parameter unit is marked as 1, and the remaining parameter units are marked as 0, thus obtaining the client... consensus mask The server will use the client consensus mask. Distribute to the corresponding clients for use in the subsequent final mask generation.

[0134] In step 3, after receiving the consensus mask distributed by the server, the client completes mask merging, pruning rate verification and updating to ensure that the final mask meets the hardware requirements. At the same time, it prioritizes retaining key parameter units of the client's local task to obtain the final mask. The specific steps include the following:

[0135] Step 3.3.1: Execute the client consensus mask With local initial mask The union operation yields the candidate union mask. ,statistics The number of 1s The current pruning rate is calculated based on the number of parameter units retained in the candidate union mask. It is necessary to verify whether the current pruning rate meets the set lower limit of the pruning rate:

[0136] like This indicates that the number of parameter units retained by the candidate union mask does not exceed the capacity of the client hardware, so there is no need to adjust the parameter units, and the candidate union mask can be directly used as the final mask. The current pruning rate will be used as the actual pruning rate for the client. ;

[0137] like This indicates that the candidate union mask retains too many parameter units, and it is necessary to remove some parameter units marked as 1 to improve the pruning rate. Calculate the number of units to be removed. For parameter units that need to be removed, priority should be given to selecting them from the parameter units newly added in the consensus mask. The client should then use its own consensus scoring method within these parameter units. Sort by score from lowest to highest, and select the lowest score. Each parameter unit, in The flag in the image is changed from 1 to 0 to obtain the final mask. ; Obtain the final mask Then, calculate the final mask. The final number of 1s Calculate the updated actual pruning rate ;

[0138] Step 3.3.2, the client uses the final mask. The global dense model is used as the selection criterion. Extract the parameter units marked as 1 and construct the client subnetwork. .

[0139] Step 4: The client performs multiple rounds of local training on the client sub-network constructed in Step 3 and evaluates the performance of the trained sub-network. After completing local training, the client only calculates the parameter increment for the parameter units marked as 1 in the final mask and uploads the data packets. The server receives the data packets uploaded by each client and aggregates the global update of the parameter units to obtain the updated global model. The core of Step 4 is to achieve a dense model of local data fitting and global knowledge alignment by constructing a dual-objective loss function of local task loss and global knowledge alignment loss under the generated fixed client sub-network structure. Collaborative optimization for knowledge preservation. Meanwhile, the client only has access to the final mask. The parameter units marked as 1 are trained and incrementally uploaded. The server only performs performance-weighted aggregation on parameter units updated by at least one client. For example... Figure 5 As shown. Step 4 specifically includes the following steps:

[0140] Step 4.1: The client constructs the resulting client subnetwork. Multiple rounds of local training were conducted, and during the training process, the final mask was determined. Keeping the parameters constant, the client only updates the parameter units marked as 1 by the final mask. Parameter units not retained do not participate in forward propagation, backward propagation, or parameter updates. The client uses a dual-objective loss function that balances local data fitting and global knowledge alignment for training, as shown in the formula:

[0141]

[0142] in, These are the weighting coefficients. , For local tasks, the loss is used; for classification tasks, cross-entropy loss is used. The regression task uses mean squared error loss. , Loss for global knowledge alignment:

[0143]

[0144] in, Logits output for the client subnet. for The output Logits, The temperature coefficient and To control the smoothness of the distribution;

[0145] Step 4.2: After the client completes multiple rounds of local training, the performance of the trained client subnetwork is evaluated using a local validation set independent of the training set to obtain the client performance score. Client performance score The aggregation weights are used by the server to calculate the incremental client parameters during subsequent global aggregation. To ensure the stability of the aggregation direction, in this invention... The preferred approach is to use positive performance scores, i.e. ,and The larger the value, the better the performance of the client subnet on the local validation set, and the higher the weight of its uploaded parameter increments during server aggregation;

[0146] Step 4.3: The client calculates the parameter unit increment and uploads the data packet: After completing local training, the client only calculates the final mask. The parameter increment is calculated for the parameter cell marked as 1, for any retained parameter cell. Its parameter unit increment is expressed as:

[0147]

[0148] in, For the client Subnetwork parameters before training These are the parameters of the trained sub-network. This indicates that the parameter unit is preserved in the final mask and participates in local training; client The overall parameter increment of the upload is denoted as The client uploads data packets to the server. These data packets include... , Used for server updates of the globally dense model The corresponding parameter unit; Used by the server to identify the actual training location of the client and upload incremental parameter units; The performance score of the client subnet on the local validation set is used by the server for performance-weighted aggregation.

[0149] Step 4.4: The server performs performance-weighted aggregation to update the global model: After receiving data packets uploaded by each client, the server updates the global model based on the final mask of each client. To determine which parameter units were updated, for a globally dense model... Any parameter unit The server first determines the set of clients that will participate in updating the parameter unit. The global update value of this parameter unit:

[0150]

[0151] in, Represents the parameter unit obtained from server aggregation Global update volume Indicates client Uploaded parameter unit The incremental change; the server applies the aggregated global update to the global dense model. Corresponding parameter unit: After aggregating all parameter units, the server obtains the updated global model. .

[0152] Step 5, the client uses The training round serves as a testing cycle, with the client maintaining the current final mask. Continue training using the current client subnetwork, unchanged; when completed... After each training round, the performance changes during the detection period are used to determine whether it is necessary to regenerate the mask and reconstruct the subnetwork. For example... Figure 6 As shown. Step 5 specifically includes the following steps:

[0153] Step 5.1: Set the detection cycle and performance index collection: The client follows the preset... One round of training is one detection cycle (n is a configurable parameter that can be adjusted according to task complexity and data volume). In each detection cycle, the client trains the current client sub-network according to the training process in step 4, and records the performance score on the local validation set after each round of training.

[0154] Step 5.2, End-of-cycle performance status determination: When completed... After each training round, a performance check is automatically triggered. Based on the performance score within that period, it is determined whether the client sub-network structure needs to be adjusted. Specifically:

[0155] Performance stagnation determination: calculation The performance improvement rate per round of training is calculated using the following formula: ;like All wheel boost rates are below the threshold for determining performance stagnation. This is determined to be a performance stagnation;

[0156] Performance degradation determination: If If a performance shows a decreasing trend for two or more consecutive rounds, it is considered a performance decline.

[0157] If either performance stagnation or performance degradation is met, the current client subnetwork structure is determined to need updating; if neither condition is met, the current client subnetwork structure is determined not to need updating.

[0158] Step 5.3: Based on the performance status assessment results in Step 5.2, update the client subnetwork or continue training.

[0159] If the performance status indicates that the client sub-network structure needs adjustment, repeat steps 2-3 to update the final mask. After the mask update is complete, reconstruct the client sub-network based on the new mask and continue to the next step. Rotational training;

[0160] If the performance status indicates that no adjustment to the client subnet structure is needed, the pruning rate adjustment and mask generation process will not be triggered. In this case, the server will no longer distribute the complete global model. Instead, the server will use the final mask recorded by the client. From the latest global model The latest subnet is extracted and distributed to the client. After receiving the client's subnet, the client continues to execute the next one. The training cycle continues until the next testing cycle is completed, at which point the performance evaluation process is repeated.

[0161] Step 6: Repeat steps 2-5 to determine the final sub-network model that is adapted to local hardware resources and meets the task performance requirements.

[0162] Example

[0163] This application uses the task of equipment fault detection in a heterogeneous environment at the industrial edge as an implementation case to illustrate the content of this application.

[0164] This embodiment addresses the fault detection needs of multiple types of edge devices in a smart manufacturing plant, involving three heterogeneous edge clients. The input data consists of device operating status data including sensor time-series data and device image data. The goal is to predict whether the device has malfunctioned and the specific type of malfunction (multi-classification task). The server uses ResNet18, a mainstream lightweight deep learning model for industrial fault detection, as the global model, while the client sub-network is generated by pruning based on this global model.

[0165] The configurations and data of these three heterogeneous clients differ significantly:

[0166] Client 1 is a low-configuration industrial gateway with a hardware configuration of ARM Cortex-A53 (4 cores) CPU, 300GFLOPs computing power, 8GB of available memory, and an uplink speed of 50Mbps per 10 seconds. It processes time-series data from vibration sensors of motor equipment (including 10 types of features and a sampling frequency of 10Hz), with a total of 5,000 data entries. The fault types are mainly bearing wear and rotor imbalance (accounting for 70%), and there is no image data.

[0167] Client 2 is a mid-configuration edge server (core tracking object), with hardware including an Intel Xeon D-2141I (8-core) CPU, 800 GFLOPs computing power, 32GB of available memory, and an uplink speed of 200Mbps per 10 seconds. It processes multi-dimensional data from water pumps (including pressure and flow sensor data and images), with a total of 12,000 data entries. The main fault types are impeller blockage and seal leakage (accounting for 65%), and it includes 2,000 images with a resolution of 256×256.

[0168] Client 3 is a mid-to-high-end smart camera with hardware including an NVIDIA Jetson Xavier NX CPU, 1000GFLOPs computing power, 64GB of available memory, and an uplink speed of 300Mbps per 10 seconds. It processes valve images and on / off status data, with a total of 8000 data entries. The main fault types are sealing leakage and valve core jamming (accounting for 60%), including 4000 512×512 high-definition images.

[0169] Step 1: Server-side global model and basic environment setup. The server builds the initial global dense model. In this embodiment, ResNet18 is used as the initial global model. To facilitate parameter unit-level pruning, the server divides the global model into several parameter units based on structures such as convolutional channels, residual blocks, and fully connected neurons. For ease of explanation, this embodiment assumes that the global model contains a total of Each parameter unit.

[0170] The server establishes a public proxy dataset for subsequent global knowledge alignment. This dataset contains 8,000 standard fault data points for industrial equipment, covering three types of equipment: motors, pumps, and valves, with 4,000 fault samples and 4,000 non-fault samples. The server sets global training hyperparameters, including temperature coefficients and weight coefficients for the bi-objective loss.

[0171] Step 2: Client hardware resource detection, taking client 2 as an example. Client 2 runs a lightweight resource probe to evaluate hardware capabilities from three core dimensions. The results are as follows:

[0172] computing power CPU FLOPs = 800 GFLOPs, after normalization ;

[0173] Memory capacity Available RAM = 32GB, after normalization ;

[0174] Communication bandwidth Average uplink speed over 10 seconds = 200Mbps, after normalization .

[0175] Because fault detection tasks are more sensitive to computing power and memory capacity, weighting coefficients are set. Calculated according to the comprehensive resource score formula:

[0176]

[0177] Step 3: Setting the upper and lower limits of pruning rate. Set the upper limit of the pruning rate for client 2 to [value missing]. .use The mapping function calculates the lower bound of the pruning rate for client 2, and takes... Therefore, the lower limit of the pruning rate is:

[0178]

[0179] Therefore, the actual pruning rate used by client 2 should satisfy the following: .

[0180] Step 4: Initial model performance test. Client 2 loads the initial globally dense model issued by the server. Using a local validation set (2000 water pump equipment data points, including 800 faulty samples and 1200 non-faulty samples, with 500 images), a complete forward inference was performed to obtain the initial performance benchmark: fault detection accuracy. In this embodiment, the performance values ​​are expressed as percentages.

[0181] Step 5: Client-side pruning rate adaptive adjustment. Initially, client 2 adjusts the pruning rate based on the initial pruning rate. Constructing sub-networks, i.e. Then, high-frequency training is performed, and the final round performance when the client subnetwork is updated is... Client 2 calculates performance change ,because This indicates that the current subnetwork performance of client 2 is better than the historical performance benchmark, and the pruning rate can be appropriately increased to further compress the subnetwork size.

[0182] Client 2 calculates the resource adjustment factor based on its own comprehensive resource score. Let the basic adjustment step size be... The adjusted pruning rate for client 2 is:

[0183]

[0184] Step 6: Initial mask generation on the client side. Client 2 generates the mask based on the pruning rate determined in Step 5. Calculate the number of parameter units that should be retained. Client 2 will use the globally dense model Load the data locally and freeze the parameters, then use 300 local water pump data points to create a small batch sample set. It covers core fault types such as impeller blockage and seal leakage, and includes 80 image data.

[0185] For the sample The complete global model output distribution is as follows For any parameter unit Client 2 constructs a perturbation model that only masks the parameter unit. Perturbation model The output distribution is .

[0186] With parameter unit For example, after client 2 performs masking perturbation calculations on 300 local samples, the complete global dense model... Output distribution and perturbation model The cumulative KL divergence between the output distributions is 84, the cumulative local task loss perturbation is 66, and the temperature coefficient is assumed to be... ,but:

[0187]

[0188] Therefore, parameter unit The overall disturbance retention score is:

[0189]

[0190] Client 2 calculates the integrated disturbance preservation score for each of the 100 parameter units in the manner described above. And sort them according to the score from highest to lowest. Since the number of parameter units that client 2 should retain is... Therefore, Client 2 selects the top 54 parameter units in the comprehensive perturbation preservation score and marks them as 1, while marking the remaining 46 parameter units as 0, thus obtaining the local initial mask. Subsequently, client 2 will use its local initial mask. Comprehensive disturbance maintenance score and pruning rate Uploaded to the server.

[0191] Step 7: The server generates a consensus mask for each client and distributes it to the clients. The server receives the initial local masks uploaded by the three clients. Maintain score with integrated disturbances Then, a client consensus mask is generated. This invention introduces the comprehensive perturbation-preserving score of the parameter unit into the consensus generation process, enabling the server to determine the global dense model based on the parameter unit after it is masked. The consensus value is determined by the impact of output distribution and local task losses.

[0192] For parameter units Clients 1, 2, and 3 all marked it as retained, and their overall disturbance retention scores were as follows:

[0193]

[0194] Then the parameter unit The consensus score is:

[0195]

[0196]

[0197] The server calculates the consensus score for all 100 parameter units in the manner described above. And based on the pruning rate of client 2 Determine the number of consensus parameter units that client 2 should retain: The server scores according to consensus. Sort the parameters from highest to lowest, mark the top 54 as 1, and mark the remaining 46 as 0 to obtain the consensus mask for client 2. Ultimately, the server will use client 2's consensus mask. Distribute to client 2 for subsequent fusion with the local initial mask to generate the final mask.

[0198] Step 8: Client-side final mask generation and pruning rate verification. Client 2 receives the consensus mask distributed by the server. Then, with the local initial mask Perform a union operation to obtain a union mask. The number of parameter units marked as 1 in this candidate union mask is... The current pruning rate is: ,because This indicates that the number of parameter units retained in the candidate union mask does not exceed the hardware capacity of client 2, and there is no need to reduce the number of parameter units. Therefore, client 2 directly uses the union mask as the final mask. The current pruning rate will be used as the actual pruning rate for client 2. .

[0199] Step 9: The client constructs the sub-network based on the final mask. Client 2 constructs the sub-network based on the final mask. Global dense model issued from the server Extract the parameter units marked as 1 and construct the client subnetwork.

[0200] Step 10: The client sub-network is trained using a dual-objective loss function and its performance is evaluated. Client 2 uses 12,000 local water pump data points to train the current sub-network. Multiple rounds of local training were conducted, including 2000 image data points and 4800 fault samples. Client 2 employed a bi-objective loss function consisting of local task loss and global knowledge alignment loss. In this embodiment For classification tasks, the local task loss uses cross-entropy loss: .

[0201] For a "pumper blockage" fault sample, the true category is class 2, and the subnetwork predicts the probability of class 2 as follows: Then the local cross-entropy loss of this sample is: The global knowledge alignment loss uses KL divergence. In this embodiment The global knowledge alignment loss corresponding to this sample is The total loss is:

[0202]

[0203] Client 2 trains the parameter-preserving unit based on the aforementioned bi-objective loss function, but it is not ultimately masked. The reserved parameter units do not participate in forward propagation, backward propagation, and parameter updates.

[0204] After training, Client 2 evaluates the trained subnetwork using a local validation set. The local validation set contains 2000 data points from water pump devices. The evaluation yields the performance score of Client 2's subnetwork. ,in, The positive performance score indicates that the fault detection accuracy of the client 2 subnet on the local verification set is 88.2%.

[0205] Step 11: The client calculates the parameter unit increment and uploads the data packet. After completing local training, Client 2 only performs training on the final mask. The parameter increment is calculated for the parameter unit marked as 1. (This is done using the parameter unit.) For example, the parameters before training are: The parameters after training are ,but The overall parameter increment set uploaded by client 2 is Then client 2 will send the data packet Uploaded to the server.

[0206] Step 12: The server performs performance-weighted aggregation to update the global model. After receiving data packets uploaded by the three clients, the server determines which parameter units have been updated based on the final mask of each client. For any parameter unit... The server determines the set of clients that will participate in updating the parameter unit. . In terms of parameter units For example, both client 1 and client 2 updated this parameter unit, but client 3 did not update it. The performance scores of Client 1 and Client 2 on the local validation set are respectively The parameter unit increments uploaded by client 1 and client 2 are respectively Then the server will handle the parameter unit. The global update size is:

[0207]

[0208] The server applies this global update to the corresponding parameter unit of the global model: If a parameter unit is not updated by any client, it remains unchanged. After aggregating all parameter units, the server obtains the updated global model. .

[0209] Step 13: Dynamic adjustment of client sub-network updates. Set the detection cycle. The training run consisted of multiple rounds, with performance scores recorded in real time for each round: Round 1: 87.9, Round 2: 88.1, Round 3: 88.0, Round 4: 88.2. The performance improvement rate for each round was calculated.

[0210] Second round improvement rate: ;

[0211] Third round improvement rate: ;

[0212] 4th round improvement rate: ;

[0213] Set performance stagnation threshold The current performance improvement rates in rounds 2 and 4 are both above 0.001, and there has been no consecutive performance degradation in two rounds, indicating that no adjustment to the sub-network structure is needed. Since no adjustment to the sub-network structure is required, the server will switch to the latest global model. The corresponding sub-network is extracted and distributed to client 2. The client then continues to perform the next 4 rounds of training based on the sub-network until the client sub-network structure needs to be updated.

[0214] Step 14: Final Sub-Network Structure Determined for the Client. Through multiple rounds of low-frequency exploration and high-frequency utilization iterative training, the client determines the final sub-network structure that adapts to local hardware resources and meets the task performance requirements.

[0215] This application enables each client to autonomously adjust the pruning degree of its local model based on its own resource status and training performance feedback within the constraints of terminal hardware resources, achieving adaptive optimization of model complexity. When building a local model, each terminal generates a pruning mask by integrating global consensus and terminal-specific requirements, constructing a lightweight sub-network that combines global generalization capabilities with local task adaptability. Furthermore, a global knowledge alignment constraint mechanism is introduced during the local training phase to unify local model optimization and federated collaborative consistency. This application effectively avoids resource and performance imbalances caused by fixed pruning or uniform pruning rate strategies, while simultaneously considering global generalization and local adaptability, significantly improving the practicality and reliability of federated learning systems in heterogeneous terminal scenarios.

[0216] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for dynamic pruning and alignment of models based on federated learning, characterized in that: The model dynamic pruning and alignment method specifically includes the following steps: Step 1: Data Preparation and System Initialization: The server establishes a public proxy dataset and constructs an initial global dense model. The client loads the initial global dense model. When a client first connects to the federated system, a lightweight resource probe is automatically run. This probe assesses the client's hardware capabilities using three-dimensional resource metrics. Based on these metrics, the client calculates a fusion resource score and maps the overall resource score to a lower limit for the pruning rate. ; Step 2, Adaptive Adjustment of Client-Side Pruning Rate: Calculate performance changes based on historical client training performance, and adjust based on comprehensive resource scores. A resource adjustment factor is constructed, and this factor, along with performance changes, is introduced into the pruning rate update process to obtain the adjusted pruning rate for the next cycle. ; Step 3: Generate client mask and construct client subnetwork: The client is based on the adjusted next-cycle pruning rate. Get the local initial mask The client will use the local initial mask. The comprehensive perturbation retention score for each parameter unit and the pruning rate for the next cycle. Uploaded to the server; the server receives the local initial mask uploaded by each client. Maintain score with integrated disturbances Next, a client consensus mask is generated. After receiving the consensus mask distributed by the server, the client completes mask merging, pruning rate verification and updating to obtain the final mask. The client then uses the final mask as the filtering criterion to select from the global dense model distributed by the server. Extract the parameter units marked as 1 and construct the client subnetwork. ; Step 4: The client performs multiple rounds of local training on the client subnetwork constructed in Step 3 and evaluates the performance of the trained subnetwork. After the client completes local training, it calculates the parameter increment only for the parameter units marked as 1 in the final mask and uploads the data packet. The server receives the data packets uploaded by each client and aggregates the global update of the parameter units to obtain the updated global model. Step 5, the client uses The training round serves as a testing cycle, with the client maintaining the current final mask. Continue training using the current client subnetwork, unchanged; when completed... After each training round, the performance changes during the detection period are used to determine whether it is necessary to regenerate the mask and reconstruct the subnetwork. Step 6: Repeat steps 2-5 to determine the final sub-network model that is adapted to local hardware resources and meets the task performance requirements.

2. The method for dynamic pruning and alignment of a model based on federated learning according to claim 1, characterized in that: Step 1 specifically includes the following steps: Step 1.1: Set up the initial global dense model and basic environment on the server side: The server first builds the initial global dense model. Meanwhile, the server establishes a public agent dataset and sets global training hyperparameters, including temperature coefficient and dual-objective loss weight coefficient. Step 1.2, Probe Client Hardware Resources: When a client first connects to the federated system, a lightweight resource probe is automatically run. This lightweight resource probe evaluates the client's hardware capabilities using three-dimensional resource metrics, including: computing power. Memory capacity and communication bandwidth Based on three-dimensional resource indicators, the client calculates a comprehensive resource score through weighted fusion. : in, The normalization function maps the metrics of each dimension to... interval, , , These are the weighting coefficients; Step 1.3: Set upper and lower limits for pruning rate: based on comprehensive resource score. Set a lower limit for the pruning rate and use the Sigmoid function to calculate the overall resource score. Mapped to the lower limit of pruning rate : in, To adjust the coefficient, As the offset coefficient, ensure At the same time, set an upper limit on the pruning rate. .

3. The method for dynamic pruning and alignment of a model based on federated learning according to claim 2, characterized in that: Step 2 specifically includes the following steps: Step 2.1: Calculate the performance change: Use the performance at the time of the last effective adjustment of the pruning rate as the initial performance baseline. To trigger client subnet updates The training rounds are defined as statistical intervals, and the performance value of the last round within that interval is taken. Calculate the change in performance : in, As an initial performance benchmark, if the performance evaluation metric used to quantify task completion capability during client subnetwork training is accuracy or F1-score: This indicates that the performance is better than the initial performance benchmark. , This indicates that the performance is inferior to the initial performance baseline. If the indicator is a loss value: This indicates that the performance is better than the initial performance benchmark. , This indicates that the performance is inferior to the initial performance baseline. ; Step 2.2: Calculate the resource adjustment factor based on the comprehensive resource score. Construct resource adjustment factors and define clients. Resource adjustment factors : ； Step 2.3, Adjust the pruning rate: Adjust the resource adjustment factor. With performance change A joint pruning rate update process is introduced to obtain the adjusted pruning rate for the next cycle: in, For the client Current pruning rate, Adjust the step size based on the base. It is the hyperbolic tangent function. , The pruning rate has been increasing continuously. , The pruning rate decreased continuously. This is the boundary constraint operator for the pruning rate.

4. The method for dynamic pruning and alignment of a model based on federated learning according to claim 1, characterized in that: In step 3, the client bases the adjusted pruning rate for the next cycle. Get the local initial mask Specifically, it includes the following steps: Step 3.1.1: The client bases the adjusted pruning rate for the next cycle. The initial global dense model issued by the server Perform parameter unit-level importance assessment: The client will use the initial global dense model. Load it locally and freeze its parameters without training or updating; then apply the initial globally dense model. Perform forward reasoning to obtain the complete global dense model. Output: ， in, For classification tasks, a complete globally dense model In the sample Output distribution on Represents a globally dense model For input samples The output, Temperature coefficient; Step 3.1.2, for any parameter unit The client-side construction only masks this parameter unit. The perturbation model: ， in, For parameter unit Corresponding shielding indicator matrix, parameter unit The corresponding position is assigned a value of 1, and the remaining positions are assigned a value of 0. Represents element-wise multiplication; perturbation model In the sample The output distribution on is represented as: ， The client is based on the complete global density model. Output distribution With perturbation model Output distribution Differences between them, calculation parameter unit Maintain sensitivity of local output : in, Indicates KL divergence; Step 3.1.3, Client-side parameter calculation unit Local task loss perturbation : in, This represents the local task loss function; Step 3.1.4: Based on the local task loss perturbation amount and parameter unit Maintain sensitivity of local output Client-side parameter calculation unit The overall perturbation maintenance score : in, Units for comprehensive characterization parameters The higher the score, the greater the contribution of the parameter unit to preserving global output knowledge and local task performance. The stronger the disturbance caused by the behavior, the more it should be preserved; Step 3.1.5: Set up a global dense model The total number of intermediate parameter units is Client Based on the adjusted pruning rate for the next cycle Determine the number of parameter units to retain. The client maintains a score for all parameter units based on the overall disturbance. Sort by high to low, select the top One parameter unit is marked as 1, and the remaining parameter units are marked as 0, thus obtaining the local initial mask. The client will use the local initial mask. The corresponding comprehensive disturbance preservation score And the adjusted pruning rate for the next cycle Uploaded to the server.

5. The method for dynamic pruning and alignment of a model based on federated learning according to claim 1, characterized in that: In step 3, the server receives the local initial mask uploaded by each client. Maintain score with integrated disturbances Next, a client consensus mask is generated, which includes the following steps: Step 3.2.1: Set the parameter unit The overall perturbation maintenance score Introducing a consensus generation process, for parameter units Server calculation parameter unit consensus score : in, Indicates the client Parameter unit in local initial mask The state of preservation; This indicates the number of clients participating in the current round of federated training; Step 3.2.2: The server, based on the client... Next cycle pruning rate Determine the number of consensus parameter units retained by the server. The server scores according to consensus. Sort by high to low, select the top One parameter unit is marked as 1, and the remaining parameter units are marked as 0, thus obtaining the client... consensus mask The server will use the client consensus mask. Distribute to the corresponding clients for use in the subsequent final mask generation.

6. The method for dynamic pruning and alignment of a model based on federated learning according to claim 5, characterized in that: In step 3, after receiving the consensus mask distributed by the server, the client completes mask merging, pruning rate verification and updating to obtain the final mask. This process includes the following steps: Step 3.3.1: Execute the client consensus mask With local initial mask The union operation yields the candidate union mask. ,statistics The number of 1s The current pruning rate is calculated based on the number of parameter units retained in the candidate union mask. Verify whether the current pruning rate meets the set lower limit for pruning rate: like This indicates that the number of parameter units retained by the candidate union mask does not exceed the capacity of the client hardware, so there is no need to adjust the parameter units, and the candidate union mask can be directly used as the final mask. The current pruning rate will be used as the actual pruning rate for the client. ; like This indicates that the candidate union mask retains too many parameter units, and it is necessary to remove some parameter units marked as 1 to improve the pruning rate. Calculate the number of units to be removed. For parameter units that need to be removed, priority should be given to selecting them from the parameter units newly added in the consensus mask. The client should then score these parameter units according to consensus criteria. Sort by score from lowest to highest, and select the lowest score. Each parameter unit, in The flag in the image is changed from 1 to 0 to obtain the final mask. ; Obtain the final mask Then, calculate the final mask. The final number of 1s Calculate the updated actual pruning rate ; Step 3.3.2, the client uses the final mask. The global dense model is used as the selection criterion. Extract the parameter units marked as 1 and construct the client subnetwork. .

7. The method for dynamic pruning and alignment of a model based on federated learning according to claim 6, characterized in that: Step 4 specifically includes the following steps: Step 4.1: The client constructs the resulting client subnetwork. Multiple rounds of local training were conducted, and during the training process, the final mask was determined. Keeping the parameters fixed, the client only updates the parameter units marked as 1 by the final mask. Parameter units not retained do not participate in forward propagation, backward propagation, or parameter updates. The client uses a dual-objective loss function that balances local data fitting and global knowledge alignment for training, as shown in the formula: in, These are the weighting coefficients. , For local tasks, the loss is used; for classification tasks, cross-entropy loss is used. The regression task uses mean squared error loss. , Loss for global knowledge alignment: in, Logits output for the client subnet. for The output Logits, The temperature coefficient and To control the smoothness of the distribution; Step 4.2: After the client completes multiple rounds of local training, the performance of the trained client subnetwork is evaluated using a local validation set independent of the training set to obtain the client performance score. ; Step 4.3: After the client completes local training, only the final mask is processed. The parameter increment is calculated for the parameter cell marked as 1, for any retained parameter cell. The increment of its parameter unit is expressed as: in, For the client Subnetwork parameters before training These are the parameters of the trained sub-network. This indicates that the parameter unit is preserved in the final mask and participates in local training; client The overall parameter increment of the upload is denoted as The client uploads data packets to the server. These data packets include... ; Step 4.4: After receiving the data packets uploaded by each client, the server determines the final mask for each client. Determine which parameter units were updated for a globally dense model. Any parameter unit The server first determines the set of clients that will participate in updating the parameter unit. The global update value of this parameter unit: in, Represents the parameter unit obtained from server aggregation Global update volume Indicates client Uploaded parameter unit The incremental change; the server applies the aggregated global update to the global dense model. Corresponding parameter unit: After aggregating all parameter units, the server obtains the updated global model. .

8. The method for dynamic pruning and alignment of a model based on federated learning according to claim 1, characterized in that: Step 5 specifically includes the following steps: Step 5.1: Set the detection cycle and performance index collection: The client follows the preset... Each training round is a testing cycle. In each testing cycle, the client trains the current client subnetwork according to the training process in step 4, and records the performance score on the local validation set after each training round. Step 5.2, End-of-cycle performance status determination: When completed... After each training round, a performance check is automatically triggered. Based on the performance score within that period, it is determined whether the client sub-network structure needs to be adjusted. Specifically: Performance Stagnation Detection: Calculation The performance improvement rate per round of training is calculated using the following formula: ;like All wheel boost rates are below the threshold for performance stagnation. This is determined to be a performance stagnation; Performance degradation determination: If If a performance shows a decreasing trend for two or more consecutive rounds, it is considered a performance degradation. If either performance stagnation or performance degradation is met, the current client subnetwork structure is determined to need updating; if neither condition is met, the current client subnetwork structure is determined not to need updating. Step 5.3: Based on the performance status assessment results in Step 5.2, update the client subnetwork or continue training. If the performance status indicates that the client sub-network structure needs adjustment, repeat steps 2-3 to update the final mask. After the mask update is complete, reconstruct the client sub-network based on the new mask and continue to the next step. Rotational training; If the performance status indicates that no adjustment to the client subnet structure is needed, the pruning rate adjustment and mask generation process will not be triggered. In this case, the server will no longer distribute the complete global model, and the server will use the final mask recorded by the client. From the latest global model The latest subnet is extracted and distributed to the client. After receiving the client's subnet, the client continues to execute the next one. The training cycle continues until the next testing cycle is completed, at which point the performance evaluation process is repeated.

Images