Class-level federal forgetting method and system based on double-layer expert mixing and TF-IDF discriminant pruning

Abstract

The invention provides a class-level federal forgetting method and system based on double-layer expert mixing and TF-IDF discriminant pruning. The method comprises the following steps: issuing an initial or updated global feature extractor to each client participating in federal learning training; receiving a local feature extractor uploaded by each client; and aggregating all the received local feature extractors by using an outer layer expert hybrid model, specifically comprising the following steps: when a forgetting request containing a target category set from a client is received, calculating a channel-level discriminative score of each local feature extractor in each target category, carrying out pruning processing on channels with channel-level discriminative scores exceeding a set threshold value in the local feature extractors, and carrying out weighted aggregation on all the local feature extractors after pruning processing to obtain an updated global feature extractor; and when the forgetting request is not received, performing weighted aggregation on all the received local feature extractors according to a fixed gating strategy.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of federated unlearning, and particularly relates to a class-level federated unlearning method and system based on double-layer expert mixing and TF-IDF discriminative pruning. BACKGROUND

[0002] With the wide application of artificial intelligence, federated learning (FL) has attracted extensive attention due to its ability to realize multi-terminal joint modeling without sharing user original data. However, with the growing demand for privacy protection and distributed intelligent processing, and with the promulgation of privacy protection laws and regulations such as the General Data Protection Regulation (GDPR), federated learning is widely used in personalized video recommendation, image retrieval and voice interaction scenes, and how to realize the "right to be forgotten" has become an urgent problem to be solved. Among them, the traditional federated learning constructs a global model by means of local training of the client and aggregation of the server parameters without sharing the original data. However, under the constraint of privacy laws, how to realize the "forgettable" of the model to the specific user data has become a current technical difficulty. Therefore, federated unlearning technology emerges as the times require, and federated unlearning (FU) as a key technology for effectively stripping the influence of user sensitive information from the joint model has gradually attracted the attention of researchers, and its core goal is to selectively remove the influence of target data on the model without training the full model from scratch.

[0003] In a typical federated learning framework, the client trains a local model and uploads model updates, and the server aggregates to form a global model. In the FU task, after the client initiates a forgetting request, the specified data needs to be removed from the global model to remove the influence of the model behavior. The existing methods mainly include full retraining, differential privacy interference, model pruning and shielding, and local optimization, to realize the forgetting of specific data or categories. However, in the real federated scene, the client may hold different types of data (such as images, texts, and voices), and the neural network structures they use are different, which makes it difficult for the above methods to adapt to heterogeneous model architectures, and the sensitivity of different modalities to the forgetting operation is inconsistent, which further aggravates the problems of model performance degradation and training overhead increase.

[0004] To solve the problem of federated forgetting under heterogeneous conditions, the federated forgetting technology initially appeared with the "full retraining" idea, that is, after the client initiates a deletion request, the model is retrained directly on the remaining data to ensure that the contribution of the deleted sample completely disappears, but this way has high calculation and communication cost, which is difficult to apply to large-scale scenarios. To reduce the cost, researchers introduced "differential privacy noise injection", which superimposes noise in model updates to weaken the influence of a single data, but this method will amplify the precision loss in the strong non-IID scene. Subsequently, the "local rollback" or "partial retraining" strategy only revises the historical gradient and local weight involved in the deletion request, significantly reducing the global calculation amount while ensuring deletion effect.

[0005] The existing federated forgetting technology proposes a "full retraining" approach: after removing the target data, the model is completely retrained from scratch to achieve the most thorough forgetting effect. This solution is theoretically the most complete, but it is difficult to be directly applied to large-scale, real-time scenarios due to the high training cost. Another approach is to use differential privacy or noise disturbance to eliminate the contribution of specific data to the model. By injecting random noise into the update, forgetting is approximately achieved. However, under non-IID data distribution, such noise often amplifies performance loss, leading to a significant drop in model accuracy. To balance efficiency and accuracy, researchers have proposed a local retraining strategy that only adjusts the affected historical gradient or local parameters, rather than retraining the entire model. Some subsequent work also combines longitudinal data division, asynchronous training, and client cluster revocation mechanisms to further expand the method's applicability. In addition, pruning techniques assess the importance of channels or neurons and then shield or delete unnecessary parts to achieve fine-grained forgetting. Some solutions also combine this technology with dynamic client selection or knowledge distillation to improve efficiency and environmental adaptability. Although the above methods have their advantages, they still have problems such as high resource overhead, uncontrollable performance, and lack of verification mechanisms in non-independent and identically distributed (Non-IID) data and model structure heterogeneous environments. However, existing heterogeneous federated learning techniques face additional challenges when migrating to federated forgetting tasks. For example, model updates after dynamic pruning are difficult to maintain consistency among clients, collaboration conflicts frequently occur during data deletion, and communication load is significantly increased.

[0006] Therefore, there is an urgent need to design new federated forgetting methods and frameworks to better adapt to the problems existing in the above-mentioned existing federated forgetting technology, to improve overall efficiency and usability. SUMMARY

[0007] In view of this, the embodiments of the present application provide a class-level federated forgetting method and system based on double-layer expert mixing and TF-IDF discriminative pruning to eliminate or improve one or more defects in the prior art.

[0008] An aspect of the present application provides a class-level federated forgetting method based on double-layer expert mixing and TF-IDF discriminant pruning. In each communication round, the method comprises: distributing an initial or updated global feature extractor to each client participating in federated learning training; wherein each client aggregates the global feature extractor and its locally stored local feature extractor using an inner-layer expert mixing model; receiving the local feature extractors uploaded by each client; aggregating all received local feature extractors using an outer-layer expert mixing model, specifically including: when receiving a forgetting request from a client containing a target class set, calculating the channel-level TF-IDF distribution of each local feature extractor on each target class contained in the target class set, calculating the channel-level discriminant score of each local feature extractor on each target class based on the channel-level TF-IDF distribution, pruning the channels in each local feature extractor whose channel-level discriminant score exceeds a set threshold, determining the weighting coefficients of each local feature extractor based on the channel-level discriminant score, and weighting and aggregating all pruned local feature extractors according to the weighting coefficients; aligning the weighting aggregation result to obtain an updated global feature extractor; when no forgetting request is received, weighting and aggregating all received local feature extractors according to a fixed gating strategy.

[0009] In some embodiments of the present application, the inner-layer expert mixing model comprises a global feature extractor, a local feature extractor, a first gating network, and a prediction head module; each client aggregates the global feature extractor and its locally stored local feature extractor using the inner-layer expert mixing model, which comprises: in the first communication round, the first gating network fuses the global feature extractor and the local feature extractor according to a preset proportion; from the second round, the first gating network determines the channel-level discriminant score according to the channel-level TF-IDF distribution, dynamically adjusts the weighting coefficients of the global feature extractor and the local feature extractor using the channel-level discriminant score, and aggregates the global feature extractor and the local feature extractor according to the weighting coefficients.

[0010] In some embodiments of the present application, the method further comprises a federated forgetting operation verification step, specifically including: after completing each round of federated forgetting, the server records the pruning index set of the pruned channels in the global feature extractor, saves the TF-IDF distribution snapshot of each round, compares the output probabilities of the global feature extractor before and after pruning, evaluates the prediction failure degree, and directly verifies the effect of federated forgetting processing.

[0011] In some embodiments of the present application, the method further comprises: performing stability detection on the federated learning training without accessing the original data, indirectly confirming whether the influence of the deleted class has been removed from the model structure.

[0012] In some embodiments of the present application, when the target class set contains multiple target classes, in the pruning processing step, it includes: according to the multi-class order processing mechanism, separately calculating the channel-level TF-IDF distribution and the channel-level discriminative score for each target class, and then separately forming a class-independent mask for each channel, generating a channel deletion set through intersection operation, and pruning each local feature extractor according to the channel deletion set.

[0013] Corresponding to the above method, the present application also provides a class-level federated forgetting method based on double-layer expert mixing and TF-IDF discriminative pruning. In each communication round, the method includes: receiving the initial or updated global feature extractor issued by the server side; aggregating the global feature extractor and the locally stored local feature extractor using the inner-layer expert mixing model, specifically including: performing the same structural pruning on the local feature extractor according to the pruning processing of the global feature extractor, to keep the federated forgetting depth consistent between the client and the server; calculating the channel-level TF-IDF distribution of each local feature extractor on each target class, calculating the channel-level discriminative score of each local feature extractor on each target class according to the channel-level TF-IDF distribution, determining the weighting coefficients of the global feature extractor and the pruned local feature extractor based on the channel-level discriminative score, and locally weighting aggregating the global feature extractor and the pruned local feature extractor according to the weighting coefficients; after the current round of federated learning is completed, uploading the locally stored local feature extractor to the server side.

[0014] In some embodiments of the present application, the method further comprises: according to an annealing learning rate strategy, fine-tuning the global feature extractor based on the local feature extractor after structural pruning.

[0015] In some embodiments of the present application, the method further comprises performing local precise forgetting and / or local forgetting recovery at the client, specifically including: local precise forgetting, using the calculated channel-level TF-IDF distribution and channel-level discriminative score to shield the channels in the local feature extractor whose channel-level discriminative score exceeds the set threshold; local forgetting recovery, retaining each channel deleted by the local feature extractor, and when receiving a local forgetting recovery request, recovering the target channels contained in the local forgetting recovery request.

[0016] Corresponding to the above method, the application also provides a class-level federated forgetting system based on double-layer expert mixing and TF-IDF discriminant pruning, comprising a processor, a memory and computer programs / instructions stored on the memory, the processor being configured to execute the computer programs / instructions, and the system implementing the steps of the method according to any one of the above embodiments when the computer programs / instructions are executed.

[0017] Corresponding to the above method, the application also provides a computer readable storage medium having computer programs / instructions stored thereon, the computer programs / instructions being executed by a processor to implement the steps of the method according to any one of the above embodiments.

[0018] The class-level federated forgetting method based on double-layer expert mixing and TF-IDF discriminant pruning according to the application can support different modalities and different model structures of clients to perform personalized and controllable class-level forgetting operations in federated learning, avoid full model retraining, significantly reduce forgetting cost, and maximize the reasoning ability for un-forgetting classes.

[0019] Additional advantages, objects, and features of the application will be set forth in part in the description which follows, and will become apparent to those skilled in the art upon examination of the following or can be learned by practice of the application. The objects and other advantages of the application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0020] It will be understood by those skilled in the art that the objects and advantages of the present application can not be limited to the above specifically described, and the above and other objects that can be achieved by the present application will be more clearly understood according to the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0021] The drawings described herein are intended to provide further understanding of the present application, form a part of the application, and do not constitute a limitation of the present application. In the drawings:

[0022] Figure 1 Flowchart of the class-level federated forgetting method in some embodiments of the present application.

[0023] Figure 2 Flowchart of the class-level federated forgetting method in some embodiments of the present application. DETAILED DESCRIPTION

[0024] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the embodiments and drawings. Herein, the illustrative embodiments of the present application and their descriptions are used to explain the present application but not as a limitation of the present application.

[0025] Herein, it is also necessary to note that, in order to avoid obscuring the present application due to unnecessary details, only the structures and / or processing steps closely related to the solutions according to the present application are shown in the drawings, and other details not closely related to the present application are omitted.

[0026] It should be emphasized that the term "comprises / comprising" as used herein is used to indicate the presence of a feature, element, step or component but does not preclude the presence or addition of one or more other features, elements, steps or components.

[0027] Herein, it is also necessary to note that, if not specifically stated, the term "connected" as used herein can not only mean direct connection but also mean indirect connection with an intermediate.

[0028] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts or the same or similar steps.

[0029] In the existing federated forgetting technology, although multiple improvements are proposed from the dimensions of computing overhead, communication cost and deletion granularity, etc., there are still compatibility problems caused by model structure heterogeneity in the complex real scene of mixed heterogeneous clients and strong non-IID data. For example, most forgetting methods assume that the client models are homogeneous. When the terminal adopts a differentiated network, the traditional local backtracking or pruning strategy cannot guarantee that the deletion depth and global performance are synchronized, and even can cause a significant decrease in the accuracy of un-deleted classes. In addition, when the same data segment contributes different weights to different clients, the deletion demand of a certain end may damage the task performance of another end; existing works lack efficient mechanisms to coordinate such cross-end conflicts and maintain aggregation fairness. In addition, although pruning and local retraining perform well in experimental environments, they still need to frequently synchronize a large number of gradients or sub-model updates in actual deployment, and the communication volume grows rapidly with the number of nodes and the model size, making it difficult for edge devices to bear. At the same time, existing evaluations often focus on accuracy or reasoning error, and lack quantitative measurement of "whether deletion is complete", which makes it difficult to meet the compliance requirements of GDPR and other regulations on traceable and verifiable forgetting.

[0030] In order to overcome the problems of the existing federated forgetting technology as described above, the present application proposes a class-level federated forgetting method and system based on double-layer expert mixing and TF-IDF discriminative pruning, which can be compatible with heterogeneous federated environments, and can realize accurate, efficient and verifiable data deletion while maintaining the main task performance of the model.

[0031] To further reduce the overhead of federated forgetting, pruning methods can be used. Linear filtering, neuron masking and other methods can deactivate parameters that are strongly related to target data without changing the complete network structure, thereby quickly completing deletion and forgetting. On this basis, researchers have proposed "class discrimination pruning", which counts the contribution of each channel to a specific class and prunes high-contribution channels to achieve more detailed class-level forgetting under the condition of non-independent and identically distributed (Non-IID) and reduce resource consumption caused by repeated retraining.

[0032] To extend the application scenario from homogeneous models to heterogeneous client environments, personalized and heterogeneous federated learning methods can be introduced into the field of federated forgetting. For example, mutual learning alternately trains between local and shared models through low-rank adapters to adapt to different data distributions at each end; prototype alignment uses class prototypes to calibrate the global model to alleviate the aggregation drift caused by non-independent and identically distributed; distillation and integration fusion significantly reduce the communication cost of multiple-end synchronous complete weight through knowledge transfer or sub-model fusion. In the direction of parameter compression, pruning and sub-model training have been proven to significantly reduce model size and are easy to combine with the forgetting process. In addition, asynchronous forgetting of longitudinally segmented models, cross-domain random forest reversible training, and isolated coding shards provide feasible ideas for deploying FU in vertical data and large-scale node scenarios.

[0033] The present application proposes a class-level federated forgetting method suitable for heterogeneous model structures and non-independent and identically distributed (Non-IID) data environments. This method uses a double-layer expert mixed structure (Mixture-of-Experts, MoE). The present application divides federated training and federated forgetting into four key stages around the communication round: (1) client selection and model distribution; (2) client local training and uploading features; (3) server aggregation and global forgetting operation; (4) client receives updates to complete local fine-tuning and forgetting recovery. The four-stage process is executed in a loop within each communication round until all training and forgetting tasks are completed.

[0034] In terms of system structure, the application introduces a two-layer mixture of experts (MoE) structure, which is deployed on the client side and the server side respectively. The inner MoE model is built on the client side, which consists of a global feature extractor, a local feature extractor, a first gating network and a prediction head module. Among them, the global feature extractor is provided by the server to maintain semantic consistency with other clients; the local feature extractor is trained by the client according to its own private data, taking into account personalized tasks; the first gating network is used to coordinate the fusion weight between the two; the prediction head module is used to complete the final classification prediction. On the server side, the outer MoE model is built, taking all the local feature extractors uploaded by the clients as "client experts", and the global feature extractor maintained by the server itself as "global expert", and realizing expert selection and aggregated update through a centralized gating mechanism. On the client side, a local feature expert and a personalized gating network are built, and the TF-IDF mechanism is used to evaluate and select the feature channels, so as to realize accurate identification and forgetting of local sensitive categories; on the server side, the feature extractors uploaded by each client are aggregated to form a global expert set, and the global gating mechanism is used to complete the aggregated analysis and channel pruning of the influence of specific categories, so as to realize unified forgetting processing across clients and modalities. In addition, in order to reduce the communication burden and protect user privacy, the application can only transmit lightweight feature extractors during federated training and forgetting, instead of complete model parameters, so as to balance efficiency and privacy. Through the above design, the application realizes efficient, accurate and personalized forgetting of specific category information while ensuring the stability of the model performance, and is especially suitable for complex federated learning environments under non-IID conditions, and has good practical application value and promotion prospect.

[0035] Figure 1 For the class-level federated forgetting method flowchart in some embodiments of the application, the execution subject is the server side and / or the client side, and global forgetting is performed on the server side. In each communication round, the method comprises:

[0036] Step S110: Downlink the initial or updated global feature extractor to each client participating in federated learning training; wherein the global feature extractor and the locally stored local feature extractor of each client are aggregated by the inner MoE model.

[0037] Step S120: Receive the local feature extractor uploaded by each client.

[0038] Step S130: Aggregate all received local feature extractors using the outer MoE model. Specifically, it includes:

[0039] Step S131: When receiving the forgetting request containing the target category set from the client, the channel-level TF-IDF distribution of each local feature extractor on each target category contained in the target category set is calculated, the channel-level discriminative score of each local feature extractor on each target category is calculated based on the channel-level TF-IDF distribution, the channels of each local feature extractor whose channel-level discriminative score exceeds a set threshold are pruned, the weighting coefficient of each local feature extractor is determined based on the channel-level discriminative score, the weighting coefficient is used to aggregate all pruned local feature extractors, and the alignment processing is performed on the aggregation result to obtain the updated global feature extractor.

[0040] Step S132: When the forgetting request is not received, the received all local feature extractors are aggregated according to the fixed gating strategy.

[0041] The application specially designs a double-layer expert mixing structure, constructs an inner expert mixing model (InnerMoE) on the client side, and constructs an outer expert mixing model on the server side. The inner expert mixing model includes a global feature extractor, a local feature extractor, a first gating network, and a prediction head module. The outer expert mixing model uses the local feature extractor uploaded by the client as a client expert, and uses the global feature extractor maintained by the server as a global expert. The client expert and the global expert are selected and aggregated for updating. The global feature extractor is provided and maintained by the server side, the local feature extractor is obtained by training according to the private data of the client, the first gating network is used to coordinate the fusion weight between the global feature extractor and the local feature extractor, and the prediction head module is used to complete the final classification prediction.

[0042] The class-level federated forgetting method based on double-layer expert mixing and TF-IDF discriminative pruning can support different modalities and different model structures of the client to perform personalized and controllable class-level forgetting operation in federated learning, avoid full model retraining, significantly reduce the forgetting cost, and maximize the reasoning ability for non-forgetting categories. The method supports different modalities and different model structures of the client to perform personalized and controllable class-level forgetting operation in federated learning, avoid full model retraining, significantly reduce the forgetting cost, and maximize the reasoning ability for non-forgetting categories.

[0043] In some embodiments of the present application, the inner-layer expert hybrid model comprises a global feature extractor, a local feature extractor, a first gating network and a prediction head module. Accordingly, in this embodiment, the global feature extractor and its locally stored local feature extractor are aggregated by each client using the inner-layer expert hybrid model, including: in the first round of communication, the first gating network fuses the global feature extractor and the local feature extractor according to a preset proportion; starting from the second round, the first gating network determines a channel-level discriminative score according to a channel-level TF-IDF distribution, dynamically adjusts the weighting coefficients of the global feature extractor and the local feature extractor using the channel-level discriminative score, and aggregates the global feature extractor and the local feature extractor according to the weighting coefficients.

[0044] By adopting the embodiments of the present application, the problems of coarse forgetting granularity, large model performance loss, high computing and communication overhead, and lack of verifiable mechanism of existing federated forgetting methods in a heterogeneous environment can be solved.

[0045] In some embodiments of the present application, the method further comprises a federated forgetting operation verification step, specifically comprising: after completing each round of federated forgetting, the server records the pruning index set of the channels that have been removed in the global feature extractor, and saves the TF-IDF distribution snapshot of each round, compares the output probabilities of the global feature extractor before and after pruning processing in the target category, evaluates the prediction failure degree, and directly verifies the effect of federated forgetting processing.

[0046] By adopting the embodiments of the present application, the heterogeneous federated environment can be compatible, and precise, efficient and verifiable data deletion can be realized while maintaining the performance of the main task of the model.

[0047] Further, in some embodiments of the present application, the method further comprises: without accessing the original data, performing stability detection on the federated learning training to indirectly confirm whether the influence of the deleted categories has been removed from the model structure.

[0048] In some embodiments of the present application, when the target category set contains multiple target categories, in the pruning processing step, the channel-level TF-IDF distribution and the channel-level discriminative score are calculated for each target category separately according to a multi-category order processing mechanism, and then a category-independent mask is formed for each channel, a channel deletion set is generated through intersection operation, and the pruning processing is performed on each local feature extractor according to the channel deletion set.

[0049] By adopting the embodiments of the present application, the over damage of the feature space caused by the coincidence of target categories can be avoided.

[0050] On the other hand, the present application takes each client as the execution subject and proposes a class-level federated forgetting method based on double-layer expert hybrid and TF-IDF discriminative pruning.Figure 2 For the class-level federated forgetting method flowchart in some embodiments of the application, in each communication round, the method comprises:

[0051] Step S210: receiving an initial or updated global feature extractor issued by a server side.

[0052] Step S220: aggregating the global feature extractor and the locally stored local feature extractor by using an inner-layer expert hybrid model, specifically comprising: according to the pruning processing of the global feature extractor, performing the same structural pruning on the local feature extractor to keep the federated forgetting depth of the client and the server side consistent; calculating the channel-level TF-IDF distribution of each local feature extractor on each target class, calculating the channel-level discriminative score of each local feature extractor on each target class according to the channel-level TF-IDF distribution, determining the weighting coefficients of the global feature extractor and the pruned local feature extractor based on the channel-level discriminative score, and locally weighting and aggregating the global feature extractor and the pruned local feature extractor according to the weighting coefficients.

[0053] Step S230: uploading the locally stored local feature extractor to the server side when the current round of federated learning ends.

[0054] The class-level federated forgetting method based on double-layer expert hybrid and TF-IDF discriminative pruning provided by the application can support different modalities and different model structures of the client to perform personalized and controllable class-level forgetting operation in federated learning, avoid full model retraining, significantly reduce the forgetting cost, and maximize the reasoning ability for non-forgetting classes. The method supports different modalities and different model structures of the client to perform personalized and controllable class-level forgetting operation in federated learning, avoid full model retraining, significantly reduce the forgetting cost, and maximize the reasoning ability for non-forgetting classes.

[0055] In some embodiments of the application, the method further comprises: according to an annealing learning rate strategy, fine-tuning the global feature extractor based on the local feature extractor after structural pruning.

[0056] By using the embodiments of the application, structural mutations caused by pruning can be avoided, and model performance collapse can be avoided, and the model can be gradually adapted to the residual feature space after structural changes.

[0057] In some embodiments of the present application, the method further comprises local precise forgetting and / or local forgetting recovery at the client, specifically comprising: local precise forgetting, shielding the channels in the local feature extractor whose channel-level discriminative scores exceed the set threshold, using the calculated channel-level TF-IDF distribution and channel-level discriminative scores; and local forgetting recovery, retaining each channel deleted by the local feature extractor, and restoring the target channel contained in the local forgetting recovery request when receiving the local forgetting recovery request.

[0058] In a specific embodiment of the present application, in each communication round, the specific process of federated training and federated forgetting is as follows:

[0059] (1) Client selection and model issuance: At the beginning of each round of communication, the server randomly selects K clients for training from N clients.

[0060] (2) Client local training and feature uploading: The selected clients will receive the global feature extractor under the current round The feature extractor will be loaded into the local Inner MoE structure by the client, and will be trained with the local model to learn the fused feature representation. After the local training phase is completed, the client uploads the aligned local feature extractor Instead of the complete model or the prediction head parameters, which reduces the communication burden and reduces the risk of privacy leakage.

[0061] (3) Server aggregation and global forgetting operation: After receiving the feature extractors uploaded by all clients, the server will input them as "expert list" into the Outer MoE together with the existing global experts ω e to participate in aggregation. If a class-level forgetting request U is triggered in this round, the server will perform class discriminative pruning and update its global expert model accordingly; if forgetting is not triggered, the gating mechanism and expert selection strategy of the last round are followed. After updating, the server reissues the aggregation result to the client.

[0062] (4) Client receives the updated aggregation result and completes local fine-tuning and forgetting recovery: The client receives the updated aggregation result issued by the server, which in this application is updated by updating the global feature extractor contained in the client's inner expert mixing model. The client then updates the global feature extractor using the updated global feature extractor and completes the local fine-tuning process.

[0063] The specific process of the above federal training and federal forgetting can cover both regular federal training and class-level forgetting. In the normal training round, the system mainly uses fast aggregation to improve generalization ability; in the forgetting trigger round, the system focuses on the removal operation of the specified class influence, and combines the first gating network and the pruning mechanism to complete the "erasure" of the target class while maintaining the performance stability of non-target classes. By cyclically executing the above four-stage process, the present application can complete the cooperative optimization of training and forgetting in multiple communication rounds, and support flexible configuration of training period, forgetting trigger round, and class selection strategy.

[0064] In some embodiments of the present application, in order to realize efficient personalized feature learning and class-level forgetting, an inner expert mixture model (Inner MoE) is introduced at the client side. The model is composed of two feature extractors, a first gating network and a prediction head module, which are respectively used to extract shared semantic information, learn local feature bias, dynamically allocate expert weights and complete final classification prediction.

[0065] Specifically, the Inner MoE includes the following components:

[0066] (1) The global feature extractor (Global Expert) is uniformly initialized by the server and broadcast to the selected clients in each communication round. Its structure and parameters remain consistent, and it is used to capture feature information common to all clients. During the forward propagation of the model, the global feature extractor can be used to compensate for the missing semantic expression in the local model, thereby improving the prediction stability of small sample classes or rare modalities.

[0067] (2) The local feature extractor (Local Expert) is trained by the client according to the private dataset, reflecting the individualized features of the current terminal in terms of data distribution, input modalities, class distribution, etc. This extractor is relatively independent of the structure of the global expert, allowing clients to flexibly configure different network architectures (such as CNN, ResNet, BiLSTM, etc.) to adapt to heterogeneous computing needs.

[0068] (3) The first gating network (Personalized Gating Network) is responsible for dynamic weight distribution between global experts and local experts. The first gating network calculates the fusion ratio according to the response degree (i.e. activated features) of the input data on the two groups of experts and its historical performance (such as class accuracy, feature entropy, etc.). In the first round of communication, the gating mechanism uses a preset ratio (such as 0.5 / 0.5) for fusion; from the second round, the gating weight will be dynamically adjusted according to the TF-IDF distribution to emphasize the experts with stronger performance on the target class. The gating output is the weighted sum of the outputs of the two feature extractors, which constitutes the final intermediate representation.

[0069] (4) The prediction head module receives the feature representation after the gating output and performs final classification prediction. The module structure can be configured according to the specific task (such as multilayer perceptron, Softmax classifier, etc.), and is locally trained end-to-end with cross-entropy loss as the target. In order to improve the robustness of the model, a regularization term is added to the local training process to maintain the consistency of the local expert output and the global features, thereby reducing the distribution drift in the aggregation process.

[0070] In some embodiments of the present application, in the class-level forgetting case, a channel weight evaluation method based on the TF-IDF index is designed to assist the gating mechanism in determining the importance of each channel and its relevance to the target class. The index combines the activation intensity (Term Frequency, TF) of the channel in a specific class and its rarity (Inverse Document Frequency, IDF) in all classes for calculation, and its basic form is as follows:

[0071]

[0072] TF-IDFu,j=TFu,j×IDF u,j

[0073] wherein, represents the activation value of the jth channel of the client k on the class u, and |U| is the total number of classes. The higher the TF-IDF score, the higher the discriminative ability of the channel in the target class.

[0074] The first gating network uses the channel-level score to assign weights to different experts, and also provides a basis for subsequent channel pruning and forgetting mechanisms. During training, the client calculates the channel-level TF-IDF distribution for all samples, and uses this result to adjust the model structure or mask highly relevant channels to achieve precise forgetting at the class level.

[0075] The structure has high scalability, can support seamless integration between different modalities (images, text, voice, etc.) and heterogeneous model architectures, and can effectively improve personalized performance and target class recognition ability through dynamic gating mechanism and discriminative channel scoring.

[0076] In some embodiments of the present application, for the server side, in order to realize the unified coordinated class-level forgetting operation under the multi-client heterogeneous model structure, the present application introduces an outer mixture-of-experts (Outer MoE) model on the server side. The model takes the aggregated local feature extractors uploaded by the clients as the core, and through the centralized gating network, dynamically selects and updates each "client expert" to form a global expert representation with stable performance. This mechanism not only supports cross-client information fusion, but also provides a unified execution interface for target channel elimination in class-level forgetting.

[0077] The server receives the local feature extractors uploaded by each client in each round of communication These feature extractors are regarded as "client experts" and are uniformly input into the Outer MoE model, together with the global feature extractor ω e held by the server to form an expert pool. When the forgetting request is not triggered, the Outer MoE uses a fixed gating strategy to weight and aggregate the client experts to update the new global expert parameters, i.e., the updated global feature extractor; once the class forgetting request U is received, the Outer MoE will activate a series of pruning and re-aggregation mechanisms to eliminate the influence of the class in the global feature extractor.

[0078] Specifically, when the forgetting request is triggered, the server-side aggregation and global forgetting process is as follows:

[0079] First, the Outer MoE calculates the channel-level TF-IDF distribution of all client-uploaded feature extractors on the target class. This process reuses the client-side calculation method to obtain the discriminative score of each channel under the target class. Then, the server removes the channels with scores higher than the threshold in each client expert according to the set threshold R, thereby forming the pruned client feature extractor \hat{\theta}_{e}^{k}.

[0080] Next, the server uses the TF-IDF score as a weighting coefficient to weight and aggregate (fuse) all client experts to generate a new global feature extractor (in the mixture-of-experts model, the global feature extractor is also called the global expert representation). The weighted aggregation process is expressed as follows:

[0081]

[0082] where, TF-IDF k The average score of all channels of the client k under the target category is represented, and is used as an expert importance evaluation index. This aggregation method effectively suppresses the residual influence of high correlation feature channels on the target category, while retaining the generalization ability to non-target categories.

[0083] After obtaining the aggregation result, the server will also generate a new global feature extractor The input batch normalization module (Batch Normalization) is used for alignment processing to alleviate the distribution deviation problem caused by model structure differences. The mathematical expression of this step is:

[0084]

[0085] Ensure that the output result is consistent in distribution for all clients. Subsequently, the server broadcasts the updated global feature extractor and the gating strategy configuration to all online clients, so that they perform the next round of local training or forgetting operation.

[0086] It is particularly worth noting that in the case of multi-class forgetting, the present application adopts a mechanism of "pruning by class, aggregation in turn", that is, each to-be-deleted class is processed in turn, and each time channel screening and expert aggregation are performed, and in the final fusion, each class of information is independently removed without interference. This design better solves the problem of channel overlap under multi-target interference, effectively improving the coverage and accuracy of the deletion operation.

[0087] Through the above mechanism, Outer MoE not only has the ability to handle heterogeneous structures and inconsistent information of clients, but also can perform class-level influence removal in a lightweight manner, providing a unified and stable forgetting entry for the entire federated system.

[0088] In order to achieve effective deletion of target categories without sacrificing the overall performance of the model, the present application designs a complete class-level federated forgetting process, including a triggering mechanism, a local pruning strategy, a local fine-tuning process, and subsequent verification and compensation steps. This process is nested in the standard federated communication round and is started immediately when the conditions are met, which can coordinate the server and other clients to complete fast forgetting synchronization while the client requests.

[0089] In some specific embodiments of the present application, the process of federated learning and federated forgetting jointly performed by the server and the client is as follows:

[0090] When a client initiates an unlearning request at communication round t = t1, specifying the target class set U it wants to delete from the global model, the server enters the "unlearning phase". At this time, the server freezes the current global expert parameters, suspends the regular aggregation process, and instead starts a specific channel screening and global update procedure, which includes the following steps:

[0091] First, the server calls the latest version from the history uploaded feature extractor, calculates the channel-level TF-IDF distribution of all experts on the target class U. This distribution is used to identify the strongest channel set C in the model that discriminates the class to be forgotten U , to establish a global channel mask for subsequent pruning. Here, the channel importance measure still uses the formula:

[0092]

[0093] For each client expert, if the TF-IDF score of a channel j exceeds the global threshold R in any target class, the channel is considered to be removed and a structure-level pruning operation is performed. The server processes each client uploaded model independently and inputs the pruned model into the formula to complete new global expert aggregation:

[0094]

[0095] where, represents the client feature extractor copy that has completed channel pruning.

[0096] Subsequently, the server distributes the newly generated global expert to all online clients. After receiving the update, each client synchronously performs the following operations: first, re-performs the same structure pruning on the local expert according to the global channel mask, ensuring that the unlearning depth is consistent with the server; second, retains the local channels that have not been pruned, and uses the local remaining class data for a few rounds of fine-tuning. The fine-tuning follows an annealing learning rate strategy:

[0097] η (l) = η0· γ l , 0 < γ < 1, l ≤ L max ,

[0098] where l is the number of fine-tuning rounds, and η0is the initial learning rate. Through this adjustment strategy, structural mutations caused by pruning can be avoided, preventing the model from collapsing, and promoting the model to gradually adapt to the residual feature space after structural changes.

[0099] Further, in some embodiments, to ensure that the forgetting operation is verifiable, the present application retains the following verification paths: first, the server side records the global channel pruning index set after completing each round of federated forgetting, and synchronously saves each round of TF-IDF distribution snapshot; second, by comparing the output probabilities of the global feature extractor before and after pruning in the target category, the prediction failure degree is evaluated to directly confirm the effect of federated forgetting execution; third, without accessing the original data, according to the (retrained) retained category data, training stability detection (such as prediction variance change, category mean drift, etc.) is performed to indirectly confirm whether the influence of the deleted category has been removed from the model structure.

[0100] In some embodiments of the present application, to alleviate the problem of channel redundancy deletion that may be caused by multi-class overlap pruning, the present application supports a multi-class sequential processing mechanism, that is, TF-IDF calculation and channel pruning are performed separately for each to-be-deleted category, a category-independent mask is formed, and a final channel deletion set is generated through intersection operation to avoid excessive damage to the feature space caused by target category overlap.

[0101] The class-level federated forgetting method and system based on double-layer expert hybridization and TF-IDF discriminative pruning proposed by the present application can support different modalities and different model structures of clients to perform personalized and controllable class-level forgetting operations in federated learning, avoid full-model retraining, significantly reduce the forgetting cost, and maximize the reasoning ability for non-forgetting categories. The method supports different modalities and different model structures of clients to perform personalized and controllable class-level forgetting operations in federated learning, avoid full-model retraining, significantly reduce the forgetting cost, and maximize the reasoning ability for non-forgetting categories.

[0102] Moreover, the entire forgetting process can be flexibly inserted into the communication cycle of model training, supporting unified deletion after training and intermittent execution during training, thereby providing technical support for different application scenarios (such as edge data compliance and user-initiated revocation). The average time delay of executing a forgetting process is much less than the time required for full-model retraining, and the accuracy drop is within the control range (usually <3%), which can meet the performance and efficiency requirements of industrial-level data deletion.

[0103] Experiments have shown that after a single forgetting trigger, the recognition accuracy of the target category can be greatly reduced, while the average accuracy of non-target categories does not decrease significantly. Under the premise of ensuring the same forgetting depth, the communication overhead is greatly reduced compared to full-volume retraining. Compared with the mainstream differential privacy noise injection or full-volume retraining strategy, the present application achieves synergistic optimization in model fidelity, communication and computation cost, and forgetting interpretability, and embodies significant technological progress.

[0104] The technical key points of the class-level federated forgetting method and system based on double-layer expert mixing and TF-IDF discriminative pruning designed by the application include but are not limited to:

[0105] (1) For the non-IID federated learning scenario, a "double-layer expert mixing" forgetting framework is constructed. The dynamic fusion of local experts and global experts is completed by the inner MoE on the client side, and the unified coordination across clients and modalities is realized by the outer MoE on the server side, which balances global consistency and local difference from the system architecture, significantly improving the portability and scope of application of class-level forgetting.

[0106] (2) The channel-level TF-IDF discriminative index is introduced to measure the discriminativeness of each channel for the specified class by the convolutional activation strength, which not only provides an interpretable quantitative standard for the forgetting range, but also ensures the consistency of the forgetting depth as a pruning threshold; Through this quantitative mechanism, accurate class-level influence elimination can be completed without relying on the original data.

[0107] (3) A distributed discriminative pruning-aggregation process is designed. The server performs discriminative pruning on the feature extractors uploaded by the client according to the TF-IDF weight, and generates a new global expert using a weighted aggregation strategy; The client synchronously performs the same pruning and performs a lightweight fine-tuning, which reduces the target class accuracy to zero while only slightly reducing the performance of non-target classes while maintaining stable convergence of the global model.

[0108] (4) A lightweight communication and batch normalization alignment mechanism is proposed. Only the aligned feature extractor is transmitted instead of the complete model weight, and the batch normalization is used to suppress model distribution drift, which reduces the communication cost to one-fifteenth of the traditional retraining scheme, the server computing cost to one-seventh, and the client computing cost to one-sixtieth while ensuring privacy and security.

[0109] (5) A verifiable forgetting criterion is constructed. The TF-IDF score and pruning record are used to provide a traceable and auditable objective evidence chain for forgetting effect, meeting the rigid demand for post-validation in compliance scenarios, and laying a technical foundation for industrial-level privacy governance and responsibility tracing.

[0110] The class-level federated forgetting method and system based on double-layer expert mixing and TF-IDF discriminative pruning proposed by the application can solve the technical bottleneck of class-level data forgetting in heterogeneous federated learning environment, which cannot balance efficiency, accuracy and verifiability, and reduce communication and computing cost while meeting the "right to be forgotten" requirements of regulations such as GDPR.

[0111] Corresponding to the above method, the application also provides a class level federal forgetting system based on double-layer expert mixing and TF-IDF discriminant pruning, which comprises a computer device including a processor and a memory, and the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory, and the system realizes the steps of the method as described above.

[0112] Corresponding to the above method, the application also provides a computer readable storage medium, which stores computer programs / instructions, and the computer programs / instructions are executed by a processor to realize the steps of the method in any one of the above embodiments. The computer readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the art.

[0113] Corresponding to the above method, the application also provides a computer program product, which comprises computer programs / instructions, and the computer programs / instructions are executed by a processor to realize the steps of the method in any one of the above embodiments.

[0114] Those of ordinary skill in the art should understand that the exemplary components, systems and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software, or a combination thereof. The exact implementation depends on the specific application and design constraints imposed on the overall system. Those of skill can use various approaches to implement the described functionality depending on the specific application, and the present application is not limited to a particular approach. When implemented in hardware, the hardware can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), a suitable firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the present application are the program or code segments to perform the necessary tasks. The program or code segments can be stored in a machine readable medium, or transmitted by a carrier wave in a transmission medium or communication link.

[0115] It should be understood that the present application is not limited to the particular configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of known methods are omitted. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of the steps, after understanding the spirit of the present application.

[0116] Features described and / or illustrated with respect to one implementation can be used in the same manner or in a similar manner in one or more other implementations and / or in combination with or in place of features of other implementations.

[0117] The preferred embodiments of the present application are described above with reference to the drawings, and it will be apparent that modifications and changes can be made thereto without departing from the spirit and scope of the application. Therefore, it is to be understood that the application is not to be limited to the particular embodiments disclosed, but it is intended to cover all modifications and changes that are within the scope and spirit of the application.

Claims

1. A class-level federated forgetting method based on double-layer expert mixing and TF-IDF discriminative pruning, characterized in that, In each communication round, the method comprises: downloading the initial or updated global feature extractor to each client participating in federated learning training; wherein each client aggregates the global feature extractor and its locally stored local feature extractor using an inner expert hybrid model; receiving the local feature extractors uploaded by each client; aggregating all received local feature extractors using an outer expert hybrid model, specifically including: when receiving a forgetting request from a client containing a target category set, calculating the channel-level TF-IDF distribution of each local feature extractor on each target category contained in the target category set, calculating the channel-level discriminative score of each local feature extractor on each target category based on the channel-level TF-IDF distribution, pruning the channels in each local feature extractor whose channel-level discriminative score exceeds a certain threshold, determining the weighting coefficients of each local feature extractor based on the channel-level discriminative score, and weighting and aggregating all pruned local feature extractors according to the weighting coefficients, and performing alignment processing on the weighting aggregation result to obtain an updated global feature extractor; when no forgetting request is received, weighting and aggregating all received local feature extractors according to a fixed gating strategy.

2. The method of claim 1, wherein, The inner expert hybrid model comprises a global feature extractor, a local feature extractor, a first gating network, and a prediction head module; Each client aggregates the global feature extractor and its locally stored local feature extractor using an inner expert hybrid model, which comprises: In the first round of communication, the first gating network fuses the global feature extractor and the local feature extractor according to a preset proportion; From the second round, the first gating network determines the channel-level discriminative score based on the channel-level TF-IDF distribution, dynamically adjusts the weighting coefficients of the global feature extractor and the local feature extractor using the channel-level discriminative score, and aggregates the global feature extractor and the local feature extractor according to the weighting coefficients.

3. The method of claim 1, wherein, The method further comprises a federated forgetting operation verification step, specifically including: the server records the pruning index set of the pruned channels in the global feature extractor after completing each round of federated forgetting, saves the TF-IDF distribution snapshot of each round, compares the output probabilities of the global feature extractor before and after pruning processing on the target category, evaluates the prediction failure degree, and directly verifies the effect of federated forgetting processing.

4. The method of claim 3, wherein, The method further comprises: Without accessing the original data, the stability of federated learning training is detected to indirectly confirm whether the influence of the deleted categories has been removed from the model structure.

5. The method of claim 1, wherein, When the target category set contains multiple target categories, in the pruning processing step, it comprises: According to the multi-category order processing mechanism, the channel-level TF-IDF distribution and the channel-level discriminative score are calculated for each target category, and then a category-independent mask is formed for each channel. The channel deletion set is generated by intersection operation, and each local feature extractor is pruned according to the channel deletion set.

6. A class-level federated forgetting method based on double-layer expert mixing and TF-IDF discriminative pruning, characterized in that, In each communication round, the method comprises: receive an initial or updated global feature extractor issued by a server side; aggregate the global feature extractor and the locally stored local feature extractor using an inner-layer expert mixing model, specifically including: pruning the local feature extractor according to the pruning of the global feature extractor to keep the federated forgetting depth of the client and the server consistent; calculating the channel-level TF-IDF distribution of each local feature extractor on each target class, calculating the channel-level discriminativeness score of each local feature extractor on each target class according to the channel-level TF-IDF distribution, determining the weighting coefficients of the global feature extractor and the pruned local feature extractor based on the channel-level discriminativeness score, and aggregating the global feature extractor and the pruned local feature extractor locally according to the weighting coefficients; upload the locally stored local feature extractor to the server side when the current round of federated learning ends.

7. The method of claim 6, wherein, The method further includes fine-tuning the global feature extractor based on the pruned local feature extractor according to an annealing learning rate strategy.

8. The method of claim 6, wherein, The method further includes local precise forgetting and / or local forgetting recovery at the client, specifically including: local precise forgetting, shielding the channels of the local feature extractor whose channel-level discriminativeness score exceeds a set threshold, using the calculated channel-level TF-IDF distribution and channel-level discriminativeness score; local forgetting recovery, retaining each channel deleted by the local feature extractor, and restoring the target channel contained in the local forgetting recovery request when receiving the local forgetting recovery request.

9. A class-level federated forgetting system based on double-layer expert mixing and TF-IDF discriminative pruning, comprising a processor, a memory and computer programs / instructions stored on the memory, characterized in that, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method according to any one of claims 1 to 8.

10. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to implement the steps of the method according to any one of claims 1 to 8.

Images