Longitudinal federal model forgetting method and device for lightweight adaptive optimizer scheduling

Abstract

The invention provides a longitudinal federated model forgetting method and device for lightweight adaptive optimizer scheduling, and relates to the technical field of federated learning. The method comprises the following steps: acquiring local data sets of an active party and a passive party and a local model adopting an underlying model check point strategy, and acquiring a forgetting request of the passive party; deleting local target data according to the forgetting request and initializing a local model; and performing performance recovery by adopting an RAdam stage or an SGDM stage according to the convergence index to obtain a forgotten local model. According to the method, a bottom model check point strategy is adopted in the aspect of storage, only necessary bottom layer states are reserved, and redundant intermediate states and logs are prevented from being stored; in the aspect of calculation, hot start recovery is realized by relying on self-adaptive optimizer switching, dynamic alternation is carried out between an RAdam stage and an SGDM stage according to a simple convergence index, and the performance of the model after forgetting is recovered to be close to a retraining level with low communication, low calculation and low storage overhead.

Description

Technical Field

[0001] This invention relates to the fields of federated learning and privacy computing, and in particular to a forgetting method and apparatus for a longitudinal federated model using a lightweight adaptive optimizer scheduler. Background Technology

[0002] Federated learning is a distributed machine learning paradigm that involves multiple clients collaborating with one or more servers. This paradigm reduces the risk of privacy breaches by jointly training a global model by sharing local model parameters rather than transmitting raw data. Based on the distribution of data feature space, it can be divided into horizontal federated learning and vertical federated learning. Vertical federated learning is suitable for situations where each participant holds a non-overlapping subset of features for the same batch of samples, and it has demonstrated good results in various real-world business applications and continues to attract attention. In the vertical federated learning framework, the participants typically consist of an active party and multiple passive parties: the active party is responsible for coordinating the training process and model updates, while the passive parties provide computation on the feature side and collaborate with the active party to complete model training.

[0003] With increasing emphasis on data privacy protection, numerous laws, including the California Consumer Privacy Act and the EU General Data Protection Regulation (GDPR), are increasingly stressing users' "right to be forgotten." In response to these demands, longitudinal federated forgetting learning is considered a promising solution: removing the impact of forgotten data on the global model while maintaining model performance consistent with the pre-forgotten state. Depending on the type of forgetting request, longitudinal federated forgetting learning primarily comprises three levels: sample-level forgetting, feature-level forgetting, and client-level forgetting, corresponding to forgetting specific samples, specific features, and local data on a particular client, respectively.

[0004] Existing vertical federated forgetting schemes can be broadly categorized into two types: schemes tailored to specific model architectures and schemes tailored to general architectures. For schemes tailored to specific model architectures, current research is relatively limited. For example, SecureCut (Federated Gradient Boosting Decision Trees with EfficientMachine Unlearning) achieves feature-level and sample-level forgetting on gradient boosting decision trees without retraining; VFU-LR (Vertical Federated Unlearning Logistic Regression) proposes constraining client-oriented intermediate parameters during training for logistic regression and eliminating their contribution by subtracting the client's updates from the global model, achieving efficient forgetting with only one additional communication round. However, in practical applications, these architecture-specific schemes are less widely applicable than those tailored to general neural network structures.

[0005] Despite the increasing number of designs targeting general neural networks, many existing vertical federated forgetting schemes can only respond to forgetting requests at a limited number of levels. For example, FeaUn-IoT (Feature-Based Machine Unlearning for Vertical Federated Learning in IoT Networks) achieves accurate feature-level forgetting in IoT scenarios by scoring the input dimensions; however, due to its reliance on feature scoring and update protocols, it lacks mechanisms for deleting samples or eliminating client contributions, making it difficult to scale to all forgetting levels. R2S (RAdam to SGDM) combines a bottom-model checkpointing strategy with hybrid first-order optimization, employing Radam (Rectified Adaptive Moment Estimation) and SGDM (Stochastic Gradient Descent Momentum) for performance recovery and preserving model parameters at different training stages to reduce backpropagation overhead; however, when scaled to support sample-level, feature-level, and client-level forgetting requests simultaneously, the computational cost increases significantly. In existing research, only VFU-KD / GA (Vertical Federated Unlearning with Knowledge Distillation / Gradient Ascent) and FADAT (Forgetting Any Data at Any Time) simultaneously support neural network architectures and can handle three levels of forgetting requests. VFU-KD / GA eliminates client and feature contributions through knowledge distillation without additional communication and uses gradient ascent combined with subsequent fine-tuning to achieve sample-level forgetting, but its space complexity is high. FADAT proposes a model- and data-independent, theoretically guaranteed vertical federated forgetting learning framework that can remove arbitrary data at any time and supports asynchronous forgetting across clients; however, the accuracy of this method is relatively low.

[0006] The above limitation analysis shows that there is an urgent need for a lightweight vertical federated forgetting framework that is compatible with any model architecture and supports three types of forgetting requests simultaneously, restores the original accuracy after forgetting, and maintains low computational and storage overhead.

[0007] Most existing vertical federated forgetting methods are either tailored to shallow models or only cover a few forgetting levels, making it difficult to meet real-world deployment needs. In the shallow-model customization direction, SecureCut and VFU-LR are representative examples: the former achieves sample and feature forgetting on gradient boosting trees but is difficult to transfer to deep networks, while the latter, geared towards logistic regression, achieves forgetting by subtracting client increments but has limited adaptability to complex neural networks. In the general architecture direction, some methods, while supporting deep networks, only cover some levels: FeaUn-IoT relies on input dimension scoring to achieve feature-level forgetting but lacks sample-level and client-level mechanisms; R2S combines bottom-model checkpointing and first-order optimization to reduce backpropagation overhead but does not cover the client level, and extending it to three levels significantly increases computational costs. Even the few solutions that simultaneously address both aspects have key shortcomings: FADAT performs forgetting using a confidence matrix, resulting in a significant and difficult-to-recover accuracy drop; VFU-KD / GA handles different levels separately with gradient ascent and knowledge distillation, leading to high space complexity. Therefore, there is an urgent need for a lightweight vertical federated forgetting framework that can be compatible with any network architecture and uniformly support three types of requests, while also being able to stably restore performance to near the level of retraining with low communication and low computational overhead.

[0008] Given the shortcomings of existing vertical federated forgetting methods, there is an urgent need to build a lightweight framework that, without sharing the original data, is compatible with various neural network architectures, uniformly supports sample-level, feature-level, and client-level forgetting, and can stably restore model performance to near-retraining levels with low communication, low computation, and low storage overhead. However, such frameworks face two key challenges: first, to maintain accuracy at different forgetting levels, many methods rely on additional model states or intermediate variables, leading to a significant increase in space complexity; second, performance recovery often requires partial or complete retraining, which often necessitates tens to hundreds of optimization iterations, resulting in high computational costs and making deployment difficult in time- or resource-constrained environments. Summary of the Invention

[0009] To address the challenges of balancing deletion effectiveness, model availability, and system resource overhead in implementing the "right to be forgotten" mechanism in existing technologies, and to improve the stability and accuracy retention after forgetting while reducing communication and storage pressure and shortening recovery latency, this invention provides a lightweight adaptive optimizer-scheduled vertical federated model forgetting method and apparatus. The technical solution is as follows:

[0010] On the one hand, a lightweight adaptive optimizer scheduling method for forgetting longitudinal federated models is provided, which is implemented by a longitudinal federated model forgetting device. The method includes:

[0011] S1. Obtain the local datasets of the active party and multiple passive parties, as well as the local model obtained through vertical federated learning, and obtain the forgetting requests of the passive parties; among them, the local model adopts the underlying model checkpoint strategy, and the forgetting requests include sample-level, feature-level and client-level forgetting requests.

[0012] S2. Delete the local target data of the passive party according to the forgetting request, and initialize the local models of the active party and multiple passive parties.

[0013] S3. Based on the convergence index, the RAdam stage or SGDM stage is used to restore the performance of the initialized local model, and the forgotten local model is obtained.

[0014] Optionally, the underlying model checkpointing strategy in S1 includes:

[0015] In the longitudinal federated learning process, intermediate activations, optimizer states, and training logs of the local model are deleted, while key gradient-related parameters are retained. This is done to reduce communication overhead and shorten the local model recovery time.

[0016] Optionally, the deletion of the passive party's local target data based on the forgetting request in S2 includes:

[0017] When the forget request is a sample-level forget request, the passive party uses the sample alignment result of vertical federated learning to assign a unique identifier to each sample in the local dataset, and the active party locates the corresponding sample based on the identifier and clears the feature value of the sample to zero.

[0018] When the forget request is a feature-level forget request, the passive party determines the feature dimension of the target data to be forgotten and clears the values ​​of the samples in the local dataset in the feature dimension to zero.

[0019] When the forget request is a client-level forget request, identify all feature dimensions held in the passive party's local dataset and perform feature-level forgetting on each feature dimension one by one.

[0020] Optionally, the local model for the active party and multiple passive parties in S2 is initialized, including:

[0021] Based on the forgetting request, multiple passive parties are divided into those involved in forgetting data and those not involved in forgetting data; for active parties and those involved in forgetting data, the parameters of the local model are initialized based on the minimum set of checkpoints maintained by the underlying model checkpoint strategy; for those not involved in forgetting data, residual knowledge related to forgotten data is cleared to initialize the parameters of the local model.

[0022] Optionally, in S3, the performance of the initialized local model is restored using either the RAdam or SGDM stage based on the convergence index, including:

[0023] Determine if the convergence index is greater than the preset relative convergence threshold; if it is, the active party loads the underlying model checkpoints of the passive party for each passive party and enters the RAdam stage; if it is not greater, enter the SGDM stage.

[0024] In the RAdam stage, a modified adaptive moment estimator optimizer is used, and performance recovery training is carried out based on the checkpoints of the underlying model as the starting point, without triggering cross-square gradient communication.

[0025] The SGDM stage employs a momentum stochastic gradient descent optimizer, enabling full backpropagation and cross-sigma gradient.

[0026] Optionally, convergence exponents include: accuracy, gradient, and learning rate.

[0027] The calculation process of the convergence index includes:

[0028] Calculate the ratio of the learning rate to the gradient, calculate the rate of change of accuracy, and obtain the convergence index based on the ratio and the rate of change of accuracy, as shown in equations (1)-(3) below:

[0029] (1)

[0030] (2)

[0031] (3)

[0032] In the formula, This represents the ratio of the learning rate to the gradient. Indicates the first The gradient of the wheel, Indicates the first Model parameters in round iteration, Indicates the first Learning rate of the round, Indicates the rate of change in accuracy. Indicates the first The accuracy of the global model. This represents the convergence index.

[0033] On the other hand, a lightweight adaptive optimizer scheduling longitudinal federated model forgetting device is provided, which is applied to the lightweight adaptive optimizer scheduling longitudinal federated model forgetting method, and the device includes:

[0034] The data acquisition module is used to acquire local datasets of the active party and multiple passive parties, as well as local models obtained through vertical federated learning, and to acquire forgetting requests from the passive parties. The local models adopt a low-level model checkpoint strategy, and forgetting requests include sample-level, feature-level, and client-level forgetting requests.

[0035] The data deletion module is used to delete the local target data of the passive party based on the forgetting request, and to initialize the local models of the active party and multiple passive parties.

[0036] The model performance recovery module is used to recover the performance of the initialized local model by using either the RAdam stage or the SGDM stage based on the convergence index, thus obtaining the forgotten local model.

[0037] Optionally, the data acquisition module is further used for:

[0038] In the longitudinal federated learning process, intermediate activations, optimizer states, and training logs of the local model are deleted, while key gradient-related parameters are retained. This is done to reduce communication overhead and shorten the local model recovery time.

[0039] Optionally, the data deletion module is further used for:

[0040] When the forget request is a sample-level forget request, the passive party uses the sample alignment result of vertical federated learning to assign a unique identifier to each sample in the local dataset, and the active party locates the corresponding sample based on the identifier and clears the feature value of the sample to zero.

[0041] When the forget request is a feature-level forget request, the passive party determines the feature dimension of the target data to be forgotten and clears the values ​​of the samples in the local dataset in the feature dimension to zero.

[0042] When the forget request is a client-level forget request, identify all feature dimensions held in the passive party's local dataset and perform feature-level forgetting on each feature dimension one by one.

[0043] Optionally, the data deletion module is further used for:

[0044] Based on the forgetting request, multiple passive parties are divided into those involved in forgetting data and those not involved in forgetting data; for active parties and those involved in forgetting data, the parameters of the local model are initialized based on the minimum set of checkpoints maintained by the underlying model checkpoint strategy; for those not involved in forgetting data, residual knowledge related to forgotten data is cleared to initialize the parameters of the local model.

[0045] Optionally, the model performance recovery module is further used for:

[0046] Determine if the convergence index is greater than the preset relative convergence threshold; if it is, the active party loads the underlying model checkpoints of the passive party for each passive party and enters the RAdam stage; if it is not greater, enter the SGDM stage.

[0047] In the RAdam stage, a modified adaptive moment estimator optimizer is used, and performance recovery training is carried out based on the checkpoints of the underlying model as the starting point, without triggering cross-square gradient communication.

[0048] The SGDM stage employs a momentum stochastic gradient descent optimizer, enabling full backpropagation and cross-sigma gradient.

[0049] Optionally, convergence exponents include: accuracy, gradient, and learning rate.

[0050] The calculation process of the convergence index includes:

[0051] Calculate the ratio of the learning rate to the gradient, calculate the rate of change of accuracy, and obtain the convergence index based on the ratio and the rate of change of accuracy, as shown in equations (1)-(3) below:

[0052] (1)

[0053] (2)

[0054] (3)

[0055] In the formula, This represents the ratio of the learning rate to the gradient. Indicates the first The gradient of the wheel, Indicates the first Model parameters in round iteration, Indicates the first Learning rate of the round, Indicates the rate of change in accuracy. Indicates the first The accuracy of the global model. This represents the convergence index.

[0056] On the other hand, a longitudinal federated model forgetting device is provided, the longitudinal federated model forgetting device comprising: a processor; a memory storing computer-readable instructions, which, when executed by the processor, implement any of the longitudinal federated model forgetting methods such as the lightweight adaptive optimizer scheduling described above.

[0057] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the methods in the above-described longitudinal federated model forgetting method for lightweight adaptive optimizer scheduling.

[0058] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0059] This invention proposes a lightweight longitudinal federated model forgetting framework LVFUS, which is compatible with any model architecture and can uniformly support sample-level, feature-level, and client-level forgetting with extremely low resource overhead, while maintaining high model accuracy after forgetting.

[0060] This paper proposes an adaptive optimizer switching strategy and improves the bottom model checkpoint strategy. By dynamically selecting the optimizer and normalizing the management of bottom model checkpoints, this invention accelerates convergence while reducing storage overhead, achieving efficient model recovery.

[0061] Through theoretical analysis and empirical verification, this invention has significant advantages over existing technologies: accuracy is improved by up to 15.00% after forgetting; recovery speed is improved by 1.08 times to 4.68 times; and storage overhead is controlled at the constant level through a lightweight checkpoint mechanism. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a flowchart of a lightweight adaptive optimizer scheduling forgetting method for a longitudinal federated model provided in an embodiment of the present invention;

[0064] Figure 2 This is an overall flowchart of a forgetting method for a longitudinal federated model with a lightweight adaptive optimizer scheduling provided in an embodiment of the present invention;

[0065] Figure 3 This is a block diagram of a lightweight adaptive optimizer scheduling vertical federated model forgetting device provided in an embodiment of the present invention;

[0066] Figure 4 This is a schematic diagram of the structure of a vertical federated model forgetting device provided in an embodiment of the present invention. Detailed Implementation

[0067] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0068] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0069] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0070] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0071] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0072] This invention provides a lightweight adaptive optimizer scheduling method for forgetting a vertical federated model. This method can be implemented using a vertical federated model forgetting device, which can be a terminal or a server. Figure 1 The flowchart shown is for a forgetting method in a longitudinal federated model using a lightweight adaptive optimizer. The processing flow of this method may include the following steps:

[0073] S1. Obtain the local datasets of the active party and multiple passive parties, as well as the local model obtained through vertical federated learning, and obtain the forgetting requests of the passive parties.

[0074] The local model employs a low-level model checkpointing strategy, and forgetting requests include sample-level, feature-level, and client-level forgetting requests.

[0075] Optionally, the underlying model checkpointing strategy in S1 includes:

[0076] In the longitudinal federated learning process, intermediate activations, optimizer states, and training logs of the local model are deleted, while key gradient-related parameters are retained. This is done to reduce communication overhead and shorten the local model recovery time.

[0077] In one feasible implementation, the present invention proposes a bottom-model checkpoint strategy, which can alleviate the space complexity problem.

[0078] Specifically, to minimize the storage overhead caused by additional model states or intermediate variables, this strategy does not retain intermediate activations, optimizer states, and training logs, but only retains the periodic bottom model checkpoints necessary for model recovery. By reducing the parameters to a small and predictable set of key states, the storage requirements do not increase with the size of the dataset; at the same time, only a small amount of communication is required to reload checkpoints during the recovery process.

[0079] Furthermore, the formal description of the bottom model checkpoint strategy, and the corresponding update rules, include: Let... This is the current iteration step in the model performance recovery phase. Let this be the number of passive parties. Indicates the first Passive side during turn The current base model parameters, This indicates the checkpoints that have been saved. The core idea of ​​the bottom model checkpoint strategy is to significantly reduce recovery time by storing historical model parameters, with a small amount of additional communication overhead. In this invention, the number of saved bottom model checkpoints and their generation rhythm are considered adjustable hyperparameters. In the first round of performance recovery and in subsequent specified rounds, the active party records the bottom model checkpoint parameters of each passive party. ,get In each round during the RAdam phase, each passive player... All from its saved checkpoints Instead of starting from scratch, retrain the model.

[0080] S2. Delete the local target data of the passive party according to the forgetting request, and initialize the local models of the active party and multiple passive parties.

[0081] Optionally, the deletion of the passive party's local target data based on the forgetting request in S2 includes:

[0082] When the forget request is a sample-level forget request, the passive party uses the sample alignment result of vertical federated learning to assign a unique identifier to each sample in the local dataset, and the active party locates the corresponding sample based on the identifier and clears the feature value of the sample to zero.

[0083] When the forget request is a feature-level forget request, the passive party determines the feature dimension of the target data to be forgotten and clears the values ​​of the samples in the local dataset in the feature dimension to zero.

[0084] When the forget request is a client-level forget request, identify all feature dimensions held in the passive party's local dataset and perform feature-level forgetting on each feature dimension one by one.

[0085] In one feasible implementation, during the data deletion phase, this invention assumes that only the passive party that initiates the forgetting request has direct access to the data to be deleted. Three types of forgetting requests are handled separately: sample-level forgetting requires a sample identifier; the passive party locates the corresponding record locally based on the sample alignment relationship formed during the longitudinal federated learning phase and deletes it from the dataset; feature-level forgetting involves the passive party deleting or masking the specified feature dimension according to the request; and client-level forgetting involves the passive party being removed from the federated collaboration and ceasing participation in subsequent training. After the above operations are completed, all passive parties and active parties jointly enter the model initialization phase.

[0086] Specifically, sample-level forgetting: Each sample is assigned a unique identifier using the sample alignment results of longitudinal federated learning. Upon receiving a sample-level forgetting request, the active party locates all passive parties containing the features of that sample and clears the feature values ​​corresponding to that sample to zero.

[0087] Feature-level forgetting: Due to the vertical federated architecture, each feature is stored only in the local dataset of one of the passive parties. Upon receiving a feature-level forgetting request, the corresponding passive party determines the nth feature dimension of the target and clears the values ​​of all local training samples in that dimension to zero, thereby eliminating its influence.

[0088] Client-level forgetting: Client-level forgetting can be viewed as a combination of multiple feature-level forgettings of all feature dimensions of the client. When a client-level forgetting request occurs, all feature dimensions held in the passive party's local dataset are first identified, and then processed one by one in the manner of feature-level forgetting.

[0089] Optionally, the local model for the active party and multiple passive parties in S2 is initialized, including:

[0090] Based on the forgetting request, multiple passive parties are divided into those involved in forgetting data and those not involved in forgetting data; for active parties and those involved in forgetting data, the parameters of the local model are initialized based on the minimum set of checkpoints maintained by the underlying model checkpoint strategy; for those not involved in forgetting data, residual knowledge related to forgotten data is cleared to initialize the parameters of the local model.

[0091] In one feasible implementation, during the model initialization phase, passive parties are first categorized into two groups based on their forgetting requests: those involved in forgetting data and those not involved. For active parties and all passive parties involved in forgetting data, the model parameters are reinitialized using the minimum set of checkpoints maintained by the bottom-model checkpoint strategy. This achieves a compact hot start without saving redundant intermediate states and logs, ensuring the consistency and traceability of the initialization process. For passive parties not involved in forgetting data, residual knowledge related to forgotten data needs to be removed to eliminate potential unintended retention and avoid interference with subsequent collaborative training.

[0092] S3. Based on the convergence index, the RAdam stage or SGDM stage is used to restore the performance of the initialized local model, and the forgotten local model is obtained.

[0093] Optionally, in S3, the performance of the initialized local model is restored using either the RAdam or SGDM stage based on the convergence index, including:

[0094] Determine if the convergence index is greater than the preset relative convergence threshold; if it is, the active party loads the underlying model checkpoints of the passive party for each passive party and enters the RAdam stage; if it is not greater, enter the SGDM stage.

[0095] In the RAdam stage, a modified adaptive moment estimator optimizer is used, and performance recovery training is carried out based on the checkpoints of the underlying model as the starting point, without triggering cross-square gradient communication.

[0096] The SGDM stage employs a momentum stochastic gradient descent optimizer, enabling full backpropagation and cross-sigma gradient.

[0097] In one feasible implementation, during the model performance recovery phase, this invention divides the optimizer into an RAdam phase and an SGDM phase, and dynamically switches between them based on the convergence index until a relative convergence threshold is met or the resource budget limit is reached. First, in the RAdam phase, parameters are loaded from the bottom model checkpoint, and backpropagation and parameter updates are performed locally without triggering cross-division gradient communication, achieving fast convergence and providing a warm start with minimal overhead. When convergence improvement slows or global consistency correction is needed, the system switches to the SGDM phase, initiating full backpropagation and cross-division gradient synchronization to globally refine each bottom model to improve stability and generalization. If the marginal benefit of global improvement weakens or communication cost constraints increase, the system switches back to the RAdam phase to continue. This process can repeat multiple times, ultimately terminating when the convergence condition is met.

[0098] Furthermore, the adaptive optimizer switching strategy proposed in this invention reduces computational overhead by dividing the recovery process into RAdam and SGDM stages. This strategy alleviates computational overhead in three ways: (i) it uses pre-saved checkpoints as the starting point for recovery training, shortening the time to reach the feasible region; (ii) it employs a simple switching rule, initially operating in the RAdam stage for rapid advancement, switching to the SGDM stage only when fine-tuning is needed, thus concentrating complete backpropagation and cross-gradient synchronization primarily in later rounds; and (iii) the early iterations do not update the entire base model, thereby reducing computationally expensive iterations. Therefore, while maintaining accuracy, it reduces both the total number of optimization steps and the computational load per round.

[0099] Furthermore, inspired by the dynamic learning rate adjustment mechanism of the RAdam optimizer, this invention proposes an optimizer switching metric. This metric integrates model accuracy, gradient norm, and learning rate trends to adaptively select a suitable optimizer in each round. Using this metric, this invention significantly reduces storage overhead and shortens recovery time while maintaining model accuracy.

[0100] Specifically, since the learning rate determines the update magnitude and the gradient reflects the direction and steepness of the descent, this invention uses a ratio to quantify the interaction between the two:

[0101] (1)

[0102] in For the first The gradient of the wheel, Indicates the first Model parameters in round iteration, This represents the learning rate for that round. To characterize the model's learning progress, this invention introduces the rate of change of accuracy between adjacent rounds:

[0103] (2)

[0104] As an intuitive measure of iterative effectiveness, considering the above factors, the convergence index is defined as:

[0105] (3)

[0106] In the formula, This represents the ratio of the learning rate to the gradient. Indicates the rate of change in accuracy. Indicates the first The accuracy of the global model. This represents the convergence index.

[0107] Convergence indices are used to comprehensively evaluate convergence behavior. When If the model has not yet converged, the next round enters the RAdam phase to achieve faster convergence; before entering this phase, the active party loads its base model checkpoint for each passive party, i.e., lets This allows training to continue from the most recently saved state instead of starting from scratch. Accordingly, the SGDM phase is used to improve generalization ability, and backpropagation is performed only in this phase to update the model parameters.

[0108] like Figure 2As shown, this invention proposes a lightweight model forgetting framework LVFUS for vertical federated learning. This framework is suitable for scenarios with multiple participants and features distributed across institutions. It is compatible with general neural network architectures and uniformly supports multi-granularity forgetting at the sample, feature, and client levels, without sharing the original data. It employs a collaborative mechanism of "low-level model checkpoint management + adaptive optimizer switching" to achieve fast and stable recovery with low storage and low communication.

[0109] The lightweight vertical federated model forgetting framework proposed in this invention consists of three steps: Step 1: Data deletion. The passive party initiating the forgetting process performs sample, feature, or client-level deletion locally upon request and completes consistency registration. Step 2: Model initialization. Relevant parties reset parameters from the bottom model checkpoint, while uninvolved parties clear residual associations with forgotten data. Step 3: Performance recovery. Based on an improved bottom model checkpoint and adaptive optimizer switching, the framework dynamically switches between the RAdam and SGDM stages according to the convergence exponent: the RAdam stage achieves rapid convergence based on local checkpoints without triggering cross-party gradient communication; the SGDM stage enables full backpropagation and cross-party gradient synchronization for global refinement, reducing backpropagation and cross-party communication, and stops when the relative convergence threshold and resource budget are reached. The former proceeds with a local hot start, while the latter enables backpropagation and cross-party gradient synchronization until relative convergence is achieved and the global model is updated. Only the data deletion step needs adjustment to flexibly adapt to sample-level, feature-level, and client-level forgetting requests. This design achieves accurate forgetting and efficient recovery under strict resource constraints, making the model accuracy after forgetting close to retraining, while maintaining constant-level storage and low communication overhead.

[0110] To reduce space complexity, an improved bottom-model checkpointing strategy is implemented. Key gradient-related parameters are selectively saved during training, omitting intermediate activations, optimizer states, and training logs. Checkpoint-based hot-start ensures consistent recovery and constant-level storage. To reduce computational complexity, an adaptive optimizer switching strategy is proposed. A convergence index composed of accuracy gain, gradient norm, and learning rate trend is constructed and dynamically alternates between the SGDM and RAdam stages based on this index. Leveraging the strong generalization of the SGDM optimizer and the fast convergence of the RAdam optimizer, performance recovery is accelerated with fewer iterations. Communication gating is implemented, enabling full backpropagation and cross-lateral gradient synchronization only during the SGDM stage. The RAdam stage operates based on local checkpoints, eliminating the need for cross-lateral gradient communication, thus reducing computational and communication costs. A unified handling and role division for three types of forgetting are established: the passive party initiating forgetting performs deletion and consistency registration locally, while uninvolved parties perform residual clearing and differentiated initialization, ensuring the effectiveness of deletion and the stability of subsequent training. Through this collaborative design, even under strict resource constraints, model performance can be restored to near-original levels after forgetting.

[0111] This invention can be widely applied to data governance scenarios such as medical compliance deletion, financial risk control compliance governance, and recommendation and advertising system compliance filtering. It meets the requirements of the EU General Data Protection Regulation and the California Consumer Privacy Act, as well as the Personal Information Protection Law, Data Security Law, and Cybersecurity Law of the People's Republic of China. It is also compatible with supporting systems such as the Several Provisions on the Management of Automotive Data Security, the Standard Contract Measures for the Export of Personal Information, and the Measures for Security Assessment of the Export of Important Data.

[0112] Specifically, in the medical compliance deletion scenario (corresponding to sample-level forgetting), the input data is the trained VFL (Vertical federated learning) diagnostic model and the sample identifier corresponding to the patient who made the forgetting request, and the output is the VFL diagnostic model after forgetting.

[0113] In the financial risk control and compliance governance scenario (corresponding to feature-level forgetting), the input data is the trained VFL risk control model and the feature identifiers corresponding to the data to be deleted, and the output is a compliant new risk control model.

[0114] In the compliance filtering scenario of recommendation and advertising systems (corresponding to client-level forgetting), the input data is the trained VFL recommendation model and all feature identifiers of the partners to be withdrawn, and the output is an independent recommendation and advertising platform model.

[0115] In this embodiment of the invention, a lightweight longitudinal federated model forgetting framework LVFUS is proposed. It is compatible with any model architecture, can uniformly support sample-level, feature-level and client-level forgetting with extremely low resource overhead, and maintains high model accuracy after forgetting.

[0116] This paper proposes an adaptive optimizer switching strategy and improves the bottom model checkpoint strategy. By dynamically selecting the optimizer and normalizing the management of bottom model checkpoints, this invention accelerates convergence while reducing storage overhead, achieving efficient model recovery.

[0117] Through theoretical analysis and empirical verification, this invention has significant advantages over existing technologies: accuracy is improved by up to 15.00% after forgetting; recovery speed is improved by 1.08 times to 4.68 times; and storage overhead is controlled at the constant level through a lightweight checkpoint mechanism.

[0118] Figure 3 This is a block diagram illustrating a lightweight adaptive optimizer scheduling longitudinal federated model forgetting apparatus according to an exemplary embodiment. The apparatus is used in a lightweight adaptive optimizer scheduling longitudinal federated model forgetting method. (Refer to...) Figure 3 The device includes a data acquisition module 310, a data deletion module 320, and a model performance recovery module 330. Wherein:

[0119] The data acquisition module 310 is used to acquire the local datasets of the active party and multiple passive parties, as well as the local model obtained through vertical federated learning, and to acquire the forgetting requests of the passive parties. The local model adopts the underlying model checkpoint strategy, and the forgetting requests include sample-level, feature-level and client-level forgetting requests.

[0120] The data deletion module 320 is used to delete the local target data of the passive party according to the forgetting request, and to initialize the local model of the active party and multiple passive parties.

[0121] The model performance recovery module 330 is used to perform performance recovery on the initialized local model based on the convergence index using either the RAdam stage or the SGDM stage, to obtain the forgotten local model.

[0122] In this embodiment of the invention, a lightweight longitudinal federated model forgetting framework LVFUS is proposed. It is compatible with any model architecture, can uniformly support sample-level, feature-level and client-level forgetting with extremely low resource overhead, and maintains high model accuracy after forgetting.

[0123] This paper proposes an adaptive optimizer switching strategy and improves the bottom model checkpoint strategy. By dynamically selecting the optimizer and normalizing the management of bottom model checkpoints, this invention accelerates convergence while reducing storage overhead, achieving efficient model recovery.

[0124] Through theoretical analysis and empirical verification, this invention has significant advantages over existing technologies: accuracy is improved by up to 15.00% after forgetting; recovery speed is improved by 1.08 times to 4.68 times; and storage overhead is controlled at the constant level through a lightweight checkpoint mechanism.

[0125] Figure 4 This is a schematic diagram of the structure of a vertical federated model forgetting device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the forgetting device in the longitudinal federated model can include the above-mentioned Figure 3 The illustrated lightweight adaptive optimizer-scheduled longitudinal federated model forgetting device. Optionally, the longitudinal federated model forgetting device 410 may include a first processor 2001.

[0126] Optionally, the longitudinal federated model forgetting device 410 may also include a memory 2002 and a transceiver 2003.

[0127] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0128] The following is combined with Figure 4 A detailed description of each component of the longitudinal federated model forgetting device 410 is provided below:

[0129] The first processor 2001 is the control center of the vertical federated model forgetting device 410. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0130] Optionally, the first processor 2001 can perform various functions of the longitudinal federated model forgetting device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0131] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0132] In a specific implementation, as one example, the vertical federated model forgetting device 410 may also include multiple processors, for example... Figure 4 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0133] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0134] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the vertical federated model forgetting device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0135] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0136] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0137] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be connected to the interface circuit of the vertical federated model forgetting device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0138] It should be noted that, Figure 4 The structure of the vertical federation model forgetting device 410 shown does not constitute a limitation on the router. Actual knowledge structure identification devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0139] Furthermore, the technical effect of the longitudinal federated model forgetting device 410 can be referred to the technical effect of the longitudinal federated model forgetting method with lightweight adaptive optimizer scheduling described in the above method embodiments, and will not be repeated here.

[0140] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0141] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0142] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0143] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0144] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0145] It should be understood that, in various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0146] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0147] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0148] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0149] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0150] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0151] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A forgetting method for a longitudinal federated model with lightweight adaptive optimizer scheduling, characterized in that, The method includes: S1. Obtain the local datasets of the active party and multiple passive parties, as well as the local model obtained through vertical federated learning, and obtain the forgetting requests of the passive parties; wherein, the local model adopts the underlying model checkpoint strategy, and the forgetting requests include sample-level, feature-level and client-level forgetting requests. S2. Delete the local target data of the passive party according to the forgetting request, and initialize the local models of the active party and multiple passive parties. S3. Based on the convergence index, the RAdam stage or SGDM stage is used to restore the performance of the initialized local model, and the forgotten local model is obtained.

2. The forgetting method for a longitudinal federated model with lightweight adaptive optimizer scheduling according to claim 1, characterized in that, The underlying model checkpointing strategy in S1 includes: In the longitudinal federated learning process, intermediate activations, optimizer states, and training logs of the local model are deleted, while key gradient-related parameters are retained. This is done to reduce communication overhead and shorten the local model recovery time.

3. The forgetting method for a longitudinal federated model with lightweight adaptive optimizer scheduling according to claim 1, characterized in that, The deletion of the passive party's local target data based on the forgetting request in S2 includes: When the forgetting request is a sample-level forgetting request, the passive party uses the sample alignment results of vertical federated learning to assign a unique identifier to each sample in the local dataset, and the active party locates the corresponding sample based on the identifier and clears the feature value of the sample to zero. When the forgetting request is a feature-level forgetting request, the passive party determines the feature dimension of the target data to be forgotten and clears the values ​​of the samples in the local dataset on the feature dimension to zero. When the forget request is a client-level forget request, identify all feature dimensions held in the passive party's local dataset and perform feature-level forgetting on each feature dimension one by one.

4. The forgetting method for a longitudinal federated model with lightweight adaptive optimizer scheduling according to claim 1, characterized in that, The initialization of the local model for the active party and multiple passive parties in S2 includes: Based on the forgetting request, multiple passive parties are divided into those involved in forgetting data and those not involved in forgetting data; for active parties and those involved in forgetting data, the parameters of the local model are initialized based on the minimum set of checkpoints maintained by the underlying model checkpoint strategy; for those not involved in forgetting data, residual knowledge related to forgotten data is cleared to initialize the parameters of the local model.

5. The forgetting method for a longitudinal federated model with lightweight adaptive optimizer scheduling according to claim 1, characterized in that, The performance restoration of the initialized local model in S3, based on the convergence index, using either the RAdam or SGDM stage, includes: Determine if the convergence index is greater than the preset relative convergence threshold; if it is, the active party loads the underlying model checkpoints of the passive party for each passive party and enters the RAdam stage; if it is not, enter the SGDM stage. In the RAdam stage, a modified adaptive moment estimator optimizer is used, and performance recovery training is carried out based on the checkpoints of the underlying model as the starting point, without triggering cross-square gradient communication. The SGDM stage employs a momentum stochastic gradient descent optimizer, enabling full backpropagation and cross-sigma gradient.

6. The forgetting method for a longitudinal federated model with lightweight adaptive optimizer scheduling according to claim 5, characterized in that, The convergence indices include: accuracy, gradient, and learning rate; The calculation process of the convergence index includes: Calculate the ratio of the learning rate to the gradient, calculate the rate of change of accuracy, and obtain the convergence index based on the ratio and the rate of change of accuracy, as shown in equations (1)-(3) below: （1） （2） （3） In the formula, This represents the ratio of the learning rate to the gradient. Indicates the first The gradient of the wheel, Indicates the first Model parameters in round iteration, Indicates the first Learning rate of the round, This indicates the rate of change in accuracy. Indicates the first The accuracy of the global model. This represents the convergence index.

7. A lightweight adaptive optimizer-scheduled vertical federated model forgetting device, wherein the lightweight adaptive optimizer-scheduled vertical federated model forgetting device is used to implement the lightweight adaptive optimizer-scheduled vertical federated model forgetting method as described in any one of claims 1-6, characterized in that, The device includes: The data acquisition module is used to acquire local datasets of the active party and multiple passive parties, as well as local models obtained through vertical federated learning, and to acquire forgetting requests from the passive parties. The local models adopt a low-level model checkpoint strategy, and forgetting requests include sample-level, feature-level, and client-level forgetting requests. The data deletion module is used to delete the local target data of the passive party based on the forgetting request, and to initialize the local models of the active party and multiple passive parties. The model performance recovery module is used to recover the performance of the initialized local model by using either the RAdam stage or the SGDM stage based on the convergence index, thus obtaining the forgotten local model.

8. The forgetting device for a longitudinal federated model with lightweight adaptive optimizer scheduling according to claim 7, characterized in that, The performance restoration of the initialized local model based on the convergence index using either the RAdam or SGDM stage includes: Determine if the convergence index is greater than the preset relative convergence threshold; if it is, the active party loads the underlying model checkpoints of the passive party for each passive party and enters the RAdam stage; if it is not, enter the SGDM stage. In the RAdam stage, a modified adaptive moment estimator optimizer is used, and performance recovery training is carried out based on the checkpoints of the underlying model as the starting point, without triggering cross-square gradient communication. The SGDM stage employs a momentum stochastic gradient descent optimizer, enabling full backpropagation and cross-sigma gradient.

9. A forgetting device for a vertical federated model, characterized in that, The longitudinal federation model forgetting device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 6.

Images