Federal forgetting model maintenance method based on double-flow update and federal learning system

Abstract

This application provides a federated forgetting model maintenance method and federated learning system based on dual-stream updates, belonging to the field of federated learning technology. The method is applied to a federated learning system comprising a central server and multiple clients. The method includes: first, the target client, based on a received sample deletion request, divides its local training sample set into a set of samples to be deleted and a set of samples to be retained; second, it applies the latest shared model parameters received from the central server to its local model; third, based on the set of samples to be deleted and the set of samples to be retained, it performs retention stream updates and forgetting stream updates on the local model to obtain target local model parameters, which are then sent to the central server; finally, the central server performs a single-round weighted aggregation based on the received target local model parameters to obtain target shared model parameters, which are then broadcast to multiple clients. This application can reduce retraining overhead while simultaneously mitigating the impact of deletion and maintaining the utility of retained data.

Description

Technical Field

[0001] This application relates to the field of federated learning technology, and in particular to a method for maintaining a federated forgetting model based on two-stream updates and a federated learning system. Background Technology

[0002] In existing technologies, when a target client receives a sample deletion request, the most direct method is to re-perform a complete federated retraining based on the retained sample set. While logically straightforward, this requires multiple rounds of local training and global aggregation, resulting in high time and computational costs, which is detrimental to the continuous maintenance of deployed systems. Furthermore, while a local repair approach based solely on the retained sample set can quickly restore some model utility, it typically lacks explicit constraints on the model's behavior on the sample set to be deleted. Therefore, it's difficult to directly determine whether the impact of the requested deletion samples has truly been suppressed. If applied to a high-class imbalance task like financial fraud detection, a poorly designed deletion and update process can easily lead to an increased false alarm rate, impacting alert budgets, increasing the burden of manual review, and reducing business operability. Summary of the Invention

[0003] The main objective of this application is to propose a federated forgetting model maintenance method and federated learning system based on dual-stream updates, which can reduce the retraining overhead while suppressing the effects of deletion and maintaining the utility of retained data.

[0004] To achieve the above objectives, one aspect of this application proposes a federated forgetting model maintenance method based on dual-stream updates, applied to a federated learning system. The federated learning system includes a central server and multiple clients, with the client receiving the sample deletion request designated as the target client. The method includes: The target client first divides the local training sample set into a sample set to be deleted and a sample set to be retained according to the received sample deletion request, and applies the latest shared model parameters issued by the central server to the local model. Then, according to the sample set to be deleted and the sample set to be retained, the local model is updated by the retention stream and the forgetting stream to obtain the target local model parameters and send them to the central server. The central server performs a single-round weighted aggregation based on the received target local model parameters to obtain target shared model parameters, which are then broadcast to the multiple clients.

[0005] Further, the step of performing retention stream update and forget stream update on the local model based on the set of samples to be deleted and the set of samples to be retained, to obtain the target local model parameters, includes: The local model is trained in multiple rounds of iteration. In each round of training, the set of samples to be deleted is first divided into multiple subsets of samples to be deleted. Then, based on the multiple subsets of samples to be deleted and the set of retained samples, the local model obtained from the previous round of training is updated sequentially using the retention stream and the forget stream. After completing multiple rounds of iterative training on the local model, the parameters of the target local model are obtained.

[0006] Further, the step of sequentially updating the local model obtained from the previous training round with the retention stream and the forgetting stream based on the plurality of subsets of samples to be deleted and the set of retained samples includes: For the current subset of samples to be deleted: the local model updated based on the previous subset of samples to be deleted and the retained sample set is denoted as the first local model. First, the first local model is trained multiple times based on the retained sample set. Then, the model retention flow loss is calculated based on the retained sample set, and the model forgetting flow loss is calculated based on the current subset of samples to be deleted. Subsequently, the parameters of the first local model obtained after multiple trainings are updated based on the model retention flow loss and the model forgetting flow loss.

[0007] Furthermore, the step of training the first local model multiple times based on the retained sample set includes: In each training process: the reserved sample set is sampled to obtain a first reserved sample subset, which contains multiple first sample data, each carrying a first category label; the multiple first sample data are respectively input into the first local model obtained from the previous training for analysis to obtain multiple corresponding first category prediction results; based on the multiple first category prediction results and the multiple first category labels carried by the multiple first sample data, the cross-entropy loss function is used to calculate the first model loss; based on the first model loss, the parameters of the first local model obtained from the previous training are updated.

[0008] Further, the calculation of the model retention flow loss based on the retained sample set includes: The reserved sample set is sampled to obtain a second reserved sample subset, which contains multiple second sample data, each of which carries a second category label; The multiple second sample data are respectively input into the first local model obtained through multiple trainings for analysis to obtain the corresponding multiple second category prediction results; Based on the multiple second-category prediction results and the multiple second-category labels carried by the multiple second-sample data, the model retention flow loss is calculated using the cross-entropy loss function.

[0009] Furthermore, the current subset of samples to be deleted contains multiple third-party sample data; the calculation of the model forgetting flow loss based on the current subset of samples to be deleted includes: The multiple third sample data are respectively input into the first local model obtained through multiple trainings for analysis to obtain the corresponding multiple third category prediction results; For each third sample data: filter out the prediction probability that matches the specified category label from the third category prediction results corresponding to the third sample data, and then calculate the deviation value between the prediction probability and the preset neutral probability value; The model forgetting flow loss is obtained by averaging the multiple deviation values ​​corresponding to the multiple third sample data.

[0010] Further, updating the parameters of the first local model obtained through multiple training iterations based on the model retention flow loss and the model forgetting flow loss includes: The model forgetting flow loss is weighted, and then the weighted model forgetting flow loss is added to the model retention flow loss to obtain the second model loss; Based on the second model loss, the parameters of the first local model obtained through multiple training iterations are updated.

[0011] Furthermore, clients that do not receive a sample deletion request are recorded as non-target clients; the process of obtaining target shared model parameters by performing a single-round weighted aggregation based on the received target local model parameters through the central server includes: The central server takes the latest shared model parameters sent to the target client as the local model parameters of the non-target client, and then combines them with the received target local model parameters to perform a weighted summation to obtain the target shared model parameters.

[0012] Furthermore, for each client: the local model applied by the client is a financial fraud detection model, which is constructed based on a multilayer perceptron network. The financial fraud detection model is used to analyze the financial risk feature data received by the client to obtain a detection result representing whether financial fraud has occurred.

[0013] To achieve the above objectives, another aspect of this application proposes a federated learning system, which includes a central server and multiple clients, wherein the client that receives the sample deletion request is designated as the target client. The target client is used to first divide the local training sample set into a sample set to be deleted and a sample set to be retained according to the received sample deletion request, and to apply the latest shared model parameters issued by the central server to the local model. Then, according to the sample set to be deleted and the sample set to be retained, the local model is updated by the retention stream and the forgetting stream to obtain the target local model parameters and send them to the central server. The central server is used to perform a single-round weighted aggregation based on the received target local model parameters to obtain target shared model parameters and broadcast them to the multiple clients.

[0014] This application includes at least the following beneficial effects: For a federated learning system comprising a central server and multiple clients, the client receiving the sample deletion request is designated as the target client. The target client first divides its local training sample set into a set of samples to be deleted and a set of samples to be retained based on the received sample deletion request. It then applies the latest shared model parameters issued by the central server to its local model. Next, based on the set of samples to be deleted and the set of samples to be retained, the local model undergoes retain-stream updates and forget-stream updates to obtain the target local model parameters, which are then sent to the central server. During this process, the retain-stream update mechanism effectively maintains the model's detection capability on the retained samples after deletion, while the forget-stream update mechanism effectively weakens the model's high-confidence dependence on the samples to be deleted. The central server performs a single-round weighted aggregation based on the received target local model parameters to obtain the target shared model parameters, which are then broadcast to multiple clients. In this entire implementation scheme, by completing dual-stream local maintenance on the target client and then performing a single-round weighted aggregation on the central server, the need for re-executing a complete multi-round federated retraining process is eliminated. This reduces the update time and computational cost of the model after sample deletion while simultaneously mitigating the impact of deletion and maintaining the utility of the retained data. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a federated forgetting model maintenance method based on dual-stream updates provided in an embodiment of this application. Figure 2 This is a schematic diagram of the composition of a federated learning system provided in an embodiment of this application. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of systems and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0017] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0018] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0020] With the rapid development of digital payments, cross-institutional transactions, and online financial services, financial fraud detection increasingly demands collaborative identification across institutions, platforms, and business chains. A single institution relying solely on local samples to train its detection model often struggles to promptly cover risk patterns arising from cross-institutional migration, group reuse, or geographical diffusion. However, user data such as transaction records, identity attributes, device fingerprints, and behavioral characteristics are typically highly sensitive and subject to governance constraints, making direct centralized sharing difficult. Against this backdrop, Federated Learning (FL) strategies, by proposing a method for collaborative model training without data leaving the domain, offer a feasible path for cross-entity joint modeling and are gradually becoming an important technological direction for intelligent financial risk control under privacy constraints. In financial fraud and anomaly detection scenarios, researchers have proposed applying federated learning strategies to credit card fraud detection, federated graph learning modeling, and financial anomaly detection combined with multi-party secure computation, demonstrating the practical significance of federated collaborative modeling in financial scenarios. However, most existing federated learning research focuses on training a stronger shared model, paying less attention to the handling of deletion requests after the model completes federated training and enters the deployment phase. For financial institutions, model training completion does not mean the end of data governance issues. With the increasing emphasis on data governance requirements such as personal information deletion, data retraction, and minimization, deletion requests are no longer just a data removal issue at the storage level, but further transform into an impact mitigation issue at the model level. This is because even if a transaction record or a user sample has been deleted from the local database, the influence of that sample on the model parameters during historical training may still remain in the deployed model. To address this issue, research on machine forgetting (MU) has emerged, with its core objective being to remove the persistent influence of specified data on the model as much as possible without completely retraining from scratch.

[0021] However, the difficulty increases significantly when the machine forgetting problem is applied to federated learning scenarios. This is because the impact of samples to be deleted is not limited to a single model but is incorporated into the shared model parameters through multiple rounds of local training and aggregation on a central server. To address this issue, researchers have proposed various federated forgetting approaches, such as rapid retraining for approximate recovery after deletion, historical information playback and replacement mechanisms, communication-efficient and provable federated forgetting, rapid deletion based on synthetic data, differential privacy restoration mechanisms, and verifiable federated forgetting. These studies have propelled federated forgetting from concept to algorithmic implementation. However, the main validation scenarios remain focused on general classification benchmarks, recommender systems, or general machine learning tasks. Coverage of scenarios with high class imbalance and strong business constraints, such as financial fraud detection, remains insufficient. If the model experiences excessive perturbation during deletion and update processes, the false positive rate may increase significantly, thereby increasing the burden on manual review and customer friction costs.

[0022] In summary, in existing technologies, when a target client receives a sample deletion request, the most direct method is to re-execute a complete federated retraining based on the retained sample set. While logically straightforward, this requires multiple rounds of local training and global aggregation, resulting in high time and computational costs, which is detrimental to the continuous maintenance of deployed systems. Furthermore, while a localized fast repair method based solely on the retained sample set can quickly restore some model utility, it typically lacks explicit constraints on the model's behavior on the sample set to be deleted. Therefore, it's difficult to directly determine whether the impact of the requested deletion samples on the model has been truly suppressed. Conversely, using a forgetting update method based solely on the sample set to be deleted is more likely to damage the overall model utility. Moreover, directly applying existing federated forgetting strategies to a high-class imbalance task like financial fraud detection, poorly designed deletion and update processes can easily lead to increased false alarm rates, impacting alert budgets, increasing the burden of manual review, and reducing business operability.

[0023] In view of this, this application provides a federated forgetting model maintenance method and federated learning system based on dual-stream updates. This scheme proposes a federated learning system containing a central server and multiple clients. The client receiving the sample deletion request is designated as the target client. The target client first divides its local training sample set into a set of samples to be deleted and a set of samples to be retained based on the received sample deletion request. It also applies the latest shared model parameters issued by the central server to its local model. Then, based on the set of samples to be deleted and the set of samples to be retained, the local model undergoes retention stream updates and forgetting stream updates to obtain the target local model parameters, which are then sent to the central server. During this process, the retention stream update mechanism effectively maintains the model's detection capability on the retained samples after deletion, while the forgetting stream update mechanism effectively reduces the model's high-confidence dependence on the samples to be deleted. The central server performs a single-round weighted aggregation based on the received target local model parameters to obtain the target shared model parameters, which are then broadcast to multiple clients. In the entire implementation plan, by completing dual-stream local maintenance on the target client and then performing a single round of weighted aggregation on the central server, there is no need to re-execute the full multi-round federated retraining process. This approach can reduce the model update time and computational cost after sample deletion while simultaneously mitigating the impact of deletion and maintaining the utility of the retained data.

[0024] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user identity information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0025] Please see Figure 1 , Figure 1 This is an optional flowchart illustrating a federated forgetting model maintenance method based on dual-stream updates provided in this application embodiment. The method is applied to a federated learning system, which includes a central server and multiple clients. Clients that receive sample deletion requests are designated as target clients, and clients that do not receive sample deletion requests are designated as non-target clients. The method may include, but is not limited to, the following steps S101 to S102: Step S101: The target client first divides the local training sample set into a sample set to be deleted and a sample set to be retained according to the received sample deletion request, and applies the latest shared model parameters issued by the central server to the local model. Then, according to the sample set to be deleted and the sample set to be retained, the local model is updated by the retention stream and the forget stream to obtain the target local model parameters and send them to the central server. Step S102: The central server performs a single-round weighted aggregation based on the received target local model parameters to obtain the target shared model parameters and broadcasts them to multiple clients.

[0026] Steps S101 to S102 shown in the embodiments of this application, by completing dual-stream local maintenance on the target client and then completing single-round weighted aggregation on the central server, can reduce the retraining overhead while taking into account the suppression of deletion effects and the maintenance of data utility.

[0027] In some embodiments, when a federated learning system is applied to a cross-institutional financial fraud detection scenario, for each client it includes, the local model applied to that client is a financial fraud detection model. This financial fraud detection model is built based on a multi-layer perceptron network (MLP), but other types of classifier structures can also be used. This financial fraud detection model is essentially a binary classification prediction model, mainly used to analyze the financial risk feature data received by the client to obtain a detection result that represents whether financial fraud has occurred.

[0028] The financial risk characteristic data may include at least one of the following: financial transaction data, identity characteristic data, device characteristic data, and risk control-related characteristic data. The explanations for each of these data types are as follows: Financial transaction data can be understood as records of fund flows and transaction behaviors generated during a user's financial activities, such as transaction amount, transaction time, counterparty, payment channel, transaction type, transaction frequency, IP address, and geographical location. This data can serve as a core basis for assessing risks such as fraud, money laundering, and cash-out schemes. Identity feature data can be understood as a set of information used to verify a user's true identity and attributes, such as the user's name, ID number, mobile phone number, bank card number, real-name authentication status, facial recognition results, and fingerprint authentication results. This data can be used to determine if there is identity theft, fraudulent registration, or malicious group activity. Device feature data can be understood as attribute information about the hardware devices, operating systems, and network environments used by the user during financial operations, such as device ID, device model, operating system version, whether VPN / proxy is enabled, and emulator detection indicators. This data can be used to determine if the device is genuine, whether a large number of accounts are operating on the same device, and whether the device environment is controlled by malicious software. Risk control-related feature data can be behavioral sequence feature data, such as the interval between a user's login and transaction completion and the degree of abnormality in click patterns.

[0029] Based on this, the local training sample set mentioned in step S101 contains several sample data. Each sample data may contain at least one of financial transaction data, identity feature data, device feature data, and risk control related feature data. Each sample data carries a corresponding category label and index. The category label is a label that represents the occurrence of financial fraud or a label that represents the absence of financial fraud.

[0030] In step S101 of some embodiments, the sample deletion request can be submitted directly by the user to the target client, or it can be submitted by the user to the central server first, and then forwarded to the target client by the central server. This application does not limit this. Regarding the division of the local training sample set into a set of samples to be deleted and a set of samples to be retained by the target client based on the received sample deletion request, the corresponding implementation may include, but is not limited to, the following: the sample deletion request at least contains the index of each sample data that the user requests to delete; the target client directly extracts all matching sample data from its local training sample set based on the index of each sample data contained in the sample deletion request and combines them to form the set of samples to be deleted; and then combines the remaining unmatched sample data to form the set of samples to be retained. Specifically, this can be implemented using the following mathematical expression: ; In the formula, Refers to the target client. Refers to the local training sample set within the target client. Refers to the set of retained samples within the target client. Refers to the set of samples to be deleted within the target client. The set difference operation is used to remove elements from the set on the left of the symbol from the set on the right.

[0031] In step S101 of some embodiments, regarding the process of updating the local model using a retention stream and a forgetting stream based on the set of samples to be deleted and the set of samples to be retained, to obtain the target local model parameters, the corresponding implementation may include, but is not limited to, performing multiple rounds of iterative training on the local model. In each round of training, the set of samples to be deleted is first divided into multiple subsets of samples to be deleted. This division can be done randomly. The number of all sample data contained in each subset of samples to be deleted can be the same or different. The number of all sample data contained in any two subsets of samples to be deleted can be partially the same or completely different. This application does not limit this. Then, based on the multiple subsets of samples to be deleted and the set of samples to be retained, the local model obtained from the previous round of training is sequentially updated using a retention stream and a forgetting stream, so that the model can gradually and orderly process each small batch of samples to be deleted in the set of samples to be deleted, thereby effectively suppressing the influence of the samples to be deleted. After completing multiple rounds of iterative training on the local model, the target local model parameters are obtained.

[0032] Understandably, if the current training round is the first round, the local model running with the latest shared model parameters issued by the central server will be recorded as the local model obtained from the previous round of training.

[0033] The section on sequentially updating the local model trained in the previous round with both retain flow and forget flow based on multiple subsets of samples to be deleted and the set of retained samples can be understood as follows: Let N be the number of subsets of samples to be deleted; based on the first subset of samples to be deleted and the set of retained samples, update the local model trained in the previous round with both retain flow and forget flow; then based on the second subset of samples to be deleted and the set of retained samples, update the local model after the first update with both retain flow and forget flow; then based on the third subset of samples to be deleted and the set of retained samples, update the local model after the second update with both retain flow and forget flow; and so on, until based on the Nth subset of samples to be deleted and the set of retained samples, update the local model after the N-1th update with both retain flow and forget flow.

[0034] Taking any one of the sample subsets to be deleted as an example, and defining it as the current sample subset to be deleted, the corresponding implementation of the dual-stream local maintenance process based on the current sample subset to be deleted and the retained sample set mentioned above may include, but is not limited to, the following steps S201 to S203: Step S201: The local model updated based on the previous subset of samples to be deleted and the set of retained samples is denoted as the first local model. The first local model is trained multiple times based on the set of retained samples. Step S202: Calculate the model retention flow loss based on the retained sample set, and calculate the model forgetting flow loss based on the current subset of samples to be deleted; Step S203: Update the parameters of the first local model obtained after multiple training iterations based on the model retention flow loss and the model forgetting flow loss.

[0035] By first performing multiple model parameter updates based solely on the retained sample set, the model can first repair the utility of the retained samples, avoiding premature impact from the forgetting flow that would cause an overall decrease in utility. Then, by performing model parameter updates jointly on the retained sample set and the sample set to be deleted, the model can simultaneously achieve both utility repair of the retained samples and suppression of the impact of the samples to be deleted.

[0036] It is understandable that if the current subset of samples to be deleted is the first subset of samples to be deleted, then the local model obtained from the previous round of training will be recorded as the first local model by default.

[0037] In step S201 of some embodiments, for each model training process based on the retained sample set, the corresponding implementation method may include, but is not limited to, the following: firstly, sampling the retained sample set to obtain a first retained sample subset, which can be implemented by random sampling according to a given first sample number. The first retained sample subset contains multiple first sample data, each carrying a first category label. Then, the multiple first sample data are respectively input into the first local model obtained from the previous training for analysis to obtain multiple corresponding first category prediction results. Next, based on the multiple first category prediction results corresponding to the multiple first sample data and the multiple first category labels carried by the multiple first sample data, the first model loss is calculated using the cross-entropy loss function. Finally, based on the first model loss, the parameters of the first local model obtained from the previous training are updated, which can be implemented using the existing gradient descent method.

[0038] The loss of the first model can be calculated using the following mathematical expression: ; In the formula, The loss of the first model. These are the parameters of the first local model obtained from the previous training, which may include model weights and model biases. Refers to the first reserved subset of samples. The first reserved subset contains the number of all first sample data. The first sample data contained in the first reserved sample subset. For the first sample data The first category label carried, For the first sample data The corresponding first-category prediction result, This represents the cross-entropy loss function. When a federated learning system is applied to cross-institutional financial fraud detection scenarios, the first-class prediction result output by the model can include the probability of predicting the occurrence of financial fraud and the probability of predicting the absence of financial fraud. The binary cross-entropy loss function is preferred. It should be noted that this mathematical expression can also be used to describe the optimization objective of the retainer update phase. Its core function is to allow the model to continue supervised learning on the retained samples, thereby maintaining the model's detection capability on the remaining samples after a deletion request arrives. This avoids a decline in overall utility caused by the model focusing only on the samples to be deleted, primarily achieving model repair.

[0039] Understandably, if the current training iteration based solely on the retained sample set is the first iteration, then by default, the local model updated based on the previous subset of samples to be deleted and the retained sample set will be recorded as the first local model obtained from the previous training.

[0040] In step S202 of some embodiments, the implementation of calculating the model retention flow loss based on the retained sample set can include, but is not limited to, the following: first, sampling the retained sample set to obtain a second retained sample subset, which can be achieved by random sampling according to a given number of second samples. The second retained sample subset contains multiple second sample data, each carrying a second category label. Then, the multiple second sample data are respectively input into a first local model that has been trained multiple times for analysis to obtain corresponding multiple second category prediction results. Finally, based on the multiple second category prediction results corresponding to the multiple second sample data and the multiple second category labels carried by the multiple second sample data, the model retention flow loss is calculated using the cross-entropy loss function. The calculation method for the model retention flow loss is the same as the calculation method for the first model loss described above, and will not be repeated here.

[0041] In step S202 of some embodiments, the current subset of samples to be deleted contains multiple third sample data. Regarding the calculation of the model forgetting flow loss based on the current subset of samples to be deleted, the corresponding implementation may include, but is not limited to, the following: first, inputting the multiple third sample data into a first local model that has been trained multiple times for analysis to obtain multiple corresponding third category prediction results; then, for each third sample data, selecting the prediction probability that matches the specified category label from the third category prediction results corresponding to the third sample data, where the specified category label can be understood as the model's target positive class label or attention class label, and calculating the deviation value between the prediction probability and the preset neutral probability value; finally, averaging the multiple deviation values ​​corresponding to the multiple third sample data to obtain the model forgetting flow loss.

[0042] The model forgetting flow loss can be calculated using the following mathematical expression: ; In the formula, For the model's forgetting flow loss, These are the parameters of the first local model obtained through multiple training iterations, which also include model weights and model biases. Refers to the current subset of samples to be deleted. This represents the number of all third-party sample data contained in the current subset of samples to be deleted. This refers to the third sample data contained in the current subset of samples to be deleted. For the third sample data The corresponding third-category prediction results include the predicted probabilities that match the specified category label. The preset neutral probability value, preferably set to 0.5, represents the neutral uncertainty output center in a binary classification task, meaning the model has no significant bias towards whether a sample belongs to the target class or not. This value measures the deviation between the predicted probability matching a specified category label and a preset neutral probability value. When the federated learning system is applied to cross-institutional financial fraud detection scenarios, the third-category prediction output of the model can include the probability of predicting financial fraud and the probability of predicting no financial fraud, where the specified category label is the label representing the occurrence of financial fraud. It should be noted that this mathematical expression, as the optimization objective describing the forgetting flow update stage, plays a core role in gradually pulling the output probability of the sample to be deleted in the current model towards the maximum uncertainty state by using neutral erasure, rather than simply forcibly pushing the sample to be deleted to the opposite category for retraining, thereby weakening the model's high confidence dependence on the sample to be deleted.

[0043] In step S203 of some embodiments, the content of updating the parameters of the first local model obtained through multiple trainings based on the model retention flow loss and the model forgetting flow loss can be implemented, but is not limited to, by: first weighting the model forgetting flow loss, then adding the weighted model forgetting flow loss to the model retention flow loss to obtain the second model loss; and then updating the parameters of the first local model obtained through multiple trainings based on the second model loss, which can also be achieved using the existing gradient descent method.

[0044] The loss of the second model can be calculated using the following mathematical expression: ; In the formula, For the second model loss, To preserve the flow loss in the model, Refers to the second reserved subset of samples. The preset balance parameter, preferably within the range of (0, 0.5), is used to adjust the weight of the forgetting stream loss in the second model loss. It should be noted that this mathematical expression unifies the retaining stream update and the forgetting stream update into a single local optimization objective. This can be understood as the core overall objective function for implementing the dual-stream update mechanism. Its purpose is to simultaneously maintain the model utility on retained samples and reduce the impact of deleted samples on the model, ensuring that model maintenance after sample deletion no longer relies solely on a single local repair strategy based on retained data.

[0045] In step S102 of some embodiments, regarding the process of obtaining target shared model parameters by performing a single-round weighted aggregation based on the received target local model parameters by the central server, the corresponding implementation may include, but is not limited to, the following: Since the central server sends the latest shared model parameters to the target client and also to non-target clients before executing step S101, the central server directly uses the latest shared model parameters sent to the target client as the local model parameters of the non-target clients. In this case, the non-target clients are kept in a frozen state during the execution of steps S101 to S102. The target shared model parameters are then obtained by weighted summation of the received target local model parameters sent by the target client. Specifically, this can be implemented using the following first mathematical expression: ; In the formula, To share model parameters for the target, This refers to the total number of all clients in the federated learning system that participated in this round of aggregation. The weight corresponding to the k-th client during aggregation can be set by technical personnel based on the size of the local training sample set used by the k-th client or a preset aggregation rule. Let be the local model parameters for the k-th client in this round of aggregation. If the k-th client is the target client, then... The target local model parameters are sent to the target client. It should be noted that the core function of the first mathematical expression above is to directly utilize the update results from the target client and the current model state of the non-target client to form a new shared model in one go, instead of re-executing a complete multi-round federated retraining process. This reduces the ongoing maintenance cost of the model after sample deletion.

[0046] As an optional implementation, regarding the process of obtaining target shared model parameters by performing a single-round weighted aggregation of the received target local model parameters through a central server, the corresponding implementation methods may include, but are not limited to: Since the central server sends the latest shared model parameters to both the target client and non-target clients before executing step S101, the non-target clients, in an unfrozen state, apply the latest shared model parameters received from the central server to their local models, and then update the parameters of their local models based on their local training sample sets to obtain the latest local model parameters, which are then sent to the central server. During this process, the cross-entropy loss function can be used to calculate the model loss, and the existing gradient descent method can be used to assist in completing the model parameter update. Alternatively, the central server can perform a weighted summation of the target local model parameters sent by the target client and the latest local model parameters sent by the non-target clients to obtain the target shared model parameters, similarly using the first mathematical expression described above, which will not be elaborated further here. In this implementation, by setting both the target client and non-target clients to participate in the current round of shared model aggregation task of the central server, the overall stability of the updated shared model can be maintained.

[0047] As an optimized implementation method, after determining the target shared model parameters, the central server can determine the deployment threshold for the shared model after sample deletion based on verification data or preset false alarm rate constraints. This ensures that the shared model after sample deletion meets the business availability requirements under strict operational points. Subsequently, the target shared model parameters after threshold constraints are broadcast to multiple clients, enabling each client to normally execute subsequent model detection tasks or the next round of model maintenance tasks.

[0048] In some embodiments, when no client receives a sample deletion request, all clients collaborate with the central server to train a shared model based on their respective local training sample sets. The central server obtains and saves the shared model for completing the specified task through multiple rounds of federated aggregation, and broadcasts the shared model parameters to each client participating in federated learning in normal mode.

[0049] Compared to methods that only pursue the overall performance of the shared model during the model training phase, the federated forgetting model maintenance method based on dual-stream updates proposed in this application is mainly aimed at the post-deployment maintenance scenario. It can perform local updates and re-form a deployable shared model more quickly when a deletion request arrives, making it more suitable for cross-institutional financial fraud detection scenarios that are jointly affected by privacy governance, alert budgets, and continuous operation constraints.

[0050] Please see Figure 2 , Figure 2 This is an optional schematic diagram of a federated learning system provided in an embodiment of this application, which can implement the above-described federated forgetting model maintenance method based on dual-stream updates. The federated learning system includes a central server and multiple clients, and each client communicates with the central server. The client that receives a sample deletion request is recorded as the target client. The target client first divides its local training sample set into a set of samples to be deleted and a set of samples to be retained based on the received sample deletion request. It then applies the latest shared model parameters issued by the central server to its local model. Based on the set of samples to be deleted and the set of samples to be retained, it performs retention stream updates and forget stream updates on the local model to obtain the target local model parameters and sends them to the central server. The central server performs a single-round weighted aggregation based on the received target local model parameters to obtain the target shared model parameters and broadcasts them to multiple clients.

[0051] In some embodiments, when the federated learning system is applied to cross-institutional financial fraud detection scenarios, multiple clients may include bank branches, insurance companies, and securities institutions, while the central server may be a financial regulatory agency or a financial industry association.

[0052] In some embodiments, without departing from the core technical concept of this application, the federated learning system can also be applied to other classification scenarios with high class imbalance, such as malware / Trojan detection scenarios, etc., and this application does not limit it.

[0053] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those implemented in the above method embodiments, and the beneficial effects achieved by this system embodiment are also the same as those achieved by the above method embodiments.

[0054] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described federated forgetting model maintenance method based on dual-stream updates.

[0055] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented by this storage medium embodiment are the same as those implemented by the above method embodiments, and the beneficial effects achieved by this storage medium embodiment are also the same as those achieved by the above method embodiments.

[0056] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0057] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0058] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0059] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process or method comprising a series of steps is not necessarily limited to those explicitly listed, but may include other steps not explicitly listed or inherent to such processes and methods.

[0060] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0061] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for maintaining a federated forgetting model based on two-stream updates, characterized in that, The method is applied to a federated learning system, which includes a central server and multiple clients, and designates the client that receives the sample deletion request as the target client; the method includes: The target client first divides the local training sample set into a sample set to be deleted and a sample set to be retained according to the received sample deletion request, and applies the latest shared model parameters issued by the central server to the local model. Then, according to the sample set to be deleted and the sample set to be retained, the local model is updated by the retention stream and the forgetting stream to obtain the target local model parameters and send them to the central server. The central server performs a single-round weighted aggregation based on the received target local model parameters to obtain target shared model parameters, which are then broadcast to the multiple clients.

2. The federated forgetting model maintenance method based on dual-stream update according to claim 1, characterized in that, The step of updating the local model using the retention stream and the forgetting stream based on the set of samples to be deleted and the set of samples to be retained, to obtain the target local model parameters, includes: The local model is trained in multiple rounds of iteration. In each round of training, the set of samples to be deleted is first divided into multiple subsets of samples to be deleted. Then, based on the multiple subsets of samples to be deleted and the set of retained samples, the local model obtained from the previous round of training is updated sequentially using the retention stream and the forget stream. After completing multiple rounds of iterative training on the local model, the parameters of the target local model are obtained.

3. The federated forgetting model maintenance method based on dual-stream update according to claim 2, characterized in that, The step of sequentially updating the local model obtained from the previous training round with the retention stream and the forgetting stream based on the plurality of subsets of samples to be deleted and the set of retained samples includes: For the current subset of samples to be deleted: the local model updated based on the previous subset of samples to be deleted and the retained sample set is denoted as the first local model. First, the first local model is trained multiple times based on the retained sample set. Then, the model retention flow loss is calculated based on the retained sample set, and the model forgetting flow loss is calculated based on the current subset of samples to be deleted. Subsequently, the parameters of the first local model obtained after multiple trainings are updated based on the model retention flow loss and the model forgetting flow loss.

4. The federated forgetting model maintenance method based on dual-stream update according to claim 3, characterized in that, The step of training the first local model multiple times based on the retained sample set includes: In each training process: the reserved sample set is sampled to obtain a first reserved sample subset, which contains multiple first sample data, each carrying a first category label; the multiple first sample data are respectively input into the first local model obtained from the previous training for analysis to obtain multiple corresponding first category prediction results; based on the multiple first category prediction results and the multiple first category labels carried by the multiple first sample data, the cross-entropy loss function is used to calculate the first model loss; based on the first model loss, the parameters of the first local model obtained from the previous training are updated.

5. The federated forgetting model maintenance method based on dual-stream update according to claim 3, characterized in that, The calculation of the model retention flow loss based on the retained sample set includes: The reserved sample set is sampled to obtain a second reserved sample subset, which contains multiple second sample data, each of which carries a second category label; The multiple second sample data are respectively input into the first local model obtained through multiple trainings for analysis to obtain the corresponding multiple second category prediction results; Based on the multiple second-category prediction results and the multiple second-category labels carried by the multiple second-sample data, the model retention flow loss is calculated using the cross-entropy loss function.

6. The federated forgetting model maintenance method based on two-stream update according to claim 3, characterized in that, The current subset of samples to be deleted contains multiple third-party sample data; the calculation of the model forgetting flow loss based on the current subset of samples to be deleted includes: The multiple third sample data are respectively input into the first local model obtained through multiple trainings for analysis to obtain the corresponding multiple third category prediction results; For each third sample data: filter out the prediction probability that matches the specified category label from the third category prediction results corresponding to the third sample data, and then calculate the deviation value between the prediction probability and the preset neutral probability value; The model forgetting flow loss is obtained by averaging the multiple deviation values ​​corresponding to the multiple third sample data.

7. The federated forgetting model maintenance method based on dual-stream update according to claim 3, characterized in that, The step of updating the parameters of the first local model obtained through multiple training iterations based on the model retention flow loss and the model forgetting flow loss includes: The model forgetting flow loss is weighted, and then the weighted model forgetting flow loss is added to the model retention flow loss to obtain the second model loss; Based on the second model loss, the parameters of the first local model obtained through multiple training iterations are updated.

8. The federated forgetting model maintenance method based on two-stream update according to claim 1, characterized in that, Clients that do not receive a sample deletion request are considered non-target clients; The process of obtaining the target shared model parameters by performing a single-round weighted aggregation based on the received target local model parameters through the central server includes: The central server takes the latest shared model parameters sent to the target client as the local model parameters of the non-target client, and then performs a weighted summation on the received target local model parameters to obtain the target shared model parameters.

9. The federated forgetting model maintenance method based on two-stream update according to claim 1, characterized in that, For each client: the local model applied by the client is a financial fraud detection model, which is constructed based on a multilayer perceptron network. The financial fraud detection model is used to analyze the financial risk feature data received by the client to obtain a detection result representing whether financial fraud has occurred.

10. A federated learning system, characterized in that, The federated learning system includes a central server and multiple clients, and the client that receives the sample deletion request is recorded as the target client; The target client is used to first divide the local training sample set into a sample set to be deleted and a sample set to be retained according to the received sample deletion request, and to apply the latest shared model parameters issued by the central server to the local model. Then, according to the sample set to be deleted and the sample set to be retained, the local model is updated by the retention stream and the forgetting stream to obtain the target local model parameters and send them to the central server. The central server is used to perform a single-round weighted aggregation based on the received target local model parameters to obtain target shared model parameters and broadcast them to the multiple clients.

Images