A federal class incremental learning method for asynchronous new class exposure

Abstract

This invention provides a federated class incremental learning method for asynchronous new class exposure. The method executes each round of communication as follows: Step S1, the server distributes the global model and global prototype set obtained from the previous round of communication to all clients; Step S2, each client calculates its local model and local prototype set for the current round of communication and uploads them to the server; Step S3, the server divides the clients into new class clients and old class clients, and groups all new class clients to obtain multiple new class subsets. It calculates the old class group aggregation model and the new class group aggregation model for each new class subset in the current round of communication, calculates the contribution weight of the old class group aggregation model and the new class group aggregation model in the current round of communication, and aggregates them to obtain the global model for the current round of communication. It also updates the global prototype set of each image category newly added in the current class incremental task in the global prototype set of the previous round of communication to obtain the global prototype set for the current round of communication.

Description

Technical Field

[0001] This invention relates to the field of machine learning, specifically to federated learning and continuous learning techniques in machine learning, and more specifically, to a federated class incremental learning method for asynchronous new class exposure. Background Technology

[0002] With the widespread application of artificial intelligence technology in scenarios such as medical image analysis, industrial defect detection, and cross-institutional data modeling, data is often stored in different institutions or on different devices. Furthermore, it is constrained by factors such as privacy protection and data security, making centralized aggregation and unified training difficult. To address these issues, Federated Learning (FL) achieves multi-party collaborative modeling under the premise of "data not leaving the local machine." Each client trains its model based on local data, uploads model parameters or gradient information to the server, and the server then performs weighted aggregation of model updates from different clients.

[0003] In practical applications, data not only exhibits distributed characteristics, but its category information also evolves over time, meaning new categories may gradually emerge during system operation. To address this issue, incremental learning (IL) and class-incremental learning (CIL) methods have been proposed. Their core idea is to continuously expand the model's capabilities by learning new categories in stages while maintaining the ability to discriminate against older categories, without accessing all historical data. Building on this, researchers have further combined federated learning with class-incremental learning to form the Federated Class-Incremental Learning (FCIL) method. This allows the model to gradually adapt to the ever-expanding category space in a distributed environment, and improves model stability by introducing techniques such as sample replay, knowledge distillation, or parameter regularization.

[0004] However, most existing federated incremental learning methods are based on the assumption that new classes appear synchronously across clients. In real-world applications, new classes often appear only in a few clients initially, exhibiting a gradually expanding asynchronous exposure characteristic. In this case, among the clients participating in the same round of federated training, only some clients contain data on new classes, while the remaining clients still update their models based solely on old classes. Since existing federated aggregation methods typically employ a weighted strategy based on sample size, when most clients do not contain data on new classes, the model updates related to new classes have a low weight in the aggregation process, resulting in insufficient representation of new class information in the global model. The fundamental reason is that existing methods assume that all clients have the same task when designing the aggregation mechanism, failing to characterize the differentiated contributions of different clients in learning new classes, thus limiting the propagation efficiency of new classes in the federated system.

[0005] Furthermore, during the initial exposure phase of new classes, the limited number of new class samples available to each client leads to insufficient gradient estimation for new classes during local training, resulting in unstable update directions and high variance. Existing methods typically maintain the performance of old classes through knowledge distillation or regularization constraints. However, these constraints are often constructed based on the distribution of old classes, which can inhibit the learning of new classes during optimization, further weakening the discriminative ability of new classes. The fundamental reason is that existing methods primarily focus on preserving old classes on the client side, lacking specific stabilization designs for the limited sample size of new classes. This makes it difficult for the model to form stable and effective feature representations when learning new classes.

[0006] Furthermore, the aforementioned local learning instability and global aggregation bias are not independent of each other, but rather superimposed and work together during the federated incremental learning process. On the one hand, due to the scarcity of new class samples, the model updates related to the new class uploaded by the client are of low quality; on the other hand, during the aggregation phase, these already weak update signals are further diluted by the updates of most old classes, making it difficult to be effectively reflected in the global model. This coupling effect makes it difficult for the new class to be stably learned locally and to be effectively propagated globally, ultimately leading to slow performance improvement of the model on the new class, or even significant lag.

[0007] In summary, existing federated incremental learning methods have the following problems in practical applications: (1) They fail to explicitly distinguish the differences in contributions of different clients in the learning and propagation of new classes, which can easily lead to the new class update being overwhelmed by a large number of old class updates, thus limiting the propagation efficiency of the new class in the federated system; (2) Clients focus on the problem of maintaining old classes and lack stabilization design for the few-sample stage of new classes, which makes it difficult for the model to form a stable and effective representation when learning new classes, thus resulting in slow performance improvement of the model on new classes, or even significant lag.

[0008] It should be noted that the background information presented here is only for illustrating relevant information about the present invention to aid in understanding the technical solution of the present invention, and does not imply that the relevant information is necessarily prior art. The relevant information was submitted and disclosed together with the present invention, and should not be considered prior art unless there is evidence that the relevant information was disclosed before the filing date of the present invention. Summary of the Invention

[0009] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a federated class incremental learning method for asynchronous new class exposure.

[0010] The objective of this invention is achieved through the following technical solution:

[0011] According to a first aspect of the present invention, a federated class incremental learning method for asynchronous new class exposure is provided, which is used to perform multiple class incremental tasks in a federated system to obtain a target global model. The federated system includes a server and multiple clients. During each class incremental task, the server communicates with each client in multiple rounds. In each round of communication, some clients only include the old class dataset, while the remaining clients include the old class dataset and locally added new class datasets. The old class dataset includes multiple image categories, and the new class dataset includes one or more newly added image categories. Each image category contains multiple training samples. The method is to configure the federated system to perform each round of communication as follows: Step S1: The server distributes the global model and global prototype set obtained in the previous round of communication to all clients, wherein the global prototype set includes the global prototype of each image category that has appeared; Step S2: Each client calculates the local prototype of each image category according to a preset class prototype calculation rule, obtains the local prototype set of each client in this round of communication, and performs multiple local iterations on the global model obtained in the previous round of communication based on all training samples to obtain... In this round of communication, each client's local model and the local model and local prototype set of this round of communication are uploaded to the server. In each local iteration, the loss is calculated according to the preset loss function and the parameters of the local model obtained from the previous local iteration are updated. Step S3: The server divides the clients containing the newly added image categories into new class clients and the remaining clients into old class clients. According to the preset grouping strategy, all new class clients are grouped to obtain multiple new class subsets. The local models of all old class clients and the local models of all new class clients in each new class subset are weighted averaged and aggregated to obtain the old class group aggregation model and the new class group aggregation model of each new class subset of this round of communication. According to the preset contribution weighted aggregation rule, the contribution weight of the old class group aggregation model and the new class group aggregation model of this round of communication are calculated and aggregated to obtain the global model of this round of communication. Based on the local prototype set of each client in this round of communication, the global prototype of each image category added in this class increment task in the global prototype set of the previous round of communication is updated according to the preset class update rule to obtain the global prototype set of this round of communication.

[0012] According to some embodiments of the present invention, the preset class prototype calculation rule is as follows:

[0013]

[0014] in, Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype, Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The number of training samples included. Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The set consisting of all training samples. Indicates training samples, Represents the feature extraction function. Indicates the first The client received the first The sub-category of incremental tasks The global model obtained from round-robin communication.

[0015] According to some embodiments of the present invention, the preset loss function is:

[0016]

[0017] in,

[0018]

[0019]

[0020]

[0021] in,

[0022]

[0023] in, Indicates the first The sub-category of incremental tasks Round communication Total loss of each local iteration Indicates the first The sub-category of incremental tasks Round communication Cross-entropy loss in the next local iteration Indicates the first The sub-category of incremental tasks Round communication Distillation loss in the next local iteration Indicates the first The sub-category of incremental tasks Distillation loss weight in round-robin communication Indicates the first The sub-category of incremental tasks Round communication Prototype alignment loss in the next local iteration. Indicates the first The sub-category of incremental tasks Prototype alignment loss weights in round-robin communication Indicates the first The sub-category of incremental tasks The total number of training samples included in all new image categories added in round-robin communication. This represents the maximum storage limit for each image category. Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The set consisting of all training samples. Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The corresponding difference value, Indicates the first The sub-category of incremental tasks Image categories in round-robin communication global prototype Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype, It is a regulating factor.

[0024] According to some embodiments of the present invention, the preset grouping strategy is as follows: according to preset similarity calculation rules, calculate the similarity between the local model of each new type of client in this round of communication and the local model of other new type of clients in this round of communication to obtain the similarity matrix of new type of clients in this round of communication; perform clustering based on the similarity matrix of new type of clients in this round of communication to obtain multiple new type subsets, wherein each new type subset includes one or more new type of clients.

[0025] According to some embodiments of the present invention, the preset similarity calculation rule is as follows:

[0026]

[0027] in,

[0028]

[0029]

[0030] in, Indicates the first The sub-category of incremental tasks In round-robin communication, the first The local model of the first client and the first Similarity between local models of each client Represents the balance factor. Represents the cosine function. Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype, Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype, Indicates the first The sub-category of incremental tasks Round communication The update direction of the local model on each client. Indicates the first The sub-category of incremental tasks Round communication The update direction of the local model on each client; Indicates the first The sub-category of incremental tasks The global model obtained from round-robin communication.

[0031] According to some embodiments of the present invention, the preset contribution-weighted aggregation rule is as follows:

[0032]

[0033] in,

[0034]

[0035]

[0036]

[0037]

[0038] in,

[0039]

[0040]

[0041] in, Indicates the first The sub-category of incremental tasks The global model obtained through round-robin communication Indicates the first The sub-category of incremental tasks The old class group aggregation model of round communication, Represents the old class group aggregation model Contribution weight, Indicates the first The sub-category of incremental tasks Round communication A new class group aggregation model for a new subset of classes. Indicates the number of subsets of the new class. Represents a new class group aggregation model Contribution weight, Indicates the first The sub-category of incremental tasks In round-robin communication, the set of all legacy clients is composed of... Indicates the first The number of training samples on each client Indicates the first The number of training samples on each client Indicates the first The client in the first The sub-category of incremental tasks Local model of round-robin communication Represents the old class group aggregation model Contribution, normalized get , Represents a new class group aggregation model Contribution, normalized get , Indicates the first The sub-category of incremental tasks Old class group aggregation model in round communication The direction of updates; Indicates the first The sub-category of incremental tasks New class group aggregation model in round communication The direction of updates.

[0042] According to some embodiments of the present invention, the preset class update rule is as follows:

[0043]

[0044] in, Indicates the first The sub-category of incremental tasks Image categories in round-robin communication global prototype Indicates the first The sub-category of incremental tasks The set of all new client classes in round-robin communication. Indicates the first The number of training samples on each client Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The number of samples, Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype.

[0045] According to some embodiments of the present invention, the method further includes: in the last round of communication of each class increment task, each client determines the global prototype of each new image category in the current class increment task based on the global prototype set obtained in the current round of communication; based on the global prototype of each new image category in the current class increment task, each client calculates the distance between each training sample contained in each new image category and the global prototype of the image category, and sorts the calculated distances in ascending order to retain a preset number of training samples at the top of the sorting, and merges all the training samples retained for each new image category with the old class dataset corresponding to the current class increment task to obtain the old class dataset of each client in the next class increment task.

[0046] Compared with the prior art, the advantages of the present invention are: (1) On the client side, a prototype-constrained class incremental learning mechanism is designed. This mechanism maintains the old class discrimination boundary through prototype-guided playback, slows down the forgetting of existing classes by the local model through knowledge distillation, and enhances the feature aggregation ability of the new class under few sample conditions through prototype alignment, thereby generating a more stable and discriminative local model update to enhance the representation stability of the few sample new class learning stage; (2) On the server side, a client contribution-aware hierarchical aggregation mechanism is designed. This mechanism divides the participating clients by whether the client has a new class increment, and on this basis, it combines update similarity and group-level contribution to perform grouping and weighted aggregation, so as to reduce the risk that the updates of a few new class clients are diluted by the updates of most old class clients in the global aggregation process, and improve the effectiveness of the new class propagation to the global model. Attached Figure Description

[0047] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0048] Figure 1 A schematic diagram illustrating a scenario for exposing asynchronous new classes;

[0049] Figure 2 This is a schematic diagram of the incremental learning method for federated classes oriented towards asynchronous new class exposure according to an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of a research framework for incremental learning of federated classes oriented towards asynchronous new class exposure according to an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram illustrating the execution flow of each round of communication in a single incremental task according to an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0053] As mentioned in the background section, existing federated incremental learning methods have the following problems in practical applications: (1) They fail to explicitly distinguish the differences in contributions of different clients in the learning and propagation of new classes, which can easily lead to the new class update being overwhelmed by a large number of old class updates, thus limiting the propagation efficiency of the new class in the federated system; (2) Clients focus on the problem of maintaining old classes and lack stabilization design for the few-sample stage of new classes, which makes it difficult for the model to form a stable and effective representation when learning new classes, thus leading to slow performance improvement of the model on new classes, or even significant lag.

[0054] To address the aforementioned issues, the inventors researched existing federated incremental learning methods and found that they typically handle client-side local incremental learning and server-side model aggregation separately. Specifically, on the client side, they mitigate catastrophic forgetting through methods like sample replay and knowledge distillation, while on the server side, they employ a weighted aggregation strategy based on sample size to update the global model. However, this approach implicitly assumes that all clients have a consistent category learning task at the same stage, failing to consider the asynchronous exposure of new classes across different clients. Furthermore, the inventors discovered that when this assumption is invalid, new classes only appear in a few clients, causing their corresponding model updates to be overwhelmed by updates of the majority of older classes during the aggregation process. Simultaneously, due to the limited number of new class samples, the client-side local learning process itself is inherently unstable. These two types of problems stem from the lack of targeted design on the client side for learning new classes with few samples, and the server-side aggregation mechanism's failure to characterize the differentiated contributions of clients in learning new classes. These are the fundamental reasons why existing methods struggle to effectively learn and propagate new class updates.

[0055] Furthermore, the inventors analyzed the above problems in conjunction with practical application scenarios, such as... Figure 1 As shown, in the federated system, the data distribution received by different clients varies significantly. New classes often appear first in a few clients with limited samples, gradually expanding to other clients during training. In this process, on the one hand, learning new classes under limited sample conditions exhibits significant instability, and the model is easily influenced by the optimization objective of old classes. On the other hand, during the federated aggregation phase, since most clients still only contain old class data, their model updates dominate in terms of quantity and weight, further amplifying the weak position of new class updates. This process manifests as: new classes struggle to form stable representations locally, fail to achieve effective global representation, and ultimately exhibit significant performance lag or even degradation during continuous training. This phenomenon indicates that the slow performance improvement, or even significant lag, of the model on new classes is a result of the instability of local limited sample new class learning and the global old class-dominated aggregation bias under asynchronous incremental conditions.

[0056] Based on the above analysis, the inventors propose introducing a client-server collaborative optimization mechanism into the existing federated incremental learning framework to simultaneously alleviate the problems of local learning instability and global aggregation bias. In simple terms, a prototype-constrained incremental learning mechanism is constructed on the client side. Through sample replay, knowledge distillation, and prototype alignment, it collaboratively improves the stability of new class learning under few-sample conditions while maintaining the ability to retain old classes. On the server side, a client contribution-aware hierarchical aggregation mechanism is designed. This involves first coarse-grained grouping of new and old class clients, then fine-grained clustering of new class clients based on prototype similarity and update direction consistency. The effective contribution of each client group is characterized by the parameter update magnitude relative to the previous round of the global model. Weights are then calculated for inter-group aggregation, thereby alleviating aggregation bias and promoting the effective propagation of new classes. By introducing a prototype alignment strategy on the client side to improve the stability of new class learning under few-sample conditions, and by introducing a client contribution-aware hierarchical aggregation mechanism on the server side to aggregate client models, the propagation ability of new class updates in the global model is enhanced, thus alleviating the problems of local learning instability and global aggregation bias, achieving efficient learning and effective propagation of new classes in the federated system.

[0057] In summary, such as Figure 2As shown, this invention proposes a Federated Class Incremental Learning (FedHCCA) method for asynchronous new class exposure, used to execute multiple class increment tasks in a federated system to obtain a target global model. The federated system includes a server and multiple clients. During each class increment task, the server communicates with each client in multiple rounds. In each round, some clients only have the old class dataset, while the remaining clients have both the old class dataset and locally added new class datasets. The old class dataset includes multiple image categories, and the new class dataset includes one or more newly added image categories. Each image category contains multiple training samples. The method involves configuring the federated system to execute each round of communication as follows: Step S1: The server distributes the global model and global prototype set obtained in the previous round of communication to all clients. The global prototype set includes the global prototypes of each image category that have appeared. Step S2: Each client calculates the local prototype for each image category according to a preset class prototype calculation rule, obtaining the local prototype set for each client in this round of communication. Based on all training samples, the global model obtained in the previous round of communication is iterated multiple times locally. The local model of each client in this round of communication is obtained, and the local model and local prototype set of this round of communication are uploaded to the server. In each local iteration, the loss is calculated according to the preset loss function and the parameters of the local model obtained in the previous local iteration are updated. Step S3: The server divides the clients containing the newly added image categories into new class clients and the remaining clients into old class clients. According to the preset grouping strategy, all new class clients are grouped to obtain multiple new class subsets. The local models of all old class clients and the local models of all new class clients in each new class subset are weighted averaged and aggregated to obtain the old class group aggregation model and the new class group aggregation model of each new class subset in this round of communication. According to the preset contribution weighted aggregation rule, the contribution weight of the old class group aggregation model and the new class group aggregation model in this round of communication is calculated and aggregated to obtain the global model of this round of communication. Based on the local prototype set of each client in this round of communication, the global prototype of each image category added in this class increment task in the global prototype set of the previous round of communication is updated according to the preset class update rule to obtain the global prototype set of this round of communication.

[0058] To better understand the present invention, each step will be described in detail below with reference to specific embodiments.

[0059] I. Step S1

[0060] In step S1, the server distributes the global model and global prototype set obtained from the previous round of communication to all clients. Specifically, before executing the first class incremental task, the server collaborates with each client to perform basic training to build a basic global model and a basic global prototype set, and then executes the first class incremental task based on these two sets.

[0061] Specifically, basic training is performed as follows: The server initializes the global model and global prototype set and distributes them to all clients; each client iterates through the received global model on its local dataset to obtain a basic local model, and calculates the local prototype of each category in the local dataset according to a preset class prototype calculation rule to obtain a basic local prototype set for each client. The basic local model and basic local prototype set are then uploaded to the server. During each training iteration, the loss is calculated using the cross-entropy loss function, and the parameters of the global model are updated. The server aggregates the basic local models of all clients using a weighted average to obtain a basic global model. Based on each client's basic local prototype set, it calculates the global prototype of each category that appeared during basic training according to a preset class update rule to obtain a basic global prototype set, and distributes the basic global prototype set to each client. Based on the basic global prototype set, each client calculates the distance between each training sample in each image category and the global prototype of that image category, and sorts the calculated distances in ascending order to retain a preset number of training samples at the top of the sorted order, thus obtaining the old class dataset for each client in the first class increment task.

[0062] II. Step S2

[0063] In step S2, each client calculates the local prototype for each image category according to a preset class prototype calculation rule, obtaining the local prototype set for each client in this round of communication. Based on all training samples, the global model obtained in the previous round of communication is iterated multiple times locally to obtain the local model for each client in this round of communication. The local model and local prototype set of this round of communication are then uploaded to the server. In each local iteration, the loss is calculated according to a preset loss function, and the parameters of the local model obtained in the previous local iteration are updated.

[0064] According to one embodiment of the present invention, the preset class prototype calculation rule is as follows:

[0065]

[0066] in, Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype, Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The number of training samples included. Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The set consisting of all training samples. Indicates training samples, Represents the feature extraction function. Indicates the first The client received the first The sub-category of incremental tasks The global model obtained from round-robin communication.

[0067] According to one embodiment of the present invention, the preset loss function is:

[0068]

[0069] in,

[0070]

[0071]

[0072]

[0073] in,

[0074]

[0075] in, Indicates the first The sub-category of incremental tasks Round communication Total loss of each local iteration Indicates the first The sub-category of incremental tasks Round communication Cross-entropy loss in the next local iteration Indicates the first The sub-category of incremental tasks Round communication Distillation loss in the next local iteration Indicates the first The sub-category of incremental tasks Distillation loss weight in round-robin communication Indicates the first The sub-category of incremental tasks Round communication Prototype alignment loss in the next local iteration. Indicates the first The sub-category of incremental tasks Prototype alignment loss weights in round-robin communication Indicates the first The sub-category of incremental tasks The total number of training samples included in all new image categories added in round-robin communication. This represents the maximum storage limit for each image category. Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The set consisting of all training samples. Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The corresponding difference value, Indicates the first The sub-category of incremental tasks Image categories in round-robin communication global prototype Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype, , where is the adjustment factor. Since cross-entropy loss and distillation loss are commonly used loss functions in this field, they will not be elaborated upon here; only the prototype alignment loss will be explained in detail.

[0076] As can be seen from the foregoing embodiments, this invention proposes a prototype-constrained class incremental learning mechanism on the client side. The core of this mechanism lies in constructing a category prototype and combining sample replay, knowledge distillation, and prototype alignment strategies to jointly train new class samples and old class memory samples. This allows for the establishment of a stable new class feature representation under conditions of few new class samples, while also preserving the ability to retain old classes. In short, it is... Figure 3 The research framework diagram for federated incremental learning of asynchronous new class exposure shows that, on the client side, the new class client calculates the corresponding class prototype based on the current new class sample, and introduces prototype alignment constraints during model training to make the model output converge to the smoothed new class prototype in the feature space. At the same time, the distillation loss is combined to maintain the discrimination ability of the old class, and the old class distribution is kept stable through the sample replay mechanism.

[0077] As is known from the background art, although existing federated incremental learning methods introduce distillation or replay mechanisms to alleviate the performance degradation of old classes, their optimization process still relies on implicit gradient updates and does not impose explicit structural constraints on the few-sample learning process of new classes. This leads to unstable gradient directions and easy representation shifts during the learning of new classes. Unlike existing technologies, this invention introduces a class prototype as an intermediate structure on the client side, transforming the new class learning process from unconstrained optimization into a guided, structured learning process. This significantly reduces training fluctuations and improves the stability of new class representations while also preserving the ability to maintain old classes.

[0078] III. Step S3

[0079] In step S3, the server categorizes clients containing newly added image categories into new category clients and the remaining clients into old category clients. Following a preset grouping strategy, all new category clients are grouped into multiple new category subsets. Weighted average aggregation is performed on the local models of all old category clients and the local models of all new category clients within each new category subset to obtain the old category aggregation model and the new category aggregation model for each new category subset in this round of communication. According to a preset contribution-weighted aggregation rule, the contribution weights of the old category aggregation model and each new category aggregation model in this round of communication are calculated and aggregated to obtain the global model for this round of communication. Based on the local prototype set of each client in this round of communication, the global prototype set of the previous round of communication for each newly added image category in this class increment task is updated according to a preset class update rule to obtain the global prototype set for this round of communication.

[0080] According to one embodiment of the present invention, the preset grouping strategy is as follows: Based on preset similarity calculation rules, the similarity between the local model of each new type of client in this round of communication and the local model of other new type of clients in this round of communication is calculated to obtain a similarity matrix of new type of clients in this round of communication; clustering is performed based on the similarity matrix of new type of clients in this round of communication to obtain multiple new type subsets, wherein each new type subset includes one or more new type of clients. After obtaining the similarity matrix, the server can treat it as an adjacency matrix in the client relationship graph and group the clients based on graph clustering. Specifically, spectral clustering can be used for clustering: first, a graph Laplacian matrix is ​​constructed based on the similarity matrix, and eigenvalues ​​are decomposed to obtain a low-dimensional embedding representation; then, the K-Means clustering algorithm is used in the low-dimensional space to group clients with high similarity into the same cluster. It should be noted that, in addition to spectral clustering, hierarchical clustering, density clustering, or threshold-based partitioning methods can also be used to group clients, as long as structured aggregation based on the similarity relationship between clients can be achieved.

[0081] According to one embodiment of the present invention, the preset similarity calculation rule is as follows:

[0082]

[0083] in,

[0084]

[0085]

[0086] in, Indicates the first The sub-category of incremental tasks In round-robin communication, the first The local model of the first client and the first Similarity between local models of each client Represents the balance factor. Represents the cosine function. Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype, Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype, Indicates the first The sub-category of incremental tasks Round communication The update direction of the local model on each client. Indicates the first The sub-category of incremental tasks Round communication The update direction of the local model on each client; Indicates the first The sub-category of incremental tasks The global model obtained from round-robin communication.

[0087] According to one embodiment of the present invention, the preset contribution-weighted aggregation rule is as follows:

[0088]

[0089] in,

[0090]

[0091]

[0092]

[0093]

[0094] in,

[0095]

[0096]

[0097] in, Indicates the first The sub-category of incremental tasks The global model obtained through round-robin communication Indicates the first The sub-category of incremental tasks The old class group aggregation model of round communication, Represents the old class group aggregation model Contribution weight, Indicates the first The sub-category of incremental tasks Round communication A new class group aggregation model for a new subset of classes. Indicates the number of subsets of the new class. Represents a new class group aggregation model Contribution weight, Indicates the first The sub-category of incremental tasks In round-robin communication, the set of all legacy clients is composed of... Indicates the first The number of training samples on each client Indicates the first The number of training samples on each client Indicates the first The client in the first The sub-category of incremental tasks Local model of round-robin communication Represents the old class group aggregation model Contribution, normalized get , Represents a new class group aggregation model Contribution, normalized get , Indicates the first The sub-category of incremental tasks Old class group aggregation model in round communication The direction of updates; Indicates the first The sub-category of incremental tasks New class group aggregation model in round communication The direction of updates.

[0098] According to one embodiment of the present invention, the preset class update rule is:

[0099]

[0100] in, Indicates the first The sub-category of incremental tasks Image categories in round-robin communication global prototype Indicates the first The sub-category of incremental tasks The set of all new client classes in round-robin communication. Indicates the first The number of training samples on each client Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The number of samples, Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype.

[0101] As can be seen from step S3, the present invention proposes a hierarchical aggregation mechanism for client contribution awareness on the server side. This mechanism achieves adaptive enhancement for new class updates by performing structured division and contribution modeling of clients.

[0102] Specifically, such as Figure 3 As shown, the client contribution-aware hierarchical aggregation mechanism can be divided into four steps: explicit new class identification, coarse-grained aggregation of old class clients, fine-grained aggregation of new class clients, and contribution-aware inter-group weighted aggregation.

[0103] The first step is explicit new class identification: the server divides participating clients into new class client groups and old class client groups based on whether the client contains the newly added image category.

[0104] The second step is coarse-grained aggregation of old class clients: For old class client groups that only contain old class knowledge, the server uses a sample size weighted average to perform aggregation within the group. This can yield a relatively stable old class group aggregation model for preserving old knowledge.

[0105] The third step is fine-grained aggregation of new class clients: Although all new class clients have access to the new class, the sample size, representation quality, and update direction of the new class may differ significantly among them. Therefore, this invention proposes to further cluster the new class client groups using new class prototype similarity and gradient update similarity, so that new class clients with similar semantics and consistent update directions are grouped into a new class subset. Within each new class subset, a sample size-weighted average is used for intra-group aggregation to reduce conflicts between heterogeneous new class updates.

[0106] The fourth step is contribution-aware weighted aggregation between groups: After obtaining the aggregation model of the old class group and the aggregation model of the new class group corresponding to each new class, the server estimates the contribution of each group based on the update magnitude of each group relative to the previous round of global model, and performs normalized weighting so that the truly valuable new class updates can obtain higher weight in the global aggregation, thereby obtaining the global model of this round of communication.

[0107] As is known from the background art, existing federated aggregation methods typically employ uniform weights or simple weighting strategies, failing to characterize the effective contributions of different clients in learning new classes. Under asynchronous new class exposure conditions, this can easily lead to the dilution of new class updates by a large number of old class updates. Unlike existing technologies, this invention introduces a hierarchical aggregation structure of "client partitioning—similarity modeling—contribution weighting," enabling new class updates to receive targeted reinforcement during the aggregation process, thereby improving the expressive power and propagation efficiency of new classes in the global model.

[0108] In addition to the aforementioned steps S1-S3, the federated class incremental learning method for asynchronous new class exposure proposed in this invention also includes an update step for the old class dataset.

[0109] According to an embodiment of the present invention, the method further includes: in the last round of communication of each class increment task, each client determines the global prototype of each new image category in the current class increment task based on the global prototype set obtained in the current round of communication; based on the global prototype of each new image category in the current class increment task, each client calculates the distance between each training sample contained in each new image category and the global prototype of the image category, and sorts the calculated distances in ascending order to retain a preset number of training samples at the top of the sorting, and merges all the training samples retained for each new image category with the old class dataset corresponding to the current class increment task to obtain the old class dataset of each client in the next class increment task.

[0110] Based on the foregoing content and Figure 3As can be seen, the FedHCCA (Fed Incremental Learning Method for Asynchronous New Class Exposure) proposed in this invention is designed collaboratively from two levels: the server side and the client side. On the client side, a prototype-constrained incremental learning mechanism for classes is designed, and on the server side, a hierarchical aggregation mechanism that is aware of client contributions is designed.

[0111] On the client side, the prototype-constrained incremental learning mechanism constructs a prototype-constrained locally stable learning strategy through prototype-guided old class replay, knowledge distillation, and prototype alignment. Its goal is to stabilize new class representation learning under limited sample conditions while preserving old knowledge, thereby generating more stable and discriminative local model updates.

[0112] On the server side, the client-contribution-aware hierarchical aggregation mechanism constructs a hierarchical aggregation strategy through explicit new class identification, coarse-grained grouping, and contribution-aware weighting. Its goal is to enhance the effectiveness of clients with new classes in global updates, prevent new class updates from being overwhelmed by a large number of old class updates, and improve the effectiveness of new class propagation to the global model.

[0113] Therefore, the FedHCCA (Fed Incremental Learning Method for Asynchronous New Class Exposure) proposed in this invention designs a prototype-constrained incremental learning mechanism on the client side to make the new class learn more stably, and designs a hierarchical aggregation mechanism for client contribution awareness on the server side to make the new class transmit faster. Through the synergy of the two, the problems of "slow transmission" and "poor learning" under asynchronous new class exposure are alleviated.

[0114] The "Federated Incremental Learning Method for Asynchronous New Class Exposure" proposed in this invention can be applied to various scenarios characterized by "privacy protection + distributed deployment + continuous emergence of new categories". For example, in the field of medical image analysis: when different hospitals or medical institutions continuously receive new diseases, pathological types, or image categories, this method can achieve collaborative learning of new class knowledge without sharing original patient data; in the field of industrial defect detection: when different production lines or equipment may periodically introduce new defect types, this method can support the continuous learning and propagation of new defect categories in the federated system; in the field of intelligent terminal sensing: when different terminal devices continuously encounter new user behavior patterns, environmental targets, or scene categories, this method can achieve collaborative incremental learning of new categories among edge devices; in the field of intelligent transportation and autonomous driving: when different vehicles or roadside equipment continuously encounter new traffic targets or abnormal events during operation, this method can support the continuous identification and sharing of new category targets; in the field of security monitoring and intelligent inspection: when different monitoring nodes continuously encounter new abnormal event categories during long-term operation, this method can achieve federated propagation and updating of new class knowledge among distributed nodes.

[0115] Based on the foregoing, in each round of federated communication for each class increment task, the server first distributes the global model from the previous round of communication to all clients participating in training. Subsequently, some clients perform multiple local iterations based on the old class dataset, while others perform local stabilization learning with prototype constraints based on the old class dataset and the newly added image categories, thus obtaining the local model for each client in this round of communication. Finally, the server receives the local models uploaded by each client and performs hierarchical aggregation based on the client's class increment state to obtain a new global model.

[0116] To better understand this invention, the following is combined with... Figure 4 The execution process of each round of communication in each incremental task will be illustrated using the aforementioned embodiments. Figure 4 The execution flow of each round of communication in each incremental task is shown, which includes steps T1-T6. Each step is described below.

[0117] Step T1: Distribution of Global Model and Global Prototype Set: The server distributes the global model and global prototype set obtained in the previous round of communication to all clients. Let the set of clients in the federated system be denoted as . .

[0118] Step T2, Incremental Learning of Prototype Constraint Classes on the Client Side: In each round of communication for each incremental class task, each client receives the global model and global prototype set from the previous round of communication and performs multiple local iterations. The training method for each local iteration has been detailed in the preceding embodiments and will not be repeated here.

[0119] Step T3, Explicit Recognition of New Classes and Client Grouping: Based on the local prototype set uploaded by each client, the server determines whether each client contains the newly added image category. Specifically, the determination of whether each client contains the newly added image category is as follows:

[0120]

[0121] in, This represents the set of image categories newly added in this incremental task. Indicates the first The sub-category of incremental tasks Round communication The judgment value of each client, if Then, the corresponding clients are classified into old-type clients, resulting in a set of old-type clients. ;like Then, the corresponding clients are divided into new client classes, resulting in a set of clients composed of these new client classes. .

[0122] Step T4, Coarse-grained aggregation of legacy clients: For the collection of legacy clients Since it only contains updates to the old class and has a consistent optimization objective, a weighted average method is used for aggregation. The aggregation process is as follows:

[0123]

[0124] in, Indicates the first The sub-category of incremental tasks The old class group aggregation model of round communication, Indicates the first The sub-category of incremental tasks In round-robin communication, the set of all legacy clients is composed of... Indicates the first The number of training samples on each client Indicates the first The number of training samples on each client Indicates the first The client in the first The sub-category of incremental tasks A local model for round-robin communication.

[0125] Step T5, Fine-grained aggregation of new class clients: For the collection of new class clients Because of the strong heterogeneity in its data distribution, direct averaging would lead to update conflicts. Therefore, multiple new class subsets are obtained by grouping based on dual similarity (similarity of the new class prototype and similarity of the model update direction). Then, a weighted average aggregation is performed on the local models of all clients in each new class subset to obtain the new class group aggregation model for each new class subset. Since the similarity calculation rules and the weighted average aggregation method have been described above, they will not be repeated here.

[0126] Step T6, Contribution-Aware Weighted Aggregation: The server calculates and aggregates the contribution weights of the old class group aggregation model and each new class group aggregation model in this round of communication to obtain the global model of this round of communication. Based on the local prototype set of each client in this round of communication, the server updates the global prototype of each image category added in this class incremental task in the global prototype set of the previous round of communication according to the preset class update rules to obtain the global prototype set of this round of communication.

[0127] The global model for this round of communication is calculated as follows:

[0128]

[0129] in, Indicates the first The sub-category of incremental tasks The global model obtained through round-robin communication Indicates the first The sub-category of incremental tasks The old class group aggregation model of round communication, Represents the old class group aggregation model Contribution weight, Indicates the first The sub-category of incremental tasks Round communication A new class group aggregation model for a new subset of classes. Indicates the number of subsets of the new class. Represents a new class group aggregation model The contribution weight.

[0130] Specifically, the global prototype for each image category newly added in this incremental task is calculated and updated in the global prototype set of the previous round of communication as follows:

[0131]

[0132] in, Indicates the first The sub-category of incremental tasks Image categories in round-robin communication global prototype Indicates the first The sub-category of incremental tasks The set of all new client classes in round-robin communication. Indicates the first The number of training samples on each client Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The number of samples, Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype.

[0133] To comprehensively evaluate the effectiveness of this invention, the inventors designed comparative experiments on three standard image classification datasets, and obtained the experimental results shown in Table 1. The specific settings for the comparative experiments are as follows.

[0134] The datasets include CIFAR-100 (100 classes), CIFAR-10 (10 classes), and SVHN (10 classes).

[0135] Number of clients: 10 clients in total. The data is divided into non-independent and identically distributed parts according to the Dirichlet distribution to simulate the heterogeneous data scenario in real federated learning.

[0136] Task division: A "basic phase + single-class incremental phase" model is adopted. CIFAR-100 includes one basic phase and ten class incremental tasks (each phase introduces one new class); CIFAR-10 and SVHN each include one basic phase and five class incremental tasks (each phase introduces one new class). Each incremental task executes for 10 communication rounds (Rounds 0-9).

[0137] New class exposure mechanism: Each new class in the incremental task is only exposed to 4 clients, and the new class clients are designed according to the exposure sequence, following an exponential growth curve within the exposure window. The normalized distribution (the number of new class samples obtained by each new class client in different communication rounds is [3, 9, 24, 64], that is, 100 new class samples are distributed to different new class clients according to the exposure time order. For a new class client, in the first communication round of allocation, the new class client obtains 3 new class samples; in the second communication round of allocation, the new class client obtains 9 new class samples; in the third communication round of allocation, the new class client obtains 24 new class samples; and in the fourth communication round of allocation, the new class client obtains 64 new class samples). Where Round 0: Client Start acquiring new class samples (in the first stage) In the first round of communication for the sub-incremental task, the client... Start retrieving new class samples, client The communication rounds for obtaining new class samples are rounds 1 to 4); Round 3: Client Obtain new class samples (in the first) In the fourth round of communication for the sub-incremental task, the client... The client begins retrieving samples of the new class. The communication rounds for obtaining new class samples are rounds 4 to 7. Round 5: Client Obtain new class samples (in the first) In the 6th round of communication for the sub-incremental task, the client... and Start retrieving new class samples, client and The communication rounds for obtaining new class samples are rounds 6 to 9. Round 9: The remaining clients only receive new class samples for memory replay in subsequent incremental tasks and do not participate in the local training of the current incremental task.

[0138] The comparison methods include traditional federated learning (FL) methods, classic continuous learning (CL) methods, and federated incremental learning (FCIL) methods. Traditional federated learning (FL) methods include FedAvg and FedProx; classic continuous learning (CL) methods include PODNet, iCaRL, and EWC (implemented through a joint training approach of federated learning and continuous learning); and federated incremental learning (FCIL) methods include GLFC (CVPR 2022), LGA (TPAMI 2023), AF-FCL (ICLR 2024), and SacFL (TNNLS 2025).

[0139] Evaluation metrics include overall accuracy (ACC_a), accuracy of the old class (ACC_o), and average accuracy of the new class (ACC_n).

[0140] As shown in Table 1, this invention demonstrates significant advantages in overall performance, new class performance, and new class propagation efficiency. The overall accuracy is improved by an average of 1.31 percentage points compared to the best comparison method. The new class accuracy averages 41.52% across different datasets, a significant improvement over existing methods, while maintaining stable performance for existing classes, indicating that the method can balance new class learning and existing class retention. Furthermore, in terms of the number of communication rounds required to reach the preset new class accuracy threshold, this invention requires only approximately 3–4 rounds on average, significantly lower than the 8–10 rounds or more typically required by existing methods, demonstrating higher new class propagation efficiency. In addition, this invention does not require additional data sharing or complex communication structures, achieving a comprehensive improvement in performance and efficiency while ensuring the scalability of the federated system, thus possessing significant engineering application value.

[0141] Table 1

[0142]

[0143] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) On the client side, a prototype-constrained class incremental learning mechanism is designed. This mechanism maintains the old class discrimination boundary through prototype-guided playback, slows down the forgetting of existing classes by the local model through knowledge distillation, and enhances the feature aggregation ability of the new class under few sample conditions through prototype alignment, thereby generating a more stable and discriminative local model update, so as to enhance the representation stability of the few sample new class learning stage; (2) On the server side, a client contribution-aware hierarchical aggregation mechanism is designed. This mechanism divides the participating clients according to whether the client has a new class increment, and on this basis, it combines update similarity and group-level contribution to perform grouping and weighted aggregation, so as to reduce the risk that the updates of a few new class clients are diluted by the updates of most old class clients in the global aggregation process, and improve the effectiveness of the new class propagation to the global model.

[0144] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0145] This invention can be a system, method, electronic device, computing device, computer-readable medium, and / or computer program product. A computer program product mainly refers to a software product that implements this solution through a computer program.

[0146] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A federated class incremental learning method for asynchronous new class exposure, used to perform multiple class incremental tasks in a federated system to obtain a target global model, wherein the federated system includes a server and multiple clients, and the server communicates with each client in multiple rounds during each class incremental task. In each round of communication, some clients only include the old class dataset, while the remaining clients include the old class dataset and locally added new class datasets. The old class dataset includes multiple image categories, and the new class dataset includes one or more newly added image categories. Each image category contains multiple training samples. The method is characterized by... The method involves configuring the federated system to perform each round of communication in the following manner: Step S1: The server sends the global model and global prototype set obtained in the previous round of communication to all clients. The global prototype set includes the global prototype of each image category that has appeared. Step S2: Each client calculates the local prototype of each image category according to the preset class prototype calculation rules, and obtains the local prototype set of each client in this round of communication. Based on all training samples, the global model obtained in the previous round of communication is iterated multiple times locally to obtain the local model of each client in this round of communication. The local model and local prototype set of this round of communication are uploaded to the server. In each local iteration, the loss is calculated according to the preset loss function and the parameters of the local model obtained in the previous local iteration are updated. Step S3: The server categorizes clients containing newly added image categories into new category clients and the remaining clients into old category clients. Following a preset grouping strategy, all new category clients are grouped into multiple new category subsets. Weighted average aggregation is performed on the local models of all old category clients and the local models of all new category clients within each new category subset to obtain the old category aggregation model and the new category aggregation model for each new category subset in this round of communication. According to a preset contribution-weighted aggregation rule, the contribution weights of the old category aggregation model and each new category aggregation model in this round of communication are calculated and aggregated to obtain the global model for this round of communication. Based on the local prototype set of each client in this round of communication, the global prototype set of the previous round of communication for each newly added image category in this class increment task is updated according to a preset class update rule to obtain the global prototype set for this round of communication.

2. The method according to claim 1, characterized in that, The preset class prototype calculation rule is as follows: in, Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype, Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The number of training samples included. Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The set consisting of all training samples. Indicates training samples, Represents the feature extraction function. Indicates the first The client received the first The sub-category of incremental tasks The global model obtained from round-robin communication.

3. The method according to claim 2, characterized in that, The preset loss function is: in, in, in, Indicates the first The sub-category of incremental tasks Round communication Total loss of each local iteration Indicates the first The sub-category of incremental tasks Round communication Cross-entropy loss in the next local iteration Indicates the first The sub-category of incremental tasks Round communication Distillation loss in the next local iteration Indicates the first The sub-category of incremental tasks Distillation loss weight in round-robin communication Indicates the first The sub-category of incremental tasks Round communication Prototype alignment loss in the next local iteration. Indicates the first The sub-category of incremental tasks Prototype alignment loss weights in round-robin communication Indicates the first The sub-category of incremental tasks The total number of training samples included in all new image categories added in round-robin communication. This represents the maximum storage limit for each image category. Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The set consisting of all training samples. Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The corresponding difference value, Indicates the first The sub-category of incremental tasks Image categories in round-robin communication The global prototype Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype, It is a regulating factor.

4. The method according to claim 1, characterized in that, The preset grouping strategy is as follows: According to the preset similarity calculation rules, the similarity between the local model of each new type of client in this round of communication and the local model of other new type of clients in this round of communication is calculated to obtain the similarity matrix of new type of clients in this round of communication; Clustering is performed based on the similarity matrix of the new client types in this round of communication to obtain multiple new class subsets, where each new class subset includes one or more new client types.

5. The method according to claim 4, characterized in that, The preset similarity calculation rule is as follows: in, in, Indicates the first The sub-category of incremental tasks In round-robin communication, the first The local model of the first client and the first Similarity between local models of each client Represents the balance factor. Represents the cosine function. Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype, Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype, Indicates the first The sub-category of incremental tasks Round communication The update direction of the local model on each client. Indicates the first The sub-category of incremental tasks Round communication The update direction of the local model on each client; Indicates the first The sub-category of incremental tasks The global model obtained from round-robin communication.

6. The method according to claim 1, characterized in that, The preset contribution-weighted aggregation rule is as follows: in, in, in, Indicates the first The sub-category of incremental tasks The global model obtained through round-robin communication Indicates the first The sub-category of incremental tasks The old class group aggregation model of round communication, Represents the old class group aggregation model Contribution weight, Indicates the first The sub-category of incremental tasks Round communication A new class group aggregation model for a new subset of classes. Indicates the number of subsets of the new class. Represents a new class group aggregation model Contribution weight, Indicates the first The sub-category of incremental tasks In round-robin communication, the set of all legacy clients is composed of... Indicates the first The number of training samples on each client Indicates the first The number of training samples on each client Indicates the first The client in the first The sub-category of incremental tasks Local model of round-robin communication Represents the old class group aggregation model Contribution, normalized get , Represents a new class group aggregation model Contribution, normalized get , Indicates the first The sub-category of incremental tasks Old class group aggregation model in round communication The direction of updates; Indicates the first The sub-category of incremental tasks New class group aggregation model in round communication The direction of updates.

7. The method according to claim 1, characterized in that, The preset class update rule is as follows: in, Indicates the first The sub-category of incremental tasks Image categories in round-robin communication The global prototype Indicates the first The sub-category of incremental tasks The set of all new client classes in round-robin communication. Indicates the first The number of training samples on each client Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The number of samples, Indicates the first The sub-category of incremental tasks Round communication Image categories on each client The local prototype.

8. The method according to claim 1, characterized in that, The method further includes: In the last round of communication for each class increment task, each client determines the global prototype for each new image category in this class increment task based on the global prototype set obtained in this round of communication. Based on the global prototype of each new image category added in this class increment task, each client calculates the distance between each training sample in each new image category and the global prototype of that image category, and sorts the calculated distances in ascending order to retain a preset number of training samples at the top of the sort. All training samples retained for each new image category are then merged with the old class dataset corresponding to this class increment task to obtain the old class dataset for each client in the next class increment task.

9. A computer device comprising a memory, a processor, and computer programs / instructions stored in the memory, characterized in that, The processor executes the computer program / instructions to implement the steps of the method according to any one of claims 1-8.

10. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-8.

Images