Federated domain incremental learning method based on pre-training ViT and parameter adaptation

Abstract

The invention discloses a federated domain incremental learning method based on pre-training ViT and parameter adaptation, and belongs to the technical field of federated learning. The method comprises the following steps: firstly, freezing a basic weight of a pre-trained Vision Transform model, establishing an independent domain expert knowledge network for each domain, and designing a domain knowledge extractor and a federated training strategy; when a new task appears, dynamically adding a new domain expert knowledge network in a global model structure of the server, sending the new domain expert knowledge network to a client participating in new task training, and locally training the domain expert knowledge network consisting of a domain expert knowledge module and a classification head by the client; and the cosine feature center of local data is trained to participate in the construction of a global model domain knowledge extractor. The constructed domain knowledge extractor can autonomously select the domain expert knowledge network in the reasoning stage according to the cosine similarity between the sample features and the domain center in the reasoning stage. According to the method, the performance of the global model in a domain increment scene is improved, and the communication overhead between the server and the client is greatly reduced.

Description

Technical Field

[0001] The present invention belongs to the field of application of federated learning technology, and particularly relates to a continuous learning method in federated learning, and more particularly to a federated domain incremental learning method based on pre-trained Vision Transformer and parameter adaptation. Background Art

[0002] With the wide application of intelligent devices and the development of the Internet of Things, a large amount of data is distributed among various clients. The traditional centralized data processing method faces huge privacy and data security problems. In recent years, people's attention to privacy protection has gradually increased. Federated learning is an innovative distributed machine learning method. The client uses local data to train the model and only uploads the model parameters to the server for aggregation. The server combines the updates of each client to form a global model, and finally distributes the updated model back to the client for the next round of training. This process does not require data exchange, ensuring the privacy and security of the data. Federated learning has broad application prospects, especially in scenarios that require processing large-scale distributed data and have high requirements for privacy and security, such as healthcare, financial services, intelligent transportation, and autonomous driving.

[0003] Although federated learning has significant advantages in many scenarios, it also faces some problems. Traditional federated learning methods mainly focus on training a single task with a static data distribution, where the data distribution remains unchanged throughout the training process. However, in practical applications, the data often has dynamics. Over time, clients with new data distributions may appear, or tasks with new data distributions may appear in the clients. How to avoid catastrophic forgetting and not lose the knowledge of previous tasks remains a major challenge for federated learning.

[0004] Continual learning or incremental learning is a learning strategy that enables a model to effectively learn new knowledge while continuously receiving new tasks and retain the learning from previous tasks. Incremental learning can be divided into three main variants: task-incremental learning, class-incremental learning, and domain-incremental learning. In task-incremental learning, the model knows the boundaries of each new task it receives, usually distinguished by task labels. The data for each task is separate, so the model can consciously maintain specific knowledge for each task and activate the corresponding parameters when needed. In contrast, class-incremental learning does not explicitly distinguish task boundaries. In class-incremental learning, the model may encounter multiple classes when learning a new task, but there is no information indicating the classes in the current task. The model needs to learn new classes without sufficient information and maintain performance on the classes it has already learned. Domain-incremental learning focuses on cross-domain knowledge transfer. The data received by the model in each new task comes from domains with different data distributions but the same categories. The model must adapt to the new domain features while maintaining performance on the previous domains, which is particularly important in many real-world application scenarios, such as different lighting conditions or backgrounds in visual tasks.

[0005] Existing continual learning strategies can be divided into two categories: replay-based methods and non-replay-based methods. Replay-based methods usually utilize historical data to assist current learning to reduce forgetting. However, this method has many problems. First, the replay method needs to store the data of past tasks for future training use, so in the case of an increasing number of tasks, it will occupy a large amount of storage space. Second, in many applications involving personal data, saving the data of past tasks may lead to privacy risks. In the scenario of federated learning, these problems are more prominent. The storage space of clients is usually limited and may not be able to store a large amount of historical data. The data of past tasks may not be accessible because the client no longer participates in new tasks, affecting the model's replay learning of knowledge. Even if the same client participates in multiple tasks, due to the restrictions of privacy protocols, the data of past tasks may not be accessible and utilized again due to compliance issues. These challenges have prompted researchers to explore more effective federated non-replay continual learning algorithms to achieve a more robust and flexible incremental learning mechanism.

[0006] In recent years, many works have studied federated non-replay continual learning algorithms in the field of federated learning, attempting to solve the problem of catastrophic forgetting encountered in federated learning. Although quite successful, there are still some challenges:

[0007] (1) The current work mainly focuses on the research of federated class-incremental learning, while there is little research on federated domain-incremental learning. The methods designed for class-incremental scenarios are not fully applicable to domain-incremental scenarios. The main challenge of class-incremental tasks is to retain the ability to distinguish old classes when learning new classes, usually only requiring local adjustment of the feature space. However, domain-incremental tasks require the model to be able to adapt to significantly different data distributions, often requiring global adjustment of the feature space to meet the needs of the new domain.

[0008] (2) In federated learning, the communication between clients and the server is frequent. Especially in the incremental learning scenario, as the number of tasks and the number of participating clients increase, the communication cost may become a bottleneck, affecting the scalability of the system.

[0009] (3) Although some incremental learning methods can alleviate the forgetting phenomenon without relying on replay samples, they greatly increase the local computational storage or communication overhead of the client, such as adding regularization terms, distilling the old model, and using a generator to simulate and generate old data, etc.

[0010] (4) The method of using a generative model to generate samples of old tasks to replace the storage of historical data may bring privacy risks. Research shows that generative models may leak sensitive information through model inversion attacks, threatening user privacy. In addition, the additional introduced generator needs to accurately model the data distribution and may perform unstably in non-independent and identically distributed (Non-IID) data scenarios. Summary of the Invention

[0011] Object of the Invention: Aiming at the catastrophic forgetting and communication overhead problems of domain-incremental tasks in the above-mentioned existing technologies, the present invention provides a federated domain-incremental learning method based on pre-trained ViT and parameter adaptation.

[0012] Technical Solution: A federated domain-incremental learning method based on a pre-trained Vision Transformer. This method freezes the basic weights of the pre-trained Vision Transformer model, establishes an independent domain expert knowledge network for each domain, designs a domain knowledge extractor, and a federated training strategy;

[0013] When a new task appears, a new domain expert knowledge network is dynamically added to the global model structure of the server and sent to the clients participating in the training of the new task;

[0014] The client locally trains the domain expert knowledge network composed of a domain expert knowledge module and a classification head, and participates in the construction of the global model domain knowledge extractor by training the cosine feature center of the local data;

[0015] The constructed domain knowledge extractor can autonomously select the domain expert knowledge network according to the cosine similarity between the sample features and the domain center during the inference stage.

[0016] Further, the method includes the following steps:

[0017] S1. Construct a federated learning system including a server and multiple clients. The server has a pre-trained vision transformer global model.

[0018] S2. The clients respectively collect data from different domains over time. When the central server publishes a new domain incremental task, the clients with sufficient relevant domain data can participate in this training.

[0019] S3. When there are new clients that have not participated in training in a communication round, the server sends the vision transformer global model and its pre-trained weights to each client.

[0020] S4. When a new domain incremental task appears, the server dynamically expands the domain expert knowledge module of the global model, including the domain expert knowledge module and the classification head in each Block of the model feature extraction part.

[0021] S5. The clients participating in this task training extract the features of the local corresponding data through the feature extractor of the pre-trained vision transformer model loaded locally in the first communication round with the server, and update the local cosine feature center using mini-batch gradient descent.

[0022] Assume that the number of samples in a batch is m, then the loss function used to update the local cosine feature center is defined as follows:

[0023]

[0024] where represents the cosine similarity between the feature extraction part f(·) of the model and the cosine feature center c.

[0025] S6. The clients download the domain expert knowledge network corresponding to this domain incremental task from the server. During the training process, the weights of the basic model of the vision transformer are frozen, and only the domain expert knowledge module and the classification head are trained. The cross-entropy loss function used during training is defined as follows:

[0026]

[0027] Among them, m represents the size of the mini - batch, that is, the number of samples in one training iteration, C represents the number of classes of the target task, and y i,c represents the one - hot encoding of the i - th sample at class c, and represents the probability that the model predicts the i - th sample to be class c.

[0028] S7. The client uploads the locally trained domain expert knowledge network to the server for global aggregation. Suppose a total of K clients are selected to participate in the training in this round, and the size of the local data of the k - th client regarding task t is and its updated local model weight is Then the server performs weighted aggregation on the weights according to the following formula:

[0029]

[0030] is the weight of the domain expert knowledge network of the global model regarding the t - th task after this round of aggregation;

[0031] S8. Repeat steps S6 to S7 until the iteration termination requirement of this domain - incremental task is met. The server statistics the cosine feature centers of the local data of all clients that have participated in the training of this task, calculates the global cosine feature center, and stores it in the global model;

[0032] Suppose a total of N clients have participated in the training of task t, then:

[0033]

[0034] In the formula, represents the cosine feature center of the i - th client during the training of task t, represents the size of the local relevant data volume of the i - th client during the training of task t, and c t represents the global feature center of task t;

[0035] S9. This round of federated domain - incremental learning ends. The server saves the global cosine feature center and the domain expert knowledge network of the current domain - incremental task, and when a new domain - incremental task appears, it restarts from step S4.

[0036] Further, the domain expert knowledge module added in step S4 in the vision transformer model consists of two linear layers; in order to reduce the model parameters to be trained and thus reduce the computational and communication overhead of the client, this method uses the matrix multiplication of two smaller linear layers instead of a single linear layer matrix to perform domain adaptation adjustment on the initial weight W0 of each block in the pre-trained vision transformer. The calculation of the model can be expressed as:

[0037] Y = W0X + W(W0X) = W0X + (BA)(W0X)

[0038] where X is the input of a certain block of the model, Y represents the output of the corresponding block of the model, and A and B represent two smaller linear matrices.

[0039] Further, in step S4, when a new domain incremental task appears, assuming there are K clients participating in the first communication round, if the global model of the central server already has the domain expert knowledge network obtained from the previous trained tasks. These K clients first send the trained local cosine feature centers to the server. Then, the server calculates the average cosine feature center weighted by the client data volume according to the following formula:

[0040]

[0041] In the formula, represents the cosine feature center of the k-th client in the first communication round during the training of task t, represents the size of the local relevant data volume of the k-th client in the first communication round during the training of task t, and c t0 represents the average cosine feature center of the clients participating in the first communication round of task t;

[0042] The server calculates the cosine distance between the current center and the existing domain cosine feature centers, and selects the domain expert knowledge module corresponding to the closest center as the domain initialization weight of the global model for this task, so as to accelerate convergence through knowledge transfer. The search formula is as follows:

[0043]

[0044] where is the set of cosine feature centers of the global model at the central server during the (t - 1)-th task.

[0045] For clients in non-first communication rounds, they still need to calculate the local cosine feature centers, which do not need to be sent to the server separately, but only need to be uploaded together when they first upload their local weights to the server.

[0046] Furthermore, when using the global model of the server for inference, since the domain ID is unknown, we first use the basic pre-trained feature extractor of the model to extract the features of the sample, and calculate the cosine distance between the extracted sample features and all current cosine feature centers. Then, we select the cosine feature center that is closest to the current sample features of the domain.

[0047]

[0048] where C global is the set of all cosine feature centers of the model.

[0049] For each inference sample x, we use the selected domain expert knowledge network to perform inference on the sample to obtain the final output result.

[0050] Furthermore, the data volume weighted aggregation method is used in steps 4, 7, and 8 of the method. However, the weighted aggregation in step 7 is used for the model network weights. Different from traditional federated learning, it does not aggregate the complete model parameters, but only aggregates the parameters corresponding to the domain expert network (including the domain expert module and the classification head). Steps 4 and 8 both perform weighted aggregation on the local cosine centers of the client. The center calculated in step 4 does not come from all clients. The purpose is that when switching to a new domain incremental task, compared with random initialization, the server hopes to have a more effective initialization for the domain expert network and needs to judge which domain incremental feature the current domain's features are closer to in the previous training. Therefore, in the method, the server requires the clients participating in the first round of training to calculate the local cosine feature centers and upload them before the formal training starts. At this time, what is obtained is a temporary and rough global feature center of this domain because there are other clients that have not participated in the training. The cosine feature center calculated in step 8 comes from all clients, is closer to the global distribution of the domain, and is also finally stored in the server's global model.

[0051] Beneficial effects: Compared with the prior art, the substantial progress and remarkable effects of the present invention are as follows:

[0052] 1) To solve the catastrophic forgetting problem in the federated learning domain incremental scenario, the present invention freezes the weights of the basic model and trains the weight biases of a single domain for each new domain, enabling the model to adapt to the distribution of the new domain. During testing and actual applications, since the domain ID is unknown, the present invention proposes a domain selector based on cosine distance, which selects the most suitable domain expert knowledge network for the image. The present invention significantly improves the average accuracy of each domain in the federated domain incremental scenario and is robust under different domain incremental task training orders.

[0053] 2) The present invention comprehensively considers the unknownness of the number of tasks on the server side, as well as the storage and communication overheads of the client. The server-side model domain expert knowledge network can be dynamically extended as needed, rather than stipulating the size of the knowledge base all at once. For clients with limited storage and communication capabilities, the present invention allows them to only download sub-models of the server's global model required for training. In communication rounds other than the first round, only about 0.15% of the parameter quantity of the global model needs to be downloaded and uploaded. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a schematic diagram of the overall framework process of the method described in the present invention;

[0055] Figure 2 is the structure diagram of the server-side global model constructed by the method described in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] An embodiment of the present invention is a classification task for the Sub-DomainNet dataset. To elaborate in detail on the technical solution disclosed by the present invention, the following further elaboration is made in conjunction with the specification drawings and specific embodiments.

[0057] In conjunction with Figure 1 , the implementation process of the method described in the present invention is elaborated in detail as follows.

[0058] Step 1: Construct a federated learning system including a server and multiple clients, and establish a vision transformer model on the central server.

[0059] In this embodiment, vit_base_patch16_224 is used, which is a medium-scale model based on the Vision Transformer (ViT) architecture. The size of the input image is 224×224 pixels, and the image is cut into 16×16 tiles, a total of 196 tiles. After each tile is linearly projected, it is mapped to a 768-dimensional vector to obtain the tile embedding representation. To encode the position information, the model adds position encoding to each tile embedding vector. Subsequently, the tiles pass through 12 layers of Transformer encoders, and the model captures global image information through the self-attention mechanism, uses layer normalization between layers, and performs feed-forward processing through a multi-layer perceptron (MLP). Finally, the model only uses the output of the first tile (classification token) and performs image classification through a linear classification head. The pre-trained weights of vit_base_patch16_224 can be easily obtained from multiple public channels.

[0060] Step 2: The client collects data from different local domains over time. When the central server publishes a new domain incremental task, the clients with relevant domain data can participate in this training based on the corresponding data.

[0061] Step 3: N clients with different local data distributions participate in the training of task t. If these clients have not participated in previous training, the server needs to send the general knowledge part of the vision transformer, that is, the basic model weights, to each client. After receiving the global model, the client adds a domain expert knowledge module to each Block of the vision transformer on the basis of the general knowledge part of the model to construct a local model.

[0062] Step 4: When a new domain incremental task appears, the server dynamically expands the domain expert knowledge module and the classification head of the global model, as Figure 2 shown.

[0063] Step 5: The clients participating in the training of this task extract the features of the local corresponding data through the feature extractor of the pre-trained vision transformer model loaded locally when communicating with the server for the first time. And obtain the local cosine feature center c based on the local data through gradient descent.

[0064] Step 6. The client downloads the domain expert knowledge network corresponding to this domain incremental task from the server. During the training, the basic model weights of the vision transformer are frozen, and only the domain expert knowledge module and the classification head are trained. The following are the hyperparameters that can be used during training: the number of clients num_user = 10, the proportion of clients selected in each communication round m = 0.5, the batch-size used for gradient descent is 32, the SGD optimizer is used, the learning rate is 0.001, the momentum momentum = 0.9, and the weight decay weight_decay = 5E-5 to improve the training stability and prevent overfitting. The learning rate scheduler is used to adjust the learning rate according to the training progress through the cosine annealing function.

[0065] Step 7. The client uploads the locally trained domain expert knowledge network to the server for global aggregation. Assume that a total of K clients are selected to participate in the training in this round, and the size of the local relevant data volume of the kth client is Then the server aggregates the weights according to the following formula.

[0066]

[0067] Step 8. Repeat Steps 6 to 7 until the iteration termination requirement of the current domain increment task is met. The server calculates the cosine feature centers of the local data of all clients participating in the training of this task, calculates the global cosine feature center by weighting according to the data volume, and stores it in the global model.

[0068] Step 9: The current federated domain incremental learning ends, and the server saves the global cosine feature center and the domain expert knowledge network of the current domain increment task. As Figure 2 shown, the central server has multiple domain expert knowledge modules and classification heads. In this embodiment, the global model of the server is used for inference to verify the model effect: Since the domain ID is unknown, we first use the basic pre-trained feature extractor of the model to extract the features of the samples, and calculate the cosine distance between the extracted sample features and all the current cosine feature centers. Then, the cosine feature center closest to the current sample features of the domain is selected, and the corresponding domain expert network is used for inference.

[0069] When a new domain increment task appears, start from Step 4 again.

[0070] Specific embodiment effect: The evaluation indexes of this embodiment are 1) the final average precision AN, that is, the average of the test precisions of the final global model on all N tasks. 2) The number of trainable model parameters, that is, the number of parameters that need to be trained locally by the client and communicated with the server in each round.

[0071] Let S t,τ be the evaluation score, for example, the classification accuracy of the τ-th task after the training of the t-th task. After the model completes the training of the t-th task, calculate the average accuracy A t as follows:

[0072]

[0073] The experimental results of the examples of the present invention compared with several baselines (FedAvg, FedEWC, Cfed, Fed-S-prompts) are shown in Table 1. The present invention significantly improves the average accuracy of each domain in the federated domain increment scenario. Compared with the traditional methods that need to train the complete model parameters locally by the client in each round except Fed-S-prompts, the number of trainable parameters only accounts for about 0.15% of the global model size, greatly reducing the locally trainable model parameters, thereby reducing the communication overhead between the server and the client.

[0074] Table 1

[0075]

[0076] The above embodiments are only used to help understand the present invention and do not limit the scope of the present invention. Based on the content of the present invention, other embodiments achieved by those skilled in the art without creative efforts all fall within the protection scope of the present invention. The description in this specification should not be regarded as a limitation to the present invention. Any modification, equivalent replacement or improvement made within the core idea and principle scope of the present invention should be included within the protection scope of the present invention.

Claims

1. A federated domain incremental learning method based on pre-trained ViT and parameter adaptation, characterized by: The steps include: S1. Build a federated learning system including a server and a client. A pre-trained vision transformer global model exists on the server. S2. Clients collect data from different local domains over time. When the central server releases a new domain increment task, clients whose domain data meets the requirements participate in this training. S3. When there is a new client that has not participated in training in a communication round, the server sends the vision transformer global model and its pre-trained weights to each client; S4. When a new domain incremental task appears, the server dynamically expands the domain expert knowledge network of the global model, including the domain expert knowledge module and classification head in each block in the model feature extraction part; S5. The client participating in the task training extracts the features of the local corresponding data through the feature extractor of the pre-trained vision transformer model that has been loaded locally in the first round of communication with the server, and uses mini-batch gradient descent to update the local cosine feature center; Assuming that the number of samples in a batch is m, the loss function used to update the local cosine feature center is The definition is as follows: in represents the cosine similarity between the feature extraction part f(·) of the model and the cosine feature center c; S6, the client downloads the domain expert knowledge network corresponding to this domain increment task from the server. During the training process, the basic model weights of the vision transformer are frozen, and only the domain expert knowledge module and the classification head are trained; The cross entropy loss function used in training is defined as follows: Among them, m represents the size of mini-batch, that is, the number of samples in one training iteration, C represents the number of categories of the target task, and y i,c represents the true label of the ith sample, indicating whether the ith sample belongs to category c. Indicates the probability that the model predicts category c for the i-th sample; S7. The client uploads the domain expert knowledge network obtained through local training to the server for global aggregation. Assuming that a total of K clients are selected to participate in the training in this round, the amount of local data about task t of the kth client is The updated local model domain expert knowledge network weight is The server then performs weighted aggregation on the weights according to the following formula: is the domain expert knowledge network weight of the global model obtained after this round of aggregation for the tth task; S8, repeating steps S6 to S7 until the iteration termination requirement of this domain increment task is met, the server counts the cosine feature centers of all client local data that have participated in this task training, calculates the global cosine feature center, and stores it in the global model; Assume that there are N clients participating in the training of task t, then: In the formula, represents the cosine feature center of the i-th client during task t training, represents the amount of local data related to the i-th client during task t training, c t represents the global feature center of task t; S9. After the incremental learning of the federated domain is completed, the server saves the global cosine feature center and domain expert knowledge network of the current domain incremental task, and restarts execution from step S4 when a new domain incremental task appears.

2. The federated domain incremental learning method based on pre-trained ViT and parameter adaptation according to claim 1, characterized in that: The domain expert knowledge module described in step S4 is composed of two linear layers. In order to reduce the size of the model parameters that need to be trained, two smaller linear layer matrices are multiplied instead of a single linear layer matrix to perform domain adaptation adjustment on the initial weight W0 of each block in the pre-trained vision transformer, thereby reducing the model parameters without sparsification.

3. The federated domain incremental learning method based on pre-trained ViT and parameter adaptation according to claim 2, characterized in that: The expression for domain adaptation adjustment of the initial weight W0 of each block in the pre-trained vision transformer is: Y=W0X+W(W0X)=W0X+(BA)(W0X) Among them, X represents the input of a block of the model, Y represents the output of the corresponding block of the model, and A and B represent two smaller linear matrices; In this method, the compressed linear layers remain fully differentiable and all parameters are trainable.

4. The federated domain incremental learning method based on pre-trained ViT and parameter adaptation according to claim 1, characterized in that: Step S4 means that when a new domain increment task appears, assuming that there are K clients participating in the first communication round, if the global model of the central server already has the domain expert knowledge network obtained in the previous training task, these K clients first send the trained local cosine feature center to the server, and then the server calculates the average cosine feature center c weighted by the client data amount according to the following formula t0 : In the formula, represents the cosine feature center of the kth client in the first communication round during task t training, represents the size of the local data of the kth client in the first communication round during the training of task t, c t0 represents the mean cosine feature center of the clients participating in the first communication round of task t; The server then calculates the cosine distance between the current center and the existing domain cosine feature center, selects the domain expert knowledge network corresponding to the closest center, and initializes the global model domain weight for this task, thereby accelerating convergence through knowledge transfer. The formula is as follows: in is the cosine feature center set of the global model processed by the central server at the t-1th task, c j for The cosine feature center of the jth task in * An index of the domain expert knowledge network for the selected domain.

5. The federated domain incremental learning method based on pre-trained ViT and parameter adaptation according to claim 1 or 4, characterized in that: For clients that are not in the first communication round, this method needs to calculate the local cosine feature center, but it does not need to be sent to the server separately. It only needs to be uploaded when the local weight is uploaded to the server for the first time.

6. The federated domain incremental learning method based on pre-trained ViT and parameter adaptation according to claim 1, characterized in that: In step S6, the client only trains the domain expert knowledge module and the classification head, and freezes other parameters; In step S7, the client needs to download the complete model to the local computer when communicating with the server for the first time. In subsequent communication rounds, the client does not need to upload or download the complete model.

7. The federated domain incremental learning method based on pre-trained ViT and parameter adaptation according to claim 1, characterized in that: When using the server's global model for inference, since the domain ID is unknown, the basic pre-trained feature extractor of the model is first used to extract the features of the sample, and the cosine distance between the extracted sample features and all current cosine feature centers is calculated. Then, the cosine feature center that is closest to the current sample features in the domain is selected: Where f(·) represents the feature extraction part of the model, C global is the set of all cosine feature centers of the model, c j C global The cosine feature center of the jth task in * Web indexing of expert knowledge for selected domains; For each inference sample x, the selected domain expert knowledge network is used to infer the sample to obtain the final output result.

8. The federated domain incremental learning method based on pre-trained ViT and parameter adaptation according to claim 1, characterized in that: This method freezes the basic weights of the pre-trained VisionTransformer model, builds an independent domain expert knowledge network for each domain, designs a domain knowledge extractor and a federated training strategy; When a new task appears, a new domain expert knowledge network is dynamically added to the server's global model structure and sent to the clients participating in the new task training; The client trains the domain expert knowledge network consisting of the domain expert knowledge module and the classification head locally, and participates in the construction of the global model domain knowledge extractor by training the cosine feature center of the local data; The constructed domain knowledge extractor is able to autonomously select domain expert knowledge network in the inference stage according to the cosine similarity between sample features and domain center.

Images