Collaborative distillation personalized federal learning method for multi-user semantic communication

Abstract

The invention discloses a collaborative distillation personalized federal learning method for multi-user semantic communication, and belongs to the technical field of artificial intelligence and wireless communication. The semantic consistency in a task group is ensured through a clustering method, meanwhile, personalized training is performed on a semantic encoder in the task group, the adaptability of a semantic communication model is improved in combination with a partial parameter sharing strategy, and finally, global semantic knowledge is shared through inter-group collaborative knowledge distillation; the method comprises the following specific steps: clustering and grouping clients participating in training, constructing a hierarchical semantic communication model, performing hierarchical personalized federated learning training in a task group, and performing knowledge sharing based on collaborative distillation to obtain a semantic communication model of collaborative distillation personalized federated learning. The method can accurately reflect preferences, and is high in individuation performance, high in semantic recovery accuracy, high in semantic communication model generalization ability, high in data processing heterogeneity and high in task diversity.

Description

Technical Field

[0001] The present invention relates to a collaborative distillation personalized federated learning method for multi-user semantic communication, which is suitable for collaborative optimization of data heterogeneity and task preference differences in complex multi-user scenarios and belongs to the field of artificial intelligence and wireless communication technology. Technical Background

[0002] As traditional signal transmission approaches the Shannon limit, semantic communication has emerged as an emerging communications paradigm. Semantic communication focuses on data transmission at the semantic level, rather than relying solely on precise signal transmission. By combining semantic coding with channel coding, semantic communication extracts and compresses key semantic information within the data, significantly reducing the transmission of redundant data. Compared to traditional communication, semantic communication offers significant advantages in reducing transmission overhead, optimizing bandwidth utilization, improving communication efficiency, and enhancing system robustness, particularly in complex communication environments.

[0003] In complex network environments, it is increasingly common for multiple users to access the same central node. However, traditional semantic codec training methods require the aggregation of each user's data to a central node, which not only poses a risk of privacy leakage but also faces bottlenecks in communication efficiency. The federated learning architecture provides an effective solution to this problem, but in multi-user scenarios, data heterogeneity and task diversity issues gradually become prominent. Traditional federated learning methods such as FedAvg have difficulty adapting to the non-independent and identically distributed characteristics of client data and the differences in task objectives. Improved federated learning methods such as FedProx and MOON introduce regularization or model comparison mechanisms, but optimization strategies for differences in task preferences are still insufficient.

[0004] In summary, how to solve the performance bottleneck faced by the application of federated learning methods in complex multi-user scenarios and achieve efficient task collaboration and precise personalized optimization has become an urgent problem to be solved. Summary of the Invention

[0005] In order to solve the problems of data heterogeneity and task preference differences in multi-user semantic communication scenarios, the purpose of this invention is to propose a collaborative distillation personalized federated learning method for multi-user semantic communication. By combining a hierarchical personalization strategy with a global knowledge sharing method, the trained semantic communication model maintains global consistency while enhancing the adaptability of the personalized model.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] The present invention proposes a collaborative distillation personalized federated learning method for multi-user semantic communication. This method uses a clustering method to address the differences between different tasks, ensuring semantic consistency within the task group. It also performs personalized training on the semantic encoder within the task group and combines a partial parameter sharing strategy to improve the adaptability of the semantic communication model under different tasks and data distributions. Finally, it implements global semantic knowledge sharing through collaborative knowledge distillation between groups. The method specifically includes the following steps:

[0008] Step 1: Cluster the clients participating in the training, which includes the following sub-steps:

[0009] Step 1.1: Construct the task preference vector of each client;

[0010] The task preference vector is obtained by the ratio of the weight value of each client on each semantic task type to the total weight value of the client on all tasks;

[0011] Step 1.2: Divide the client task groups;

[0012] Based on the client task preference vector, the K-means clustering algorithm is used to divide clients with consistent task preferences into the same task group;

[0013] Step 2: Construct a hierarchical semantic communication model, which includes the following sub-steps:

[0014] The semantic communication model includes a semantic encoder and a semantic decoder;

[0015] Step 2.1: Semantic encoder decomposition;

[0016] The semantic encoder is decomposed into a shared semantic encoder and a private semantic encoder. The shared semantic encoder includes a 4-layer convolutional neural network to extract common features of cross-client data, while the private semantic encoder includes a 1-layer convolutional neural network that is optimized only for local data features.

[0017] Step 2.2: Set up the semantic decoder;

[0018] The semantic decoder is set as a global shared semantic decoder, which consists of a 5-layer deconvolutional network to achieve global semantic consistency reconstruction;

[0019] Step 3: Conduct hierarchical personalized federated learning training within the task group, which includes the following sub-steps:

[0020] Step 3.1: Locally train each client;

[0021] Each client performs local training on the semantic communication model based on the local training dataset, using the mean square error (MSE) as the loss function.

[0022] Step 3.2: Update the layer parameters;

[0023] After each client completes local training, it uploads the parameters of the shared semantic encoder and shared semantic decoder to the central server, and keeps the parameters of the private semantic encoder locally;

[0024] Step 3.3: Aggregate parameters within the group;

[0025] The central server collects the shared parameters uploaded by the client according to the task group, performs weighted average aggregation on the parameters of the same task group, and sends the updated parameters to the clients in the group according to the task group.

[0026] Step 3.4: Update client parameters;

[0027] The client receives the shared parameters and updates the parameters of its own shared semantic encoder and shared semantic decoder;

[0028] Step 3.5: Execute steps 3.1 to 3.4 in a loop iteration manner until the specified number of rounds is reached;

[0029] Step 4: Conduct knowledge sharing based on collaborative distillation;

[0030] Step 4.1: Build a lightweight mapping module;

[0031] The central server adds a mapping module to the shared semantic codec model of each task group. The mapping module includes a layer of convolutional neural network and a layer of ReLU activation function.

[0032] Combining each group's shared semantic encoder-decoder model with the mapping module yields a complete shared semantic communication model for each group;

[0033] Step 4.2: Construct global feature center;

[0034] The central server uses the public dataset as the input dataset to input each group's complete shared semantic communication model, obtains the output of each group, and uses the mean of the output of each group as the global feature center;

[0035] Step 4.3: Perform feature alignment between groups;

[0036] The central server trains the complete shared semantic communication model of each group based on a public dataset. The loss function is the L2 norm between the output of each group's semantic communication model and the feature center. The central server trains the semantic communication model of each group by minimizing this loss function to ensure the consistency of the shared semantic communication model of each group in terms of global knowledge.

[0037] Step 4.4: Execute the training process of step 4.3 in a loop iterative manner until the semantic communication models of each group converge;

[0038] The central server sends the shared semantic communication model parameters after training convergence to the clients in the group according to the corresponding task group. After receiving them, each client updates its own shared semantic communication model parameters.

[0039] Step 5: Repeat step 3 and step 4 every 10 times until the semantic communication model training of all clients reaches convergence, and a semantic communication model of collaborative distillation personalized federated learning is obtained;

[0040] It also includes step six: The semantic communication model obtained based on the above steps can be used as a benchmark configuration for multi-user communication deployment. With the help of a personalized semantic communication model framework, the multi-user communication deployment solution can adapt to the data heterogeneity characteristics in multi-user scenarios and solve the industry problem of insufficient model generalization capabilities in heterogeneous data environments.

[0041] Beneficial effects:

[0042] 1. The collaborative distillation personalized federated learning method for multi-user semantic communication of the present invention constructs a task preference vector and determines the task preference vector by the ratio of the semantic task weight of each client to the total weight, accurately reflects the preference, provides a basis for clustering grouping, and improves the personalized performance of the algorithm.

[0043] 2. The present invention proposes a collaborative distillation personalized federated learning method for multi-user semantic communication, which decomposes the semantic encoder into a shared semantic encoder and a private semantic encoder, and sets the semantic decoder to be globally shared, taking into account both the commonality and individuality of data, improving the ability of the semantic communication model to process data from different clients, and improving the accuracy of semantic recovery in a non-independent and identically distributed data environment.

[0044] 3. The present invention provides a collaborative distillation personalized federated learning method for multi-user semantic communication. Through the collaborative distillation mechanism, the central server ensures the global knowledge consistency of the shared semantic communication model through mapping modules, global feature center construction and semantic communication model training, thereby enhancing the generalization ability of the semantic communication model.

[0045] 4. The collaborative distillation personalized federated learning method for multi-user semantic communication of the present invention integrates personalized federated learning and distributed multi-task learning, and uses a hierarchical personalization strategy combined with a global knowledge sharing method, which has significant advantages in handling data heterogeneity and task diversity. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of a collaborative distillation personalized federated learning method for multi-user semantic communication according to the present invention;

[0047] Figure 2 This is an overall framework diagram of a collaborative distillation personalized federated learning method for multi-user semantic communication according to the present invention;

[0048] Figure 3 This is a comparison chart of test results based on the peak signal-to-noise ratio indicator in the embodiment;

[0049] Figure 4 3 is a comparison chart of the test results based on the multi-scale structural similarity index in the embodiment. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to have a deeper understanding of the implementation ideas of the solutions of the present invention, the technical solutions in the embodiments of the present invention will be described in detail and clearly in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other implementation cases obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention. The specific steps of the embodiments of the present invention will be described below in conjunction with specific scenarios.

[0051] Example 1:

[0052] The application environment of the embodiment is a server, and the specific configuration of the server is: GPU NVIDIA GeForceRTX 309024GB, operating system is Ubuntu 20.04;

[0053] Based on the above server architecture, a collaborative distillation personalized federated learning method for multi-user semantic communication of the present invention is applied to realize the construction of semantic communication model, such as Figure 1 As shown, the following steps are included:

[0054] Step 1: Cluster the clients participating in the training, which includes the following sub-steps:

[0055] Step 1.1: Construct the task preference vector of each client;

[0056] Label each client's participation frequency in N semantic tasks, and obtain the task preference vector by the ratio of each client's weight value in each semantic task type to the client's total weight value in all tasks;

[0057] Step 1.2: Divide the client task groups;

[0058] Based on the client task preference vector, the K-means clustering algorithm is used to divide clients with consistent task preferences into the same task group;

[0059] In the embodiment, the number of cluster centers K=5, and the task preference similarity threshold ∈=0.85;

[0060] Step 2: Construct a hierarchical semantic communication model, which includes the following sub-steps:

[0061] The semantic communication model includes a semantic encoder and a semantic decoder;

[0062] Step 2.1: Semantic encoder decomposition;

[0063] Decompose the semantic encoder into a shared semantic encoder and a private semantic encoder;

[0064] The shared semantic encoder consists of a four-layer convolutional neural network. Each layer has a convolution kernel size of 5×5, 3, 16, 32, and 32 channels, with a stride of 2 or 1. It uses PReLU as the activation function and uses Kaiming initialization for weight optimization to extract common features across client data. Each convolution layer uses padding = 2 to maintain spatial consistency of the feature map.

[0065] The private semantic encoder consists of a single convolutional neural network with a kernel size of 5×5, 32 channels, a stride of 1, and a PReLU activation function, optimized only for local data features.

[0066] The feature output of the semantic encoder is mapped to a 2c-channel semantic representation through the last convolutional layer, where c is the channel bandwidth configuration parameter;

[0067] Step 2.2: Set up the semantic decoder;

[0068] The semantic decoder is set as a global shared semantic decoder, which consists of a 5-layer deconvolution network. The number of channels of the deconvolution layer is 2c, 32, 32, 32, 16, and 3, respectively. The convolution kernel size is 5×5, and the stride is 2 or 1. The last layer uses the Sigmoid activation function, and the other layers use the PReLU activation function. The weights are optimized through Kaiming initialization to gradually restore the original image resolution, so as to achieve global semantic consistency reconstruction.

[0069] Step 3: Conduct hierarchical personalized federated learning training within the task group, which includes the following sub-steps:

[0070] Step 3.1: Locally train each client;

[0071] Each client performs local training on the semantic communication model based on the local training dataset, using the mean square error (MSE) as the loss function, the Adam optimizer, and the learning rate set to 3×10 -4 , weight decay is set to 5×10-3 ;

[0072] Step 3.2: Update the layer parameters;

[0073] Each round of federated training sets the number of local iterations to 5 and the batch size to 32. After local training, each client uploads the parameters of the shared semantic encoder and shared semantic decoder to the central server, while retaining the parameters of the private semantic encoder locally.

[0074] Step 3.3: Aggregate parameters within the group;

[0075] The central server collects the shared parameters uploaded by the client according to the task group, performs weighted average aggregation on the parameters of the same task group, and sends the updated parameters to the clients in the group according to the task group.

[0076] Step 3.4: Update client parameters;

[0077] The client receives the shared parameters and updates the parameters of its own shared semantic encoder and shared semantic decoder;

[0078] Step 3.5: Repeat steps 3.1 to 3.4 in a loop until 10 rounds are completed.

[0079] Step 4: Conduct knowledge sharing based on collaborative distillation;

[0080] Step 4.1: Build a lightweight mapping module;

[0081] The central server adds a mapping module to the shared semantic codec model of each task group. The mapping module includes a layer of convolutional neural network and a layer of ReLU activation function. The kernel size of the convolutional neural network layer is 1×1 and the number of channels is 32.

[0082] Combining each group's shared semantic encoder-decoder model with the mapping module yields a complete shared semantic communication model for each group;

[0083] Step 4.2: Construct global feature center;

[0084] The central server uses the public dataset as the input dataset to input each group's complete shared semantic communication model, obtains the output of each group, and uses the mean of the output of each group as the global feature center;

[0085] Step 4.3: Perform feature alignment between groups;

[0086] The central server trains the complete shared semantic communication model of each group based on a public dataset. The loss function is the L2 norm between the output of each group's semantic communication model and the feature center. The central server trains the semantic communication model of each group by minimizing this loss function to ensure the consistency of the shared semantic communication model of each group in terms of global knowledge.

[0087] The optimizer for training is SGD, and the learning rate is set to 1×10 -3 , the momentum is set to 0.9;

[0088] Step 4.4: Execute the training process of step 4.3 in a loop iterative manner until the semantic communication models of each group converge;

[0089] In the embodiment, the maximum number of training rounds is set to 10, and the training is stopped early when the number of training rounds reaches 10;

[0090] The central server sends the shared semantic communication model parameters after training convergence to the clients in the group according to the corresponding task group. After receiving them, each client updates its own shared semantic communication model parameters.

[0091] Step 5: Repeat step 3 and step 4 every 10 times, and set the total number of training rounds to 200. If the average loss of the semantic communication model within the group decreases by less than 1×10 -4 , or when the maximum total training rounds are reached, the training is completed and the semantic communication model of collaborative distillation personalized federated learning is obtained, such as Figure 2 As shown;

[0092] Each client uses a local test dataset to perform a performance test on the trained semantic communication model based on the client's task preference indicators, and compares it with the traditional federated learning algorithm. Figure 3 As shown in Figure 2, the peak signal-to-noise ratio of the proposed method is improved by 1.2 dB compared with the traditional algorithm. Figure 4 As shown in the figure, the method of the present invention improves the multi-scale structural similarity index by 0.8 compared with the traditional algorithm. The semantic communication model of the method of the present invention can significantly improve the accuracy of semantic recovery in a non-independent and identically distributed data environment.

[0093] It also includes step six: The semantic communication model obtained based on the above steps can be used as a benchmark configuration for multi-user communication deployment. With the help of a personalized semantic communication model framework, the multi-user communication deployment solution can adapt to the data heterogeneity characteristics in multi-user scenarios and solve the industry problem of insufficient model generalization capabilities in heterogeneous data environments.

[0094] The above specific description further illustrates the purpose, technical solutions and beneficial effects of the invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A collaborative distillation personalized federated learning method for multi-user semantic communication, characterized by: Clustering methods are used to handle the differences between different tasks and ensure semantic consistency within the task group. At the same time, personalized training of the semantic encoder is performed within the task group, and combined with partial parameter sharing strategies to ensure the improvement of the adaptability of the semantic communication model under different tasks and data distributions. Finally, global semantic knowledge sharing is achieved through inter-group collaborative knowledge distillation.

2. The collaborative distillation personalized federated learning method for multi-user semantic communication according to claim 1, characterized in that: The specific steps include: Step 1: Cluster the clients participating in the training; Step 2: Build a hierarchical semantic communication model; Step 3: Conduct hierarchical personalized federated learning training within the task group; Step 4: Conduct knowledge sharing based on collaborative distillation; Step 5: Repeat step 3 and step 4 every 10 times until the semantic communication model training of all clients reaches convergence, and a semantic communication model of collaborative distillation personalized federated learning is obtained.

3. The collaborative distillation personalized federated learning method for multi-user semantic communication according to claim 2, characterized in that: Step 1 specifically includes the following sub-steps: Step 1.1: Construct the task preference vector of each client; The task preference vector is obtained by the ratio of the weight value of each client on each semantic task type to the total weight value of the client on all tasks; Step 1.2: Divide the client task groups; Based on the client task preference vector, clients with consistent task preferences are divided into the same task group.

4. The collaborative distillation personalized federated learning method for multi-user semantic communication according to claim 2, characterized in that: Step 2 specifically includes the following sub-steps: Step 2.1: Semantic encoder decomposition; Decompose the semantic encoder into a shared semantic encoder and a private semantic encoder; Step 2.2: Set up the semantic decoder; The semantic decoder is set as a global shared semantic decoder to achieve global semantic consistency reconstruction.

5. The collaborative distillation personalized federated learning method for multi-user semantic communication according to claim 2, characterized in that: Step 3 specifically includes the following sub-steps: Step 3.1: Locally train each client; Each client performs local training on the semantic communication model based on the local training dataset, using the mean square error (MSE) as the loss function. Step 3.2: Update the layer parameters; After each client completes local training, it uploads the parameters of the shared semantic encoder and shared semantic decoder to the central server, and keeps the parameters of the private semantic encoder locally; Step 3.3: Aggregate parameters within the group; The central server collects the shared parameters uploaded by the client according to the task group, performs weighted average aggregation on the parameters of the same task group, and sends the updated parameters to the clients in the group according to the task group. Step 3.4: Update client parameters; The client receives the shared parameters and updates the parameters of its own shared semantic encoder and shared semantic decoder; Step 3.5: Execute steps 3.1 to 3.4 in a loop iteration manner until the specified number of rounds is reached.

6. The collaborative distillation personalized federated learning method for multi-user semantic communication according to claim 2, characterized in that: Step 4 specifically includes the following sub-steps: Step 4.1: Build a lightweight mapping module; The central server adds a mapping module to the shared semantic codec model of each task group. The mapping module includes a layer of convolutional neural network and a layer of ReLU activation function. Combining each group's shared semantic encoder-decoder model with the mapping module yields a complete shared semantic communication model for each group; Step 4.2: Construct global feature center; The central server uses the public dataset as the input dataset to input each group's complete shared semantic communication model, obtains the output of each group, and uses the mean of the output of each group as the global feature center; Step 4.3: Perform feature alignment between groups; The central server trains the complete shared semantic communication model of each group based on a public dataset. The loss function is the L2 norm between the output of each group's semantic communication model and the feature center. The central server trains the semantic communication model of each group by minimizing this loss function to ensure the consistency of the shared semantic communication model of each group in terms of global knowledge. Step 4.4: Execute the training process of step 4.3 in a loop iterative manner until the semantic communication models of each group converge; The central server sends the parameters of the shared semantic communication model after training convergence to the clients in the group according to the corresponding task group. After receiving it, each client updates the parameters of its own shared semantic communication model.

7. The collaborative distillation personalized federated learning method for multi-user semantic communication according to claim 3, characterized in that: The K-means clustering algorithm is used to divide clients with consistent task preferences into the same task group.

8. The collaborative distillation personalized federated learning method for multi-user semantic communication according to claim 4, characterized in that: The semantic communication model includes a semantic encoder and a semantic decoder; The shared semantic encoder consists of a 4-layer convolutional neural network to extract common features across client data; The semantic decoder consists of a 5-layer deconvolutional network; The private semantic encoder consists of a 1-layer convolutional neural network optimized only for local data features.

9. The collaborative distillation personalized federated learning method for multi-user semantic communication according to any one of claims 2 to 8, characterized in that: Based on the semantic communication model obtained in steps one to five as the benchmark configuration for multi-user communication deployment, with the help of a personalized semantic communication model framework, the multi-user communication deployment solution can adapt to the data heterogeneity characteristics in multi-user scenarios and solve the problem of insufficient model generalization ability in heterogeneous data environments.

Images