Semantic communication model training method based on personalized federal learning

Abstract

The invention relates to a semantic communication model training method based on personalized federal learning. The training method comprises the following steps: S1, constructing an SC model architecture based on GAI; s2, performing local training on the personalized semantic communication model and the student model through a knowledge distillation technology; s3, uploading the student model parameters to a central server for parameter aggregation; s4, dynamically adjusting the aggregated student model parameters according to a real-time communication environment through a network pruning technology; and S5, downloading the updated student model parameters to each user device, updating the local student model, and continuing training. According to the invention, a knowledge distillation technology is introduced into local training of federated learning, so that heterogeneous user equipment can more flexibly select a personalized SC model for local training, and model parameters are adaptively adjusted according to a real-time communication environment by using a network pruning technology in a global aggregation stage. And the effective balance between the model precision and the communication overhead is realized.

Description

Technical Field

[0001] The present invention relates to the field of semantic communication, and in particular to a semantic communication model training method based on personalized federated learning. Background Art

[0002] With the rapid development of 6G communication technology, SC, as a new communication paradigm, has gradually become one of the intelligent solutions to solve the problem of spectrum resource shortage of user equipment. Unlike traditional communication methods, SC emphasizes the transmission of semantically extracted key information rather than complete data content according to specific tasks or goals. This difference enables SC to significantly reduce the consumption of spectrum resources while greatly reducing the amount of transmitted data. For example, in a fault detection scenario, the mobile user first extracts key information from the monitoring video through the SC encoder (including the semantic encoder and the channel encoder), and only transmits a small amount of semantic information to the edge server on the base station. The received semantic information is then processed and reconstructed by the SC decoder (including the semantic decoder and the channel decoder). By reducing the amount of data transmission, SC significantly reduces the consumption of spectrum resources and improves communication efficiency.

[0003] The performance of SC is highly dependent on the construction of high-quality models. However, most current SC models based on discriminative AI focus on learning local and short-term features, and are prone to falling into local minima and limited generation capabilities. In addition, such models lack adaptability in different scenarios and cannot fully cope with diverse application requirements. Moreover, the training method of the model is also crucial, especially when data resources are scattered on different user devices. Traditional centralized learning methods require users' local data to be transmitted to a central server for centralized training, which not only leads to an increase in communication energy consumption, but also poses a risk of data privacy leakage. To solve these problems, federated learning (FL) has emerged as a distributed learning method. FL achieves collaborative training between multiple clients and a central server by sharing model parameters instead of raw data, avoiding large-scale data transmission. However, traditional FL still has many limitations in dealing with the heterogeneity of user devices and the balance between communication overhead and model accuracy in a dynamic network environment.

[0004] Therefore, the application of SC on edge user devices faces three major challenges: (1) Insufficient semantic model extraction capability: Assuming that the data type transmitted between the user device and the base station is an image, an image-oriented SC system needs to be deployed. Although the traditional convolutional neural network (CNN) performs well in capturing local features of images, it is difficult to effectively obtain global semantic information. Therefore, the image semantic encoder and decoder built based on CNN are insufficient in extracting high-quality image semantic features. (2) Model adaptability of heterogeneous devices: Due to the heterogeneity of user devices, they have local data and computing resources of different scales. On devices with sufficient data and rich computing resources, complex models can provide higher accuracy. Devices with limited resources can only be trained with simpler models. This difference causes devices with sufficient resources to have to use simple models in order to adapt to the model isomorphism requirements in federated learning, thereby compromising the accuracy and performance of the model. (3) High communication overhead in dynamic networks: Traditional FL algorithms perform distributed learning by exchanging only model parameters. Although this method can ensure data privacy and reduce the risk of data transmission, frequent model parameter exchanges will bring a lot of communication overhead. Some existing communication-efficient FL methods usually reduce the amount of transmission by compressing parameters, thereby reducing communication energy consumption. However, the compression operation in these methods usually needs to be performed on the user device, resulting in additional computational burden. At the same time, these methods often ignore the impact of dynamic network conditions (such as changes in signal-to-noise ratio in wireless communications) on communication performance, and it is difficult to effectively cope with real-time fluctuations in the network environment. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a semantic communication model training method based on personalized federated learning, which solves the shortcomings of the prior art.

[0006] The object of the present invention is achieved by the following technical solution: a semantic communication model training method based on personalized federated learning, the training method comprising: S1. Build the SC model architecture based on GAI; S2, local training of personalized semantic communication model and student model through knowledge distillation technology; S3, upload the student model parameters to the central server for parameter aggregation; S4, dynamically adjust the aggregated student model parameters according to the real-time communication environment through network pruning technology; S5. Download the updated student model parameters to each user device, update its local student model and continue training.

[0007] The S1 includes the following contents: A1. At the sending end, the classic GAI model is used as the image semantic encoder to extract the semantics of the input image, and the channel encoder built based on the deep neural network performs channel coding; A2. At the receiving end, the signal decoder demodulates the received signal to extract semantic features, and the semantic decoder decodes it to obtain a restored image; A3. Assume that the SC model is aimed at classification tasks, use the classifier to perform recognition based on the restored image, and use cross entropy as the loss function of the SC model.

[0008] The A1 includes the following: A11 will image Input into a patch embedding layer, Convert to size P 2 ×C, where H represents the height of the image, W represents the width of the image, C represents the number of channels, and P 2 represents the number of segments into which the image is divided, so ; A12. For the sequence composed of these N small blocks Perform patch embedding operation. Each small block in is linearly transformed to reduce the dimension of the sequence to D and generate a linear embedding sequence as , where E is a linear transformation, express The i-th small block in , i is a value from 1 to N; A13. Compare the position vector in the position encoding with Linear combination, the input sequence of ViT is , where P i represents the i-th position vector; A14, semantic encoder from Extract features and get the original image The semantic features of A15. To ensure data transmission on the physical channel, a DNN-based channel encoder is used for encoding and modulation to reduce channel damage and improve robustness.

[0009] The S2 includes the following contents: B1. The GAI-based SC model is used as the mentor model and the student model. The loss between the model output and the hard label is calculated. The input image of the k-th user device is recorded as , the corresponding hard label is , then the classification task communication loss functions of the tutor model and the student model are expressed as and ,in, and Represents the tutor model and the student model pair The prediction results, and Respectively represent the reconstructed images generated by the tutor model and the student model after passing through the physical channel, represents the mean square error, express The cross entropy loss between express The cross entropy loss between ; B2, the mentor model and the student model transfer knowledge to each other through soft labels, and the distillation loss is weighted according to the quality of the predicted soft labels; B3, the tutor model and the student model learn by minimizing the difference between the semantic encoder output and the channel decoder output, and weighting the semantic loss according to the task loss; B4. Update the total loss function of the tutor model and the student model according to the weighted distillation loss and semantic loss.

[0010] The S4 includes the following: Prune the aggregated global student model parameters, and prune a certain proportion of the smallest positive weights and the largest negative weights in the parameters, where the pruning ratio is adaptively adjusted according to the real-time signal-to-noise ratio between the user equipment and the base station in each round of communication; Compare the current pruning ratio with the previous round of pruning ratio. If the current pruning ratio is smaller than the previous round of pruning ratio, prune more student model parameters. Otherwise, randomly add new parameters until the current pruning ratio is equal to the previous round of pruning ratio.

[0011] The present invention has the following advantages: a semantic communication model training method based on personalized federated learning, firstly, GAI is combined with SC to design a more efficient SC model, which provides a better communication experience for user devices; secondly, knowledge distillation technology is introduced in the local training of federated learning to ensure that heterogeneous user devices can more flexibly select personalized SC models for local training; finally, network pruning technology is used in the global aggregation stage to adaptively adjust model parameters according to the real-time communication environment, thereby achieving an effective trade-off between model accuracy and communication overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 Schematic diagram of the SC model architecture based on GAI. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application provided below in conjunction with the drawings is not intended to limit the scope of protection of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application. The present invention is further described below in conjunction with the drawings.

[0014] like Figure 1 As shown, the present invention specifically relates to a semantic communication model training method based on personalized federated learning, which aims to solve the key problems existing in the application of semantic communication on user devices, including "insufficient extraction capability of semantic models", "model adaptation of heterogeneous devices" and "high communication overhead in dynamic networks", and specifically includes the following contents.

[0015] S1. Build a semantic communication model architecture based on GAI; S2, introduce knowledge distillation technology to perform local training on personalized semantic communication model and student model; S3, upload the student model parameters to the central server for parameter aggregation. This process can be expressed as: , in represents the student model uploaded by the kth user device in the tth round of communication, N k represents the amount of data on the kth user device, and K represents the total number of user devices.

[0016] S4, introduce network pruning technology to dynamically adjust the parameters of the aggregated student model according to the real-time communication environment; S5. Download the updated student model parameters to each user device, update its local student model and continue training.

[0017] Furthermore, if Figure 2 As shown, step S1 specifically includes the following contents: S11. At the transmitting end, the classic GAI model - Vision Transformer (ViT) is used as the image semantic encoder to perform semantic extraction on the input image, and the channel encoder built based on DNN performs channel coding.

[0018] S12. When the signal is transmitted through the physical channel, Affected by the channel, including attenuation and noise. The transmission process of the physical channel can be expressed as: , in Indicates the received signal. represents the channel gain between the transmitter and the receiver, is additive white Gaussian noise.

[0019] S13. At the receiving end, the channel decoder demodulates the received signal to extract semantic features, which are then decoded by the semantic decoder. Restored image It can be obtained by the following formula: , in Represents a model with parameters A channel decoder; Represents a model with parameters Semantic decoder.

[0020] S14. Assuming that the SC model is for classification tasks, the classifier can be used based on Recognition is performed and cross entropy is used as the loss function of the SC model: , in Represents the category label. When the input image When it belongs to the i-th category, y i =1, otherwise 0; Indicates based on The predicted probability distribution, represents the probability of predicting the i-th category; M is the total number of categories.

[0021] Furthermore, step S11 specifically includes the following contents: S111, the image Input into a PatchEmbed layer, is converted to a size of (P 2 ×C), where H represents the height of the image, W represents the width of the image, C represents the number of channels, and P 2 represents the number of segments the image is divided into. Therefore, .

[0022] S112, for the sequence composed of these N small blocks Perform Patch Embedding operations. Specifically, Each small block in is linearly transformed to reduce the dimension of the sequence to D and generate a linear embedding sequence: , Where E is a linear transformation (i.e., a fully connected layer), and the input dimension is (P 2 ×C), the output dimension is N. express The i-th small block in .

[0023] S113, the position vector in the position encoding and Linear combination, we get the input sequence of ViT: , Where P i represents the i-th position vector.

[0024] S114, semantic encoder from Extract features and get the original image The core of ViT is the multi-head attention layer, which can learn the relationship between pixels through the multi-head attention mechanism. Therefore, the image semantic encoder based on ViT can achieve more accurate feature extraction. The multi-head attention layer is essentially composed of multiple self-attention heads. The self-attention head comes from a single self-attention layer, and its calculation formula is: , in is the query vector, is with The corresponding matching vector, is the information vector. , and All through input The linear transformation of , , ,in , and are weight matrices respectively. k is the scaling factor. Assuming the number of heads is h, multi-head self-attention can be calculated by the following formula: , in Represents the weights of the multi-head attention layer.

[0025] S115. To ensure that data can be transmitted on the physical channel, a DNN-based channel encoder is used for encoding and modulation to reduce channel damage and improve robustness. The transmission signal can be expressed as: , in Represents a model with parameters The semantic encoder is a model with parameters The channel encoder.

[0026] Furthermore, step S2 specifically includes the following contents: S21, the GAI-based SC model is used as the mentor model and the student model to calculate the loss between the model output and the hard label. Usually, the hard label is determined by the specific task, and in the classification task, it refers to the category of the input image. The input image of the kth user device is denoted as , the corresponding hard label is , then the classification task communication loss functions of the tutor model and the student model are expressed as: , , in and Represents the tutor model and the student model pair The prediction results, express The cross entropy loss between express The cross entropy loss between . Similarly, and They respectively represent the reconstructed images generated by the tutor model and the student model after passing through the physical channel. Represents the mean square error, which is used to ensure the consistency between the original image and the reconstructed image at the pixel level.

[0027] S22, the tutor model and the student model are connected by soft labels (e.g. and ) transfer knowledge to each other. Since incorrect predictions of the mentor or student model may mislead each other during the knowledge transfer process, the present invention weights the distillation loss according to the quality of the predicted soft labels: , , in Denotes the Kullback–Leibler divergence. In this way, if the predictions of the tutor and student models are unreliable (e.g., the task loss is large), the distillation strength will be weak. When the tutor and student models are well tuned, i.e., the task loss is small, the distillation loss will dominate. Therefore, adaptive distillation loss helps reduce the risk of overfitting.

[0028] S23, the tutor model and the student model learn by minimizing the difference between the output of the semantic encoder and the output of the channel decoder. Similarly, to avoid misleading the tutor / student model during interaction, the semantic loss is also weighted according to the task loss: , in and Represent the semantic encoding of the tutor model and the student model respectively, and Represent the channel decoding results of the tutor model and the student model respectively.

[0029] S24. According to the above-mentioned loss function, the total loss functions of updating the tutor model and the student model are expressed as: , , The tutor and student models update parameter weights using a stochastic gradient descent optimizer by minimizing the total loss function.

[0030] Furthermore, step S4 specifically includes the following contents: S41, pruning the aggregated global student model parameters, wherein a portion of the smallest positive weights and the largest negative weights in the parameters will be pruned. In order to avoid weakening the knowledge carried by the student model due to excessive pruning of parameters, the pruning ratio in the present invention is adaptively adjusted according to the real-time signal-to-noise ratio between the user equipment and the base station in each round of communication. Therefore, the pruning ratio in the tth round of communication can be expressed as: , in, represents the channel signal-to-noise ratio when the k-th user equipment transmits parameters to the base station in the t-th round of federated learning, represents the set of user devices, K represents the number of user devices, and Indicates the maximum and minimum signal-to-noise ratio between the user equipment and the base station.

[0031] S42, compare the current and previous round pruning ratios, if , then cut off more student model parameters, otherwise randomly add new parameters until until.

[0032] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used for various other combinations, modifications and improvements, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art do not deviate from the spirit and scope of the present invention, and should be within the scope of protection of the claims attached to the present invention.

Claims

1. A semantic communication model training method based on personalized federated learning, characterized by: The training method comprises: S1. Build the SC model architecture based on GAI; S2, local training of personalized semantic communication model and student model through knowledge distillation technology; S3, upload the student model parameters to the central server for parameter aggregation; S4, dynamically adjust the aggregated student model parameters according to the real-time communication environment through network pruning technology; S5. Download the updated student model parameters to each user device, update its local student model and continue training.

2. The method for training a semantic communication model based on personalized federated learning according to claim 1, characterized in that: The S1 includes the following contents: A1. At the sending end, the classic GAI model is used as the image semantic encoder to extract the semantics of the input image, and the channel encoder built based on the deep neural network performs channel coding; A2. At the receiving end, the signal decoder demodulates the received signal to extract semantic features, and the semantic decoder decodes it to obtain a restored image; A3. Assume that the SC model is aimed at classification tasks, use the classifier to perform recognition based on the restored image, and use cross entropy as the loss function of the SC model.

3. The method for training a semantic communication model based on personalized federated learning according to claim 2, characterized in that: The A1 includes the following: A11 will image Input into a patch embedding layer, Convert to size P 2 ×C, where H represents the height of the image, W represents the width of the image, C represents the number of channels, and P 2 represents the number of segments into which the image is divided, so ; A12. For the sequence composed of these N small blocks Perform patch embedding operation. Each small block in is linearly transformed to reduce the dimension of the sequence to D and generate a linear embedding sequence as , where E is a linear transformation, express The i-th small block in , i is a value from 1 to N; A13. Compare the position vector in the position encoding with Linear combination, the input sequence of ViT is , where P i represents the i-th position vector; A14, semantic encoder from Extract features and get the original image The semantic features of A15. To ensure data transmission on the physical channel, a DNN-based channel encoder is used for encoding and modulation to reduce channel damage and improve robustness.

4. The method for training a semantic communication model based on personalized federated learning according to claim 1, characterized in that: The S2 includes the following contents: B1. The GAI-based SC model is used as the mentor model and the student model. The loss between the model output and the hard label is calculated. The input image of the k-th user device is recorded as , the corresponding hard label is , then the classification task communication loss functions of the tutor model and the student model are expressed as and ,in, and Represents the tutor model and the student model pair The prediction results, and Respectively represent the reconstructed images generated by the tutor model and the student model after passing through the physical channel, represents the mean square error, express The cross entropy loss between express The cross entropy loss between ; B2, the mentor model and the student model transfer knowledge to each other through soft labels, and the distillation loss is weighted according to the quality of the predicted soft labels; B3, the tutor model and the student model learn by minimizing the difference between the semantic encoder output and the channel decoder output, and weighting the semantic loss according to the task loss; B4. Update the total loss function of the tutor model and the student model according to the weighted distillation loss and semantic loss.

5. The method for training a semantic communication model based on personalized federated learning according to claim 1, characterized in that: The S4 includes the following: Prune the aggregated global student model parameters, and prune a certain proportion of the smallest positive weights and the largest negative weights in the parameters, where the pruning ratio is adaptively adjusted according to the real-time signal-to-noise ratio between the user equipment and the base station in each round of communication; Compare the current pruning ratio with the previous round of pruning ratio. If the current pruning ratio is smaller than the previous round of pruning ratio, prune more student model parameters. Otherwise, randomly add new parameters until the current pruning ratio is equal to the previous round of pruning ratio.

Images