Semantic communication system and method
Patent Information
- Application Number
- PCT/CN2025/117938
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-04-14
- Filing Date
- 2025-08-29
- Publication Date
- 2026-10-22
Smart Images
Figure CN2025117938_22102026_PF_FP_ABST
Abstract
Description
Semantic communication systems and methods
[0001] Related applications
[0002] This application claims priority to Chinese patent application No. 202510463948.9, filed on April 14, 2025, the entire contents of which are incorporated herein by reference. Technical Field
[0003] This application relates to the field of intelligent communication technology, and in particular to a semantic communication system and method. Background Technology
[0004] Currently, the training process of most semantic communication systems is usually carried out in a centralized environment, which makes data collection extremely expensive, while the data collection process is crucial for creating a centrally stored training dataset.
[0005] However, in real-world applications, centralized approaches often pose a risk of user privacy data leakage, as user data may be exposed to external parties. Given the growing concern about data privacy, this drawback is becoming increasingly unfavorable. Summary of the Invention
[0006] The main purpose of this application is to provide a semantic communication system and method, which aims to solve the technical problem of how to achieve privacy-preserving distributed semantic communication in multi-user semantic communication scenarios.
[0007] To achieve the above objectives, this application provides a semantic communication system, which includes: a source-channel joint encoder, a source-channel joint decoder, and a server. The source-channel joint encoder includes a primary semantic encoder, a shared semantic encoder, and a private semantic encoder. The source-channel joint decoder includes a primary semantic decoder, a shared semantic decoder, a private semantic decoder, and a noise reduction adapter.
[0008] The source-channel joint encoder is used to extract shared semantic features and private semantic features of the input data through the primary semantic encoder, the shared semantic encoder, and the private semantic encoder.
[0009] The source-channel joint decoder is used to jointly decode the received shared semantic features and the private semantic features through the shared semantic decoder and the private semantic decoder to obtain the decoded features;
[0010] The source-channel joint decoder is further configured to process the decoded features through the noise-resistant adapter to obtain processed features, and to fuse the processed features through the primary semantic decoder to obtain reconstructed data;
[0011] The server is used to receive updated shared semantic encoder parameters and updated shared semantic decoder parameters obtained by each user terminal based on the reconstructed data;
[0012] The server is also configured to aggregate global shared parameters based on the updated shared semantic encoder parameters and the updated shared semantic decoder parameters using a federated averaging algorithm, and distribute the global shared parameters to each user terminal for semantic communication.
[0013] In one embodiment, the source-channel joint encoder is used to extract basic semantic features from the input data through the primary semantic encoder to obtain primary semantic features;
[0014] The source-channel joint encoder is also used to process the primary semantic features through the shared semantic encoder to obtain shared semantic features common to multiple users.
[0015] The source-channel joint encoder is also used to process the primary semantic features through the private semantic encoder to obtain user-specific private semantic features.
[0016] In one embodiment, the source-channel joint decoder is further configured to decode the shared semantic features through the shared semantic decoder to obtain a multi-user common decoded shared semantic feature;
[0017] The source-channel joint decoder is also used to decode the private semantic features through the private semantic decoder to obtain user-specific decoded private semantic features.
[0018] In one embodiment, the source-channel joint decoder is further configured to perform nonlinear activation processing on the decoded features through the first processing layer of the noise-resistant adapter to obtain noise-resistant features;
[0019] The source-channel joint decoder is also used to apply classification constraints to the noise-resistant features through the second processing layer of the noise-resistant adapter to obtain the processed features;
[0020] The source-channel joint decoder is further configured to fuse the processed features using the primary semantic decoder to obtain fused features;
[0021] The source-channel joint decoder is further used to perform a linear transformation on the fused features through the fully connected layer of the primary semantic decoder to obtain reconstructed data.
[0022] In one embodiment, the semantic communication system further includes a local training controller, which is configured to determine the mean square error of the reconstructed data and the input data;
[0023] The local training controller is further configured to generate a total loss by weighted summation of the mean squared error, cross-entropy loss, and separation loss;
[0024] The local training controller is further configured to determine the parameter gradients of the shared semantic encoder and the shared semantic decoder based on the total loss using a backpropagation algorithm.
[0025] The local training controller is also used to update the local shared semantic encoder parameters and shared semantic decoder parameters according to the parameter gradient based on the federated learning algorithm, so as to obtain the updated shared semantic encoder parameters and updated shared semantic decoder parameters.
[0026] The local training controller is further configured to upload the updated shared semantic encoder parameters and the updated shared semantic decoder parameters to the server if the local training rounds reach a preset number of training rounds.
[0027] In one embodiment, the local training controller is further configured to obtain a separation loss by determining the orthogonality constraints of shared semantic features and private semantic features;
[0028] The local training controller is also used to obtain cross-entropy loss by determining the difference between the classifier output of the noise-resistant adapter and the true label.
[0029] Furthermore, to achieve the above objectives, this application also provides a semantic communication method applied to a semantic communication system. The system includes: a source-channel joint encoder, a source-channel joint decoder, and a server. The source-channel joint encoder includes a primary semantic encoder, a shared semantic encoder, and a private semantic encoder. The source-channel joint decoder includes a primary semantic decoder, a shared semantic decoder, a private semantic decoder, and a noise-resistant adapter. The method includes:
[0030] The source-channel joint encoder extracts shared semantic features and private semantic features of the input data through the primary semantic encoder, the shared semantic encoder, and the private semantic encoder;
[0031] The source-channel joint decoder uses the shared semantic decoder and the private semantic decoder to jointly decode the received shared semantic features and the private semantic features to obtain the decoded features;
[0032] The source-channel joint decoder processes the decoded features through the noise-resistant adapter to obtain processed features, and then fuses the processed features through the primary semantic decoder to obtain reconstructed data;
[0033] The server receives the shared semantic encoder parameters and shared semantic decoder parameters updated by each user terminal based on the reconstructed data;
[0034] The server aggregates globally shared parameters based on the shared semantic encoder parameters and the shared semantic decoder parameters using a federated averaging algorithm, and distributes the globally shared parameters to each user terminal for semantic communication.
[0035] In one embodiment, the source-channel joint encoder extracts shared semantic features and private semantic features of the input data through the primary semantic encoder, the shared semantic encoder, and the private semantic encoder, including:
[0036] The source-channel joint encoder extracts basic semantic features from the input data through the primary semantic encoder to obtain primary semantic features;
[0037] The source-channel joint encoder processes the primary semantic features through the shared semantic encoder to obtain shared semantic features applicable to multiple users;
[0038] The source-channel joint encoder processes the primary semantic features through the private semantic encoder to obtain user-specific private semantic features.
[0039] In one embodiment, the step of the source-channel joint decoder performing joint decoding on the received shared semantic features and the private semantic features through the shared semantic decoder and the private semantic decoder to obtain the decoded features includes:
[0040] The source-channel joint decoder decodes the shared semantic features through the shared semantic decoder to obtain shared semantic features that are common to multiple users after decoding.
[0041] The source-channel joint decoder decodes the private semantic features through the private semantic decoder to obtain user-specific decoded private semantic features.
[0042] In one embodiment, the source-channel joint decoder processes the decoded features through the noise-resistant adapter to obtain processed features, and then fuses the processed features through the primary semantic decoder to obtain reconstructed data, comprising the following steps:
[0043] The source-channel joint decoder performs non-linear activation processing on the decoded features through the first processing layer of the noise-resistant adapter to obtain noise-resistant features.
[0044] The source-channel joint decoder applies classification constraints to the noise-resistant features through the second processing layer of the noise-resistant adapter to obtain the processed features.
[0045] The source-channel joint decoder fuses the processed features through the primary semantic decoder to obtain fused features;
[0046] The source-channel joint decoder performs a linear transformation on the fused features through a fully connected layer of the primary semantic decoder to obtain reconstructed data.
[0047] In addition, to achieve the above objectives, this application also proposes a semantic communication device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the semantic communication method as described above.
[0048] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the semantic communication method described above.
[0049] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the semantic communication method described above.
[0050] This application discloses a semantic communication system and method. The system includes a source-channel joint encoder, a source-channel joint decoder, and a server. The source-channel joint encoder includes a primary semantic encoder, a shared semantic encoder, and a private semantic encoder. The source-channel joint decoder includes a primary semantic decoder, a shared semantic decoder, a private semantic decoder, and a noise-resistant adapter. The source-channel joint encoder extracts shared and private semantic features from the input data. The source-channel joint decoder performs joint decoding and fusion of the above features to obtain reconstructed data. The server receives the updated shared semantic encoder / decoder parameters obtained based on the reconstructed data and aggregates the globally shared parameters using a federated averaging algorithm for semantic communication. The model decoupling architecture formed in this application improves semantic understanding capabilities while ensuring user data privacy, achieving efficient and secure multi-user semantic communication. Attached Figure Description
[0051] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0052] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0053] Figure 1 is a structural block diagram of the first embodiment of the semantic communication system of this application;
[0054] Figure 2 is a schematic diagram of the multi-user semantic communication model framework based on federated learning in this application.
[0055] Figure 3 is a schematic diagram of the multi-user semantic communication transmission process;
[0056] Figure 4 is a schematic diagram of a complete decoupled semantic communication model for a user;
[0057] Figure 5 is a flowchart illustrating the first embodiment of the semantic communication method of this application;
[0058] Figure 6 is a schematic diagram of the device structure of the hardware operating environment involved in the semantic communication method in the embodiments of this application.
[0059] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0060] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not intended to limit this application.
[0061] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.
[0062] In practical applications, centralized methods often pose a risk of user privacy data leakage, as user data may be exposed to external parties. Given the increasing concerns about data privacy, this drawback is becoming increasingly unfavorable.
[0063] Therefore, to overcome the aforementioned shortcomings, this application provides a solution that extracts shared and private semantic features from the input data through a source-channel joint encoder. A source-channel joint decoder then jointly decodes and fuses these features to obtain reconstructed data. The server receives updated shared semantic encoder-decoder parameters based on the reconstructed data and aggregates globally shared parameters using a federated averaging algorithm for semantic communication. The decoupled model architecture developed in this application enhances semantic understanding capabilities while protecting user data privacy, achieving efficient and secure multi-user semantic communication.
[0064] Based on this, this application provides a semantic communication system. Referring to Figure 1, which is a structural block diagram of the first embodiment of the semantic communication system of this application, the semantic communication system includes: a source-channel joint encoder 10, a source-channel joint decoder 20, and a server 30. The source-channel joint encoder 10 includes a primary semantic encoder, a shared semantic encoder, and a private semantic encoder. The source-channel joint decoder 20 includes a primary semantic decoder, a shared semantic decoder, a private semantic decoder, and a noise reduction adapter.
[0065] The source-channel joint encoder 10 is used to extract shared semantic features and private semantic features of the input data through the primary semantic encoder, the shared semantic encoder, and the private semantic encoder.
[0066] In practical semantic applications, there is a need for communication among multiple semantic users. In multi-user semantic communication systems, to ensure communication between users, participating semantic users need to share a single model. However, due to the differences in data among different users, the ideal approach to ensure effective model training is to use as much user data as possible for training, allowing the model to better understand the characteristics of user data and thus ensure good performance. However, in reality, data is distributed across individual users' local machines, making centralized collection relatively difficult, and incremental training is required as the data type increases. Furthermore, centralized training requires uploading all user data to a central server for processing, which not only introduces significant privacy risks but also degrades model performance due to the heterogeneity of user data. In addition, traditional communication systems often focus only on the physical layer recovery of signals during wireless channel transmission, neglecting semantic noise interference, leading to semantic information distortion during transmission.
[0067] To address the aforementioned issues, this application proposes a semantic communication system based on federated learning. This system (model) consists of three main parts: a joint source-channel encoder (JSCE), a joint source-channel decoder (JSCD), and a server. A semantic communication system model with distributed training is designed, which has the advantages of improving the model encoding and decoding capabilities of each user, reducing network resource consumption, protecting the data privacy and security of participating users, and enabling mutual communication between semantic users.
[0068] Multi-user federated learning scenarios present challenges such as non-independent and identically distributed data and high communication costs. To address these challenges and achieve distributed training of semantic communication models, this application implements a federated learning deployment framework for semantic communication systems based on model decoupling (as shown in Figure 2, which is a schematic diagram of the multi-user semantic communication model framework based on federated learning in this application). In this framework, each user (user 1 to user K) employs a joint source-channel codec and decouples it into a shared part and a private part. In each iteration of the training process, each user only uploads the model parameters of the shared part to the server. In federated scenarios, data heterogeneity can lead to a severe degradation in model performance; the above method effectively solves this problem. Since only the shared part of the model parameters is uploaded during training, compared to traditional methods that upload complete model parameters, the amount of data uploaded and updated for parameters is reduced, improving model training efficiency. Furthermore, a noise-resistant adapter is added at the receiving end to help maintain the semantic discriminativeness of the received information with channel noise, thereby enhancing the robustness of the model.
[0069] It should be understood that this step first involves multi-level feature extraction of the input data through a joint source-channel encoder. The primary semantic encoder is responsible for extracting basic semantic features from the original input data (such as images and text), outputting primary semantic features. The shared semantic encoder then extracts universally applicable shared semantic features from the primary features, enabling semantic alignment across different users. Simultaneously, a private semantic encoder specifically extracts user-specific private semantic features to preserve the uniqueness of local data. This decoupled encoder design effectively solves the two major challenges of data privacy protection and heterogeneous data processing in traditional centralized semantic communication.
[0070] The source-channel joint decoder 20 is used to jointly decode the received shared semantic features and the private semantic features through the shared semantic decoder and the private semantic decoder to obtain the decoded features.
[0071] The source-channel joint decoder 20 is further configured to process the decoded features through the noise-resistant adapter to obtain processed features, and then fuse the processed features through the primary semantic decoder to obtain reconstructed data.
[0072] It should be noted that during signal transmission, the encoded shared semantic features and private semantic features are transmitted to the receiving end via a wireless channel. Due to noise interference in the wireless channel, the received signal will contain channel noise. In existing technologies, decoders typically decode the received signal directly, which leads to severe distortion of semantic information in noisy environments.
[0073] To address this, this application introduces a noise-resistant adapter in the joint source-channel decoder. This adapter first decodes the received shared and private semantic features separately to obtain the decoded features. Then, through dual processing of nonlinear transformation and semantic classification constraints, it effectively suppresses noise interference and enhances the robustness of the semantic features. The processed features are then fused and reconstructed by the primary semantic decoder, ultimately outputting high-quality reconstructed data.
[0074] The server 30 is used to receive the updated shared semantic encoder parameters and the updated shared semantic decoder parameters obtained by each user terminal based on the reconstructed data.
[0075] The server 30 is further configured to aggregate global shared parameters using a federated averaging algorithm based on the updated shared semantic encoder parameters and the updated shared semantic decoder parameters, and distribute the global shared parameters to each user terminal for semantic communication.
[0076] The server employs a federated averaging (FedAvg) algorithm to weighted aggregate the shared parameters uploaded by all users, with the weights dynamically adjusted based on the amount of data from each user. The resulting globally shared parameters are then distributed to all participating users to update their local shared models. This distributed training approach not only avoids direct sharing of raw data but also allows the model to gradually learn more generalizable shared semantic features through multiple iterations.
[0077] Understandably, this application employs a federated learning training paradigm to train the semantic communication system. The federated averaging algorithm utilizes input data from distributed users for model training, thereby avoiding the leakage of privacy-sensitive data. The core idea of this framework is to send the learned model to user sites for local training, and then aggregate the model training parameters from different mobile users onto a server for parameter averaging to update the global model parameters. This entire process constitutes a communication round or global round, which is repeated until the model reaches the desired performance level or a specific termination condition. Under this setting, multiple users need to collaborate to train a semantic communication system model. Simultaneously, thanks to the participation of a large amount of local data from users in the joint training of the model, mutual understanding among users is effectively promoted.
[0078] For ease of understanding, please refer to Figures 3 and 4 for explanation, but this is not intended to limit the semantic communication system of this application. Figure 3 is a schematic diagram of the multi-user semantic communication transmission process, and Figure 4 is a schematic diagram of the complete decoupled semantic communication model for a single user.
[0079] Suppose we have a set of data used in training a semantic communication system. From K users. Among them, It is the kth local dataset on the user's end (the dataset contains n) k For each data pair, x represents an image, y represents the label corresponding to the image, and the total number of data is...
[0080] In the federated semantic communication task, all user terminals obtain randomly initialized source-channel joint encoders and decoders from the server. Each user terminal trains the semantic communication model and updates its parameters. For each user terminal, the input image x∈R 3×h×w Where h represents height and w represents width. The source-channel joint encoder fed into the user terminal extracts semantic information. The encoded symbol stream can be represented as:
[0081] In the formula, f e Indicates joint coding of source and channel. v represents the feature dimension. Encoded semantic information. (That is, shared semantic features and private semantic features) are transmitted through a wireless channel and have channel noise added to them. This wireless transmission process can be represented as:
[0082] In the formula, H represents the channel coefficient; n represents Gaussian noise. Then, Y (shared semantic features and private semantic features after adding channel noise) is fed into the source-channel joint decoder. The decoder decodes Y into an approximate original image. This process is expressed as:
[0083] In the formula, This indicates a joint source-channel decoder.
[0084] The Federated Semantic Communication task aims to collaboratively train a semantic communication model using local data from multiple user terminals in a distributed manner. Each user terminal uploads its locally trained model parameters and receives aggregated parameters from a server (the server aggregates the parameters uploaded by all user terminals to obtain aggregated parameters) to update its own model parameters. This model allows each user terminal to leverage knowledge learned from other user terminals to improve the performance of its local model without needing to collect local datasets from each user terminal beforehand for centralized training. The overall optimization objective is as follows:
[0085] In the formula, θ represents the trainable local model parameters (including shared model parameters and private model parameters); p θ This represents the mean square error function.
[0086] In federated semantic communication tasks, different user terminals exhibit noise and data type differences, rendering traditional federated semantic communication methods inapplicable. To facilitate the learning of data-independent and data-related features, source-channel joint coding and decoding are redefined in a decoupled manner. The source-channel joint encoder is designed to consist of a primary semantic encoder, a shared semantic encoder, and a private semantic encoder, as shown below:
[0087] In the formula, X represents the semantic features extracted by the primary semantic encoder. in, This represents a primary semantic encoder; This represents a shared semantic encoder that maps the input semantic features X to a data-difference-independent feature subspace. This refers to a private semantic encoder that calculates the specific features of the current user client. Finally, semantic information will be shared. and private semantic information All data is transmitted to the receiving end via wireless channels.
[0088] During wireless transmission, due to the influence of noise, the process can be represented as:
[0089] In the formula, Indicates shared semantic features; Represents private semantic features; Y s Y represents shared semantic information after adding noise; p This represents the private semantic information after adding noise; n and n0 represent different noise levels.
[0090] At the receiving end, the joint source-channel decoder is designed to consist of a primary semantic decoder, a shared semantic decoder, a private semantic decoder, and a noise-resistant adapter (classifier), as shown below:
[0091] In the formula, This represents the primary semantic decoder. Indicates a noise-canceling adapter. Indicates a shared semantic decoder, This indicates a private semantic decoder.
[0092] Assume that all user terminals upload the parameters of the primary semantic codec, the shared semantic encoder, and the shared semantic decoder to the server, while retaining the parameters of the private semantic encoder and the private semantic decoder. In this process, the functions defined in equations (5), (6), and (7) are instantiated into a model-decoupled federated semantic communication network. Figure 4 illustrates the proposed framework for solving the federated semantic communication task. On each local user terminal, the model learns the semantic communication system model based on the data difference-independent features and data-specific features obtained from the private and shared semantic codecs, respectively. The semantic information after wireless channel simulation contains noise; for the decoded semantic information, it is input to a noise-resistant adapter. The noise-resistant adapter consists of two layers and is connected to a classifier, which constrains the discriminativeness of the decoded features. The server receives updated local shared model parameters from different user terminals and obtains the global shared model through weighted aggregation.
[0093] This embodiment extracts shared and private semantic features from the input data through a source-channel joint encoder. A source-channel joint decoder then jointly decodes and fuses these features to obtain reconstructed data. The server receives the updated shared semantic encoder-decoder parameters based on the reconstructed data and aggregates the globally shared parameters using a federated averaging algorithm for semantic communication. The decoupled model architecture developed in this application enhances semantic understanding capabilities while protecting user data privacy, achieving efficient and secure multi-user semantic communication.
[0094] In one implementation, the source-channel joint encoder 10 is used to extract basic semantic features from the input data through the primary semantic encoder to obtain primary semantic features.
[0095] Understandably, the source-channel joint encoder achieves multi-level feature extraction and separation of input data through a three-level coding architecture. First, the primary semantic encoder performs basic semantic parsing on the original input data (such as images or text). It can use feature extraction networks such as convolutional neural networks or Transformers (deep learning model architecture) to map the original data into low-dimensional, dense primary semantic features. These features retain the core semantic information of the input data, providing a unified representation space for subsequent feature decoupling.
[0096] The source-channel joint encoder 10 is also used to process the primary semantic features through the shared semantic encoder to obtain shared semantic features applicable to multiple users.
[0097] Understandably, the shared semantic encoder further processes the primary semantic features, extracting shared semantic features with cross-user universality from them through specific feature projection layers and attention mechanisms. These features are optimized by specially designed learning objectives to capture common semantic patterns among different user data, ensuring semantic interoperability among multiple users. During feature generation, the shared semantic encoder can employ training strategies such as orthogonal constraints to actively eliminate correlations with user-private features.
[0098] The source-channel joint encoder 10 is also used to process the primary semantic features through the private semantic encoder to obtain user-specific private semantic features.
[0099] Understandably, the private semantic encoder processes the same set of primary semantic features in parallel, extracting user-specific private semantic features through independent neural network branches. The private semantic encoder, through differentiated feature activation modes and dedicated normalization layer settings, focuses on preserving personalized semantic information relevant to a specific user in the input data. To ensure thorough feature decoupling, an asymmetric network structure design can be adopted, and collaborative optimization can be performed during training using a dedicated separation loss function.
[0100] It should be understood that during the feature output stage, shared semantic features and private semantic features are processed through different transmission channels. Shared features are fed into the subsequent federated learning parameter update process to participate in global model optimization; while private features are strictly limited to local use to ensure the privacy and security of user data. This parallel feature processing mechanism enables the system to simultaneously meet the dual needs of multi-user semantic collaboration and personalized semantic preservation. The entire encoding process adopts an end-to-end training approach, with the parameters of each encoder level being collaboratively optimized through joint backpropagation to ensure the efficiency and accuracy of feature extraction.
[0101] The source-channel joint decoder 20 is also used to decode the shared semantic features through the shared semantic decoder to obtain shared semantic features that are common to multiple users after decoding.
[0102] Understandably, the source-channel joint decoder employs a dual-path decoding architecture to accurately recover transmitted semantic features. The shared semantic decoder specifically handles shared semantic features from the wireless channel. It progressively reconstructs the feature space dimension through deconvolutional networks or feature upsampling layers, and performs feature calibration using a cross-user semantic dictionary, outputting decoded shared semantic features with universal semantic representation capabilities. This decoding process particularly emphasizes preserving cross-user common information in the features, selectively enhancing key semantic elements through a pre-trained attention weight matrix to ensure consistent understanding of the same semantic concept across different user devices.
[0103] The source-channel joint decoder 20 is also used to decode the private semantic features through the private semantic decoder to obtain user-specific decoded private semantic features.
[0104] Understandably, the private semantic decoder employs an independent decoding path to process private semantic features. Its network structure includes a user-specific parameterization layer and feature modulation module, enabling it to dynamically adjust the decoding strategy based on locally stored user preference profiles. Through cascaded feature refinement units, it gradually recovers the personalized semantic details in the original data, outputting the decoded private semantic features. To ensure the exclusivity of feature decoding, the private semantic decoder and the shared semantic decoder use completely isolated parameter spaces and feature processing flows, and feature decoupling constraints are applied during backpropagation.
[0105] Before feature fusion, the decoded shared and private semantic features are scaled through a feature normalization layer to ensure numerical stability during subsequent fusion. The system maintains independent quality assessment modules for the two types of features, monitoring the fidelity of feature decoding in real time and automatically triggering a retransmission mechanism when feature distortion is detected. The entire decoding process adopts a progressive precision design concept, first establishing a basic semantic framework through the shared semantic decoder, then filling in personalized semantic details with the private semantic decoder, ultimately providing structurally complete and information-complementary dual-channel feature input for subsequent feature fusion.
[0106] The source-channel joint decoder 20 is also used to perform nonlinear activation processing on the decoded features through the first processing layer of the noise-resistant adapter to obtain noise-resistant features.
[0107] It should be understood that the source-channel joint decoder achieves the recovery and reconstruction of anti-noise semantic features through a multi-level processing flow. The first processing layer of the anti-noise adapter uses a fully connected network with a ReLU activation function (Linear Rectification Function) to perform a nonlinear transformation on the decoded features. Through feature space mapping and compression, it effectively suppresses high-frequency interference components introduced by channel noise, generating anti-noise features with stronger robustness. This processing layer is specially designed with an adaptive feature scaling mechanism, which can dynamically adjust the feature enhancement intensity according to the real-time channel state, ensuring a stable noise suppression effect under different signal-to-noise ratio conditions.
[0108] The source-channel joint decoder 20 is also used to apply classification constraints to the noise-resistant features through the second processing layer of the noise-resistant adapter to obtain the processed features.
[0109] Understandably, the second processing layer of the noise-reducing adapter integrates a lightweight classifier module to impose semantic discriminative constraints on the denoised features. This layer calculates the probability distribution of features across semantic categories using Softmax normalization and compares them with the true labels based on the cross-entropy criterion, outputting semantically calibrated processed features. The classifier uses a learnable weight matrix to project features into the semantic space, and its parameters are co-optimized with the decoder's main network during training, preserving the communicative effectiveness of features while enhancing the accuracy of semantic representation.
[0110] The source-channel joint decoder 20 is also used to fuse the processed features through the primary semantic decoder to obtain fused features.
[0111] The source-channel joint decoder 20 is also used to perform a linear transformation on the fused features through the fully connected layer of the primary semantic decoder to obtain reconstructed data.
[0112] Understandably, after receiving noise-resistant features, the primary semantic decoder achieves deep fusion of shared and private features through feature concatenation and channel attention mechanisms. The fused features contain both general semantic information across users and retain personalized detailed features. The fully connected layer at the end of the primary semantic decoder performs linear transformations and dimensionality adjustments on the fused features, ultimately outputting reconstructed data consistent with the original input data space. This fully connected layer employs a residual connection structure, effectively mitigating the gradient vanishing problem during deep network training.
[0113] In this embodiment, the primary encoder reduces data dimensionality through unified basic feature extraction, the shared encoder focuses on multi-user common semantic features to enhance cross-user collaboration capabilities and reduce redundant information transmission, the private encoder retains user-specific features to achieve customized communication needs while ensuring privacy, the shared decoder ensures cross-user information consistency, the private decoder accurately restores personalized features, the primary decoder fuses shared / private semantic information, the fully connected layer optimizes feature expression dimensionality, the noise-resistant adapter uses nonlinear activation to suppress noise interference, and classification constraints further calibrate and reconstruct data distribution, effectively improving the anti-interference capability, data fidelity, and transmission efficiency of the communication system.
[0114] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment can be referred to the above description, and will not be repeated hereafter. Based on the embodiment shown in FIG1 above, a second embodiment of the semantic communication system of this application is proposed.
[0115] In this embodiment, the semantic communication system further includes a local training controller, which is used to determine the mean square error of the reconstructed data and the input data.
[0116] The local training controller is also used to generate a total loss by weighted summation of the mean squared error, cross-entropy loss, and separation loss.
[0117] It should be understood that the local training controller employs a multi-objective joint optimization approach to iteratively update model parameters. During the loss function calculation phase, the local training controller simultaneously processes three key loss metrics: it precisely quantifies the pixel-level difference between the reconstructed data and the original input using Mean-Square Error (MSE); it constrains the semantic discriminativeness of the noise-resistant adapter's output features using cross-entropy loss; and it maintains the orthogonality between shared and private features based on separation loss. These three types of losses are weighted and fused using learnable dynamic weight coefficients to form a comprehensively optimized total loss function, where the weight coefficients are adaptively adjusted according to the convergence state of each loss term.
[0118] The local training controller is further configured to determine the parameter gradients of the shared semantic encoder and the shared semantic decoder based on the total loss using a backpropagation algorithm.
[0119] Understandably, in the gradient calculation phase, the local training controller employs a backpropagation algorithm with momentum optimization, automatically differentiating the total loss along the computation graph to accurately calculate the gradient values of each parameter node in the shared semantic encoder and decoder. To improve training stability, a gradient pruning mechanism is introduced during the gradient calculation process to prevent training oscillations caused by excessively large parameter update steps. Simultaneously, the local training controller maintains a dedicated gradient history matrix, dynamically adjusting the learning rate strategy by analyzing gradient change trends.
[0120] The local training controller is further configured to update the local shared semantic encoder parameters and shared semantic decoder parameters based on the parameter gradient using a federated learning algorithm, thereby obtaining the updated shared semantic encoder parameters and updated shared semantic decoder parameters.
[0121] Understandably, the parameter update phase strictly follows the collaborative training paradigm of Federated Learning (FL). The local training controller inputs the calculated parameter gradients into the local optimizer, employing a conditional parameter update strategy: larger update steps are used to accelerate training for convergent and stable parameters, while update constraints are imposed on parameters with large fluctuations. The updated shared semantic encoder and shared semantic decoder parameters undergo performance evaluation on a local validation set; only parameters that pass the quality check are marked as valid updates.
[0122] The local training controller is further configured to upload the updated shared semantic encoder parameters and the updated shared semantic decoder parameters to the server if the local training rounds reach a preset number of training rounds.
[0123] In terms of training cycle management, the local training controller has a built-in intelligent epoch counter. When the accumulated local training epochs reach a preset threshold (e.g., 5-20 epochs), the parameter upload protocol is automatically triggered. Before uploading, the parameters undergo lightweight encryption and format standardization to ensure compatibility with the server-side federated aggregation module. Parameter transmission employs differential privacy technology, injecting a small amount of Gaussian-distributed noise into the shared parameters to further enhance privacy protection. The entire update process retains complete operation logs, including metadata such as parameter version numbers and training epoch fingerprints, providing the system with traceable training auditing capabilities.
[0124] This embodiment ensures data reconstruction accuracy by determining the mean squared error, enhances semantic classification capabilities by cross-entropy, promotes decoupling of shared / private features by separation loss, achieves end-to-end optimization by parameter update mechanism, and periodically aggregates global parameters by combining federated learning framework, which not only ensures personalized adaptation but also promotes multi-user collaborative evolution, forming an efficient training paradigm of "local optimization-global sharing" and improving model generalization and communication efficiency.
[0125] In one implementation, the local training controller is also configured to obtain a separation loss by determining the orthogonality constraints between shared semantic features and private semantic features.
[0126] Understandably, the local training controller constructs its loss function system through two dimensions: feature space analysis and semantic discriminative evaluation. In the separation loss calculation stage, the local training controller performs high-order statistical analysis on shared and private semantic features, quantifying their correlation through Gram matrix operations on the feature vectors. Specifically, this is achieved by calculating the Frobenius inner product of the two sets of feature matrices as an orthogonality metric. This process uses a sliding window sampling technique to dynamically select feature samples, ensuring that the evaluation results comprehensively reflect the feature decoupling state. The output value of the separation loss function directly characterizes the separation degree between shared and private features in the latent space. When information redundancy exists between features, this loss term generates a significant gradient signal, driving the encoder network to adjust its feature extraction strategy.
[0127] The local training controller is also used to obtain cross-entropy loss by determining the difference between the classifier output of the noise-resistant adapter and the true label.
[0128] It should be understood that during the cross-entropy loss calculation phase, the local training controller interfaces with the classifier module embedded in the noise-resistant adapter, and performs semantic discriminative evaluation through Softmax probability distribution comparison. Specifically, the controller first obtains the probability distribution of the classifier's predicted output for the processed features, performs element-wise logarithmic operations with the one-hot encoding of the ground truth labels, and then obtains the scalar loss value through a negative summation operation. This process specifically incorporates a class balancing weight mechanism, automatically adjusting the penalty intensity for semantic classes with unbalanced data distribution to prevent the dominant class from dominating the loss calculation. The local training controller synchronously monitors the classification confidence index; when it detects fuzziness in the prediction result, it automatically enhances the gradient feedback strength of the cross-entropy loss, forcing the noise-resistant adapter to improve the semantic discriminative ability of the features. Both loss calculations use double-precision floating-point operations to ensure numerical stability, and computation graph caching technology enables rapid backtracking and verification of historical states.
[0129] For ease of understanding, Figures 3 and 4 are used for illustration, but they are not intended to limit the semantic communication system of this application. As shown in the figures, the proposed model-decoupled federated semantic communication network consists of a primary semantic encoder, a shared semantic encoder, a shared semantic decoder, a private semantic encoder, a private semantic decoder, and a noise-resistant adapter. Since the shared semantic encoder and the private semantic encoder have data feature dependencies, semantic encoders with different architectures can be used. For the shared semantic encoder, the shared semantic decoder, the private semantic encoder, the private semantic decoder, and the noise-resistant adapter, they are defined as two-layer perceptrons with ReLU activation function and Softmax output. In addition, a classifier ψ is added to the noise-resistant adapter module to help the features after passing through the wireless channel maintain their semantic discriminativeness. The classifier is defined as follows: ψ(G)=Softmax(WTG) (8)
[0130] Assuming the linear transformation layer of the classifier has no bias term, its weight matrix is represented as W∈R v×M , where M represents the total number of categories.
[0131] A separation loss was set to encourage shared and private semantic encoders to produce distinct features representing different aspects of semantic features. Inspired by domain separation networks, a soft subspace orthogonality constraint was adopted:
[0132] In the formula, ‖·‖ F It is the Frobenius norm.
[0133] For the noise-resistant adapter output, an additional classifier was used for classification constraints, employing cross-entropy constraints: L CE=-y(logψ(G)) (10)
[0134] Minimize L CE It can help maintain discriminativeness through the semantic features of wireless communication.
[0135] The goal of image (data) reconstruction is to regenerate the image using a joint decoder on the source and channel. Therefore, the most common loss in this task is the mean square error calculated between these images, denoted as:
[0136] With this constraint, the reconstructed image can learn to explicitly reduce the difference from the original image.
[0137] This application employs server-wide global aggregation for collaborative optimization. Training is performed using two alternative steps: R rounds of local updates and global aggregation. In each round, the client receives the global model and updates its local model E rounds based on its local data. In the first round, the server sends randomly initialized primary semantic encoders, primary semantic decoders, shared semantic encoders, and shared semantic decoders to each client. Each client updates its local model based on its local data and then sends the parameters of the primary semantic encoder / decoder and shared encoder / decoder, which have been trained in one round, to the server. The server performs global aggregation on all parameters and then sends the aggregated parameters back to the client.
[0138] Regarding local updates: A distributed approach is used to jointly train the semantic communication model using local data from multiple users. This model allows different users to effectively leverage shared knowledge from other users to improve the performance of their local semantic communication models without needing to collect each user's local dataset for centralized training beforehand. Specifically, separation loss, classification constraints, and mean squared error loss are used to optimize the user's local model: L = λ1L s +λ2L CE +λ3L MSE (12)
[0139] In the formula, λ1, λ2, and λ3 represent the weighting coefficients.
[0140] All parameters collected from the user end are aggregated using the same weighted averaging scheme as in FedAvg. The specific aggregation method is shown below:
[0141] In the formula, w k This represents the parameters of the shared model uploaded by the Kth user. This represents the parameters after global aggregation.
[0142] Semantic communication, a new communication paradigm, can eliminate redundant data and reduce the amount of data transmitted, meeting the needs of intelligent interaction in the 6G era. For multi-user semantic communication scenarios, this application proposes a more energy-efficient and secure system. The semantic communication system is a federated learning-assisted semantic communication system model framework based on model decoupling. By splitting the joint encoding and decoding of the semantic user-end source channel into a shared model part and a private model part, and by using an adapter to weight the server's model parameters, better semantic communication system performance is achieved, and communication costs are reduced. The innovation of this application is:
[0143] 1. A general federated semantic communication framework based on model decoupling is proposed. In this framework, a joint source-channel codec is employed, decoupled into a shared model part and a private model part. In each iteration of training, each user only needs to upload the parameters of the shared model part to the server, while the server remains with zero data. Furthermore, this approach effectively addresses the performance degradation caused by heterogeneous data from different users.
[0144] 2. Since the data originates from the actual users who need to communicate, a large amount of local user data participates in the joint training of the model, which helps users understand the communication process. However, the source and channel joint codecs at each user end are decoupled. When uploading model parameters, users retain private parts of the model parameters, which prevents other participating users from using gradient information to reconstruct sensitive training data, thus improving the overall privacy and security of the system model.
[0145] 3. A noise-resistant adapter module was added to the model, which can effectively help maintain the discriminativeness of features after passing through the wireless channel and enhance the robustness of the system model.
[0146] In this embodiment, the local training controller constructs a separation loss through orthogonality constraints, which decouples shared features from private features and avoids information redundancy. Simultaneously, it optimizes the cross-entropy loss between the classification output of the noise-resistant adapter and the true label to improve the semantic recognition accuracy of the model in noisy environments. This dual constraint mechanism works synergistically to ensure the purity of the feature space and enhance system robustness, forming an efficient feature-classification joint optimization framework.
[0147] Referring to Figure 5, the semantic communication system of this application provides a semantic communication method. Figure 5 is a flowchart of the first embodiment of the semantic communication method of this application. The semantic communication system includes: a source-channel joint encoder, a source-channel joint decoder, and a server. The source-channel joint encoder includes a primary semantic encoder, a shared semantic encoder, and a private semantic encoder. The source-channel joint decoder includes a primary semantic decoder, a shared semantic decoder, a private semantic decoder, and a noise reduction adapter. The semantic communication method includes steps S10 to S50:
[0148] In step S10, the source-channel joint encoder extracts the shared semantic features and private semantic features of the input data through the primary semantic encoder, the shared semantic encoder, and the private semantic encoder.
[0149] In practical semantic applications, there is a need for communication among multiple semantic users. In multi-user semantic communication systems, to ensure communication between users, participating semantic users need to share a single model. However, due to the differences in data among different users, the ideal approach to ensure effective model training is to use as much user data as possible for training, allowing the model to better understand the characteristics of user data and thus ensure good performance. However, in reality, data is distributed across individual users' local machines, making centralized collection relatively difficult, and incremental training is required as the data type increases. Furthermore, centralized training requires uploading all user data to a central server for processing, which not only introduces significant privacy risks but also degrades model performance due to the heterogeneity of user data. In addition, traditional communication systems often focus only on the physical layer recovery of signals during wireless channel transmission, neglecting semantic noise interference, leading to semantic information distortion during transmission.
[0150] Based on the above problems, the technical solution of this application proposes a semantic communication system based on federated learning. The system (model) consists of three main parts: a source-channel joint encoder, a source-channel joint decoder, and a server. A semantic communication system model with a distributed training method is designed. This model has the advantages of improving the model encoding and decoding capabilities of each user, reducing network resource consumption, protecting the data privacy and security of participating users, and enabling mutual communication between semantic users.
[0151] Multi-user federated learning scenarios present challenges such as non-independent and identically distributed data and high communication costs. To address these challenges and achieve distributed training of semantic communication models, this application implements a federated learning deployment framework for semantic communication systems based on model decoupling (as shown in Figure 2, which is a schematic diagram of the multi-user semantic communication model framework based on federated learning in this application). In this framework, each user (user 1 to user K) employs a joint source-channel codec and decouples it into a shared part and a private part. In each iteration of the training process, each user only uploads the model parameters of the shared part to the server. In federated scenarios, data heterogeneity can lead to a severe degradation in model performance; the above method effectively solves this problem. Since only the shared part of the model parameters is uploaded during training, compared to traditional methods that upload complete model parameters, the amount of data uploaded and updated for parameters is reduced, improving model training efficiency. Furthermore, a noise-resistant adapter is added at the receiving end to help maintain the semantic discriminativeness of the received information with channel noise, thereby enhancing the robustness of the model.
[0152] It should be understood that this step first involves multi-level feature extraction of the input data through a joint source-channel encoder. The primary semantic encoder is responsible for extracting basic semantic features from the original input data (such as images and text), outputting primary semantic features. The shared semantic encoder then extracts universally applicable shared semantic features from the primary features, enabling semantic alignment across different users. Simultaneously, a private semantic encoder specifically extracts user-specific private semantic features to preserve the uniqueness of local data. This decoupled encoder design effectively solves the two major challenges of data privacy protection and heterogeneous data processing in traditional centralized semantic communication.
[0153] In step S20, the source-channel joint decoder performs joint decoding on the received shared semantic features and the private semantic features through the shared semantic decoder and the private semantic decoder to obtain the decoded features.
[0154] In step S30, the source-channel joint decoder processes the decoded features through the noise-resistant adapter to obtain processed features, and then fuses the processed features through the primary semantic decoder to obtain reconstructed data.
[0155] It should be noted that during signal transmission, the encoded shared semantic features and private semantic features are transmitted to the receiving end via a wireless channel. Due to noise interference in the wireless channel, the received signal will contain channel noise. In existing technologies, decoders typically decode the received signal directly, which leads to severe distortion of semantic information in noisy environments.
[0156] To address this, this application introduces a noise-resistant adapter in the joint source-channel decoder. This adapter first decodes the received shared and private semantic features separately to obtain the decoded features. Then, through dual processing of nonlinear transformation and semantic classification constraints, it effectively suppresses noise interference and enhances the robustness of the semantic features. The processed features are then fused and reconstructed by the primary semantic decoder, ultimately outputting high-quality reconstructed data.
[0157] In step S40, the server receives the updated shared semantic encoder parameters and the updated shared semantic decoder parameters obtained by each user terminal based on the reconstructed data.
[0158] In step S50, the server aggregates global shared parameters using a federated averaging algorithm based on the updated shared semantic encoder parameters and the updated shared semantic decoder parameters, and distributes the global shared parameters to each user terminal for semantic communication.
[0159] The server employs a federated averaging algorithm to weight and aggregate the shared parameters uploaded by all users, with the weights dynamically adjusted based on the amount of data from each user. The resulting globally shared parameters are then distributed to all participating users to update their local shared models. This distributed training approach not only avoids direct sharing of raw data but also allows the model to gradually learn more generalizable shared semantic features through multiple iterations.
[0160] Understandably, this application employs a federated learning training paradigm to train the semantic communication system. The federated averaging algorithm utilizes input data from distributed users for model training, thereby avoiding the leakage of privacy-sensitive data. The core idea of this framework is to send the learned model to user sites for local training, and then aggregate the model training parameters from different mobile users onto a server for parameter averaging to update the global model parameters. This entire process constitutes a communication round or global round, which is repeated until the model reaches the desired performance level or a specific termination condition. Under this setting, multiple users need to collaborate to train a semantic communication system model. Simultaneously, thanks to the participation of a large amount of local data from users in the joint training of the model, mutual understanding among users is effectively promoted.
[0161] This embodiment extracts shared and private semantic features from the input data through a source-channel joint encoder. A source-channel joint decoder then jointly decodes and fuses these features to obtain reconstructed data. The server receives the updated shared semantic encoder-decoder parameters based on the reconstructed data and aggregates the globally shared parameters using a federated averaging algorithm for semantic communication. The decoupled model architecture developed in this application enhances semantic understanding capabilities while protecting user data privacy, achieving efficient and secure multi-user semantic communication.
[0162] In one implementation, step S10 in this embodiment includes:
[0163] The source-channel joint encoder extracts basic semantic features from the input data through the primary semantic encoder to obtain primary semantic features;
[0164] The source-channel joint encoder processes the primary semantic features through the shared semantic encoder to obtain shared semantic features applicable to multiple users;
[0165] The source-channel joint encoder processes the primary semantic features through the private semantic encoder to obtain user-specific private semantic features.
[0166] Understandably, the source-channel joint encoder achieves multi-level feature extraction and separation of input data through a three-level coding architecture. First, the primary semantic encoder performs basic semantic parsing on the original input data (such as images or text). It can use feature extraction networks such as convolutional neural networks or Transformers (deep learning model architecture) to map the original data into low-dimensional, dense primary semantic features. These features retain the core semantic information of the input data, providing a unified representation space for subsequent feature decoupling.
[0167] Understandably, the shared semantic encoder further processes the primary semantic features, extracting shared semantic features with cross-user universality from them through specific feature projection layers and attention mechanisms. These features are optimized by specially designed learning objectives to capture common semantic patterns among different user data, ensuring semantic interoperability among multiple users. During feature generation, the shared semantic encoder can employ training strategies such as orthogonal constraints to actively eliminate correlations with user-private features.
[0168] Understandably, the private semantic encoder processes the same set of primary semantic features in parallel, extracting user-specific private semantic features through independent neural network branches. The private semantic encoder, through differentiated feature activation modes and dedicated normalization layer settings, focuses on preserving personalized semantic information relevant to a specific user in the input data. To ensure thorough feature decoupling, an asymmetric network structure design can be adopted, and collaborative optimization can be performed during training using a dedicated separation loss function.
[0169] It should be understood that during the feature output stage, shared semantic features and private semantic features are processed through different transmission channels. Shared features are fed into the subsequent federated learning parameter update process to participate in global model optimization; while private features are strictly limited to local use to ensure the privacy and security of user data. This parallel feature processing mechanism enables the system to simultaneously meet the dual needs of multi-user semantic collaboration and personalized semantic preservation. The entire encoding process adopts an end-to-end training approach, with the parameters of each encoder level being collaboratively optimized through joint backpropagation to ensure the efficiency and accuracy of feature extraction.
[0170] In this embodiment, step S20 includes:
[0171] The source-channel joint decoder decodes the shared semantic features through the shared semantic decoder to obtain shared semantic features that are common to multiple users after decoding.
[0172] The source-channel joint decoder decodes the private semantic features through the private semantic decoder to obtain user-specific decoded private semantic features.
[0173] Understandably, the source-channel joint decoder employs a dual-path decoding architecture to accurately recover transmitted semantic features. The shared semantic decoder specifically handles shared semantic features from the wireless channel. It progressively reconstructs the feature space dimension through deconvolutional networks or feature upsampling layers, and performs feature calibration using a cross-user semantic dictionary, outputting decoded shared semantic features with universal semantic representation capabilities. This decoding process particularly emphasizes preserving cross-user common information in the features, selectively enhancing key semantic elements through a pre-trained attention weight matrix to ensure consistent understanding of the same semantic concept across different user devices.
[0174] Understandably, the private semantic decoder employs an independent decoding path to process private semantic features. Its network structure includes a user-specific parameterization layer and feature modulation module, enabling it to dynamically adjust the decoding strategy based on locally stored user preference profiles. Through cascaded feature refinement units, it gradually recovers the personalized semantic details in the original data, outputting the decoded private semantic features. To ensure the exclusivity of feature decoding, the private semantic decoder and the shared semantic decoder use completely isolated parameter spaces and feature processing flows, and feature decoupling constraints are applied during backpropagation.
[0175] Before feature fusion, the decoded shared and private semantic features are scaled through a feature normalization layer to ensure numerical stability during subsequent fusion. The system maintains independent quality assessment modules for the two types of features, monitoring the fidelity of feature decoding in real time and automatically triggering a retransmission mechanism when feature distortion is detected. The entire decoding process adopts a progressive precision design concept, first establishing a basic semantic framework through the shared semantic decoder, then filling in personalized semantic details with the private semantic decoder, ultimately providing structurally complete and information-complementary dual-channel feature input for subsequent feature fusion.
[0176] In this embodiment, step S30 includes:
[0177] The source-channel joint decoder performs non-linear activation processing on the decoded features through the first processing layer of the noise-resistant adapter to obtain noise-resistant features.
[0178] The source-channel joint decoder applies classification constraints to the noise-resistant features through the second processing layer of the noise-resistant adapter to obtain the processed features.
[0179] The source-channel joint decoder fuses the processed features through the primary semantic decoder to obtain fused features;
[0180] The source-channel joint decoder performs a linear transformation on the fused features through a fully connected layer of the primary semantic decoder to obtain reconstructed data.
[0181] It should be understood that the source-channel joint decoder achieves the recovery and reconstruction of anti-noise semantic features through a multi-level processing flow. The first processing layer of the anti-noise adapter uses a fully connected network with a ReLU activation function to perform a nonlinear transformation on the decoded features. Through feature space mapping and compression, it effectively suppresses high-frequency interference components introduced by channel noise, generating anti-noise features with stronger robustness. This processing layer is specially designed with an adaptive feature scaling mechanism, which can dynamically adjust the feature enhancement intensity according to the real-time channel state, ensuring a stable noise suppression effect under different signal-to-noise ratio conditions.
[0182] Understandably, the second processing layer of the noise-reducing adapter integrates a lightweight classifier module to impose semantic discriminative constraints on the denoised features. This layer calculates the probability distribution of features across semantic categories using Softmax normalization and compares them with the true labels based on the cross-entropy criterion, outputting semantically calibrated processed features. The classifier uses a learnable weight matrix to project features into the semantic space, and its parameters are co-optimized with the decoder's main network during training, preserving the communicative effectiveness of features while enhancing the accuracy of semantic representation.
[0183] Understandably, after receiving noise-resistant features, the primary semantic decoder achieves deep fusion of shared and private features through feature concatenation and channel attention mechanisms. The fused features contain both general semantic information across users and retain personalized detailed features. The fully connected layer at the end of the primary semantic decoder performs linear transformations and dimensionality adjustments on the fused features, ultimately outputting reconstructed data consistent with the original input data space. This fully connected layer employs a residual connection structure, effectively mitigating the gradient vanishing problem during deep network training.
[0184] In this embodiment, the primary encoder reduces data dimensionality through unified basic feature extraction, the shared encoder focuses on multi-user common semantic features to enhance cross-user collaboration capabilities and reduce redundant information transmission, the private encoder retains user-specific features to achieve customized communication needs while ensuring privacy, the shared decoder ensures cross-user information consistency, the private decoder accurately restores personalized features, the primary decoder fuses shared / private semantic information, the fully connected layer optimizes feature expression dimensionality, the noise-resistant adapter uses nonlinear activation to suppress noise interference, and classification constraints further calibrate and reconstruct data distribution, effectively improving the anti-interference capability, data fidelity, and transmission efficiency of the communication system.
[0185] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the semantic communication method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.
[0186] This application provides a semantic communication device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the semantic communication method in Embodiment 1 above.
[0187] As shown in Figure 6, the semantic communication device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the semantic communication device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the semantic communication device to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows semantic communication devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.
[0188] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.
[0189] The semantic communication device provided in this application, employing the semantic communication method described in the above embodiments, can solve the technical problem of how to achieve privacy-preserving distributed semantic communication in multi-user semantic communication scenarios. Compared with the prior art, the beneficial effects of the semantic communication device provided in this application are the same as those of the semantic communication method provided in the above embodiments, and other technical features of this semantic communication device are the same as those disclosed in the previous embodiment method, and will not be repeated here.
[0190] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.
[0191] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0192] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the semantic communication method in the above embodiments.
[0193] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the semantic communication method described above.
[0194] The computer program product provided in this application can solve the technical problem of how to achieve privacy-preserving distributed semantic communication in multi-user semantic communication scenarios. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the semantic communication method provided in the above embodiments, and will not be repeated here.
[0195] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.
Claims
1. A semantic communication system, wherein, The semantic communication system includes: a source-channel joint encoder, a source-channel joint decoder, and a server. The source-channel joint encoder includes a primary semantic encoder, a shared semantic encoder, and a private semantic encoder. The source-channel joint decoder includes a primary semantic decoder, a shared semantic decoder, a private semantic decoder, and a noise reduction adapter. The source-channel joint encoder is used to extract shared semantic features and private semantic features of the input data through the primary semantic encoder, the shared semantic encoder, and the private semantic encoder. The source-channel joint decoder is used to jointly decode the received shared semantic features and the private semantic features through the shared semantic decoder and the private semantic decoder to obtain the decoded features; The source-channel joint decoder is further configured to process the decoded features through the noise-resistant adapter to obtain processed features, and to fuse the processed features through the primary semantic decoder to obtain reconstructed data; The server is used to receive updated shared semantic encoder parameters and updated shared semantic decoder parameters obtained by each user terminal based on the reconstructed data; The server is also configured to aggregate global shared parameters based on the updated shared semantic encoder parameters and the updated shared semantic decoder parameters using a federated averaging algorithm, and distribute the global shared parameters to each user terminal for semantic communication.
2. The semantic communication system as described in claim 1, wherein, The source-channel joint encoder is used to extract basic semantic features from the input data through the primary semantic encoder to obtain primary semantic features. The source-channel joint encoder is also used to process the primary semantic features through the shared semantic encoder to obtain shared semantic features common to multiple users. The source-channel joint encoder is also used to process the primary semantic features through the private semantic encoder to obtain user-specific private semantic features.
3. The semantic communication system as described in claim 1, wherein, The source-channel joint decoder is also used to decode the shared semantic features through the shared semantic decoder to obtain shared semantic features that are common to multiple users after decoding. The source-channel joint decoder is also used to decode the private semantic features through the private semantic decoder to obtain user-specific decoded private semantic features.
4. The semantic communication system as described in claim 1, wherein, The source-channel joint decoder is further configured to perform nonlinear activation processing on the decoded features through the first processing layer of the noise-resistant adapter to obtain noise-resistant features; The source-channel joint decoder is also used to apply classification constraints to the noise-resistant features through the second processing layer of the noise-resistant adapter to obtain the processed features; The source-channel joint decoder is further configured to fuse the processed features using the primary semantic decoder to obtain fused features; The source-channel joint decoder is further used to perform a linear transformation on the fused features through the fully connected layer of the primary semantic decoder to obtain reconstructed data.
5. The semantic communication system as described in claim 1, wherein, The semantic communication system further includes a local training controller, which is used to determine the mean square error of the reconstructed data and the input data; The local training controller is further configured to generate a total loss by weighted summation of the mean squared error, cross-entropy loss, and separation loss; The local training controller is also used to determine the parameter gradients of the shared semantic encoder and the shared semantic decoder based on the total loss using a backpropagation algorithm. The local training controller is also used to update the local shared semantic encoder parameters and shared semantic decoder parameters according to the parameter gradient based on the federated learning algorithm, so as to obtain the updated shared semantic encoder parameters and updated shared semantic decoder parameters. The local training controller is further configured to upload the updated shared semantic encoder parameters and the updated shared semantic decoder parameters to the server if the local training rounds reach a preset number of training rounds.
6. The semantic communication system as described in claim 5, wherein, The local training controller is further configured to obtain a separation loss by determining the orthogonality constraints between shared semantic features and private semantic features; The local training controller is also used to obtain cross-entropy loss by determining the difference between the classifier output of the noise-resistant adapter and the true label.
7. A semantic communication method, wherein, The semantic communication method is applied to the semantic communication system, which includes: a source-channel joint encoder, a source-channel joint decoder, and a server. The source-channel joint encoder includes a primary semantic encoder, a shared semantic encoder, and a private semantic encoder. The source-channel joint decoder includes a primary semantic decoder, a shared semantic decoder, a private semantic decoder, and a noise immunity adapter. The method includes: The source-channel joint encoder extracts shared semantic features and private semantic features of the input data through the primary semantic encoder, the shared semantic encoder, and the private semantic encoder; The source-channel joint decoder uses the shared semantic decoder and the private semantic decoder to jointly decode the received shared semantic features and the private semantic features to obtain the decoded features; The source-channel joint decoder processes the decoded features through the noise-resistant adapter to obtain processed features, and then fuses the processed features through the primary semantic decoder to obtain reconstructed data; The server receives the shared semantic encoder parameters and shared semantic decoder parameters updated by each user terminal based on the reconstructed data; The server aggregates globally shared parameters based on the shared semantic encoder parameters and the shared semantic decoder parameters using a federated averaging algorithm, and distributes the globally shared parameters to each user terminal for semantic communication.
8. The semantic communication method as described in claim 7, wherein, The source-channel joint encoder extracts shared semantic features and private semantic features of the input data through the primary semantic encoder, the shared semantic encoder, and the private semantic encoder, including: The source-channel joint encoder extracts basic semantic features from the input data through the primary semantic encoder to obtain primary semantic features; The source-channel joint encoder processes the primary semantic features through the shared semantic encoder to obtain shared semantic features applicable to multiple users; The source-channel joint encoder processes the primary semantic features through the private semantic encoder to obtain user-specific private semantic features.
9. The semantic communication method as described in claim 7, wherein, The source-channel joint decoder, through the shared semantic decoder and the private semantic decoder, performs joint decoding on the received shared semantic features and the private semantic features to obtain the decoded features, including: The source-channel joint decoder decodes the shared semantic features through the shared semantic decoder to obtain shared semantic features that are common to multiple users after decoding. The source-channel joint decoder decodes the private semantic features through the private semantic decoder to obtain user-specific decoded private semantic features.
10. The semantic communication method as described in claim 7, wherein, The source-channel joint decoder processes the decoded features through the noise-resistant adapter to obtain processed features, and then fuses the processed features through the primary semantic decoder to obtain reconstructed data. The steps include: The source-channel joint decoder performs non-linear activation processing on the decoded features through the first processing layer of the noise-resistant adapter to obtain noise-resistant features. The source-channel joint decoder applies classification constraints to the noise-resistant features through the second processing layer of the noise-resistant adapter to obtain the processed features. The source-channel joint decoder fuses the processed features through the primary semantic decoder to obtain fused features; The source-channel joint decoder performs a linear transformation on the fused features through a fully connected layer of the primary semantic decoder to obtain reconstructed data.