Semantic communication method based on sparse sharing

Abstract

The invention belongs to the technical field of wireless communication, and discloses a semantic communication method based on sparse sharing, which comprises the following steps: step S1, a semantic communication system maps source data into semantic embedding representation based on a semantic perception coding mechanism; s2, introducing a lightweight dynamic mask strategy, dynamically adjusting the weight of the model through a mask generation module, and realizing efficient adaptation of a single model under different channel conditions; and S3, based on a task partitioning method of a channel condition, dividing the channel condition according to a signal-to-noise ratio range, and generating an independent sparse mask for each region so as to realize adaptive optimization. According to the semantic communication method based on sparse sharing, a dynamic mask mechanism is introduced, so that a single model can efficiently adapt to different channel conditions, meanwhile, the calculation and storage overhead is remarkably reduced, and a new possibility is provided for actual deployment of semantic communication.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a semantic communication method based on sparse sharing. Background Art

[0002] With the rapid development of 6G communication technology, semantic communication is gradually becoming an important research direction for the next generation of communication systems. The core of semantic communication lies in the semantic content of transmitted information, rather than simply focusing on the integrity of the bit stream. Compared with traditional source-channel separation coding technology, semantic communication can achieve more efficient communication under limited bandwidth resources by jointly optimizing source and channel coding.

[0003] Traditional communication systems usually use separate source-channel coding methods, the purpose of which is to accurately reproduce the transmitted signal at the receiving end. However, when the channel conditions deteriorate below the system design target, the quality of the decoded signal will drop sharply. Traditional digital communication systems lack robustness when channel conditions change. Once the channel coding and modulation scheme is determined according to a specific signal-to-noise ratio value, the number of compressed bits is fixed, resulting in the quality of the reconstructed image not improving as the signal-to-noise ratio increases. When the channel quality is below the target signal-to-noise ratio value, the channel code cannot effectively cope with the increasing error rate, resulting in a significant decrease in the quality of the reconstructed image.

[0004] To solve these problems, semantic communication came into being, whose goal is to convey the semantics of information to the receiver rather than accurately reproduce the signal itself. By selectively transmitting data related to the meaning of the information, semantic communication can reduce the amount of transmission, thereby improving efficiency. At the same time, semantic communication is more robust to channel noise and interference, and the receiver can understand the core content of the information even if some data is lost. In addition, semantic communication can adaptively adjust the encoding and decoding strategies according to channel conditions, thereby maintaining stable performance in a changing channel environment.

[0005] Despite the huge potential of semantic communication, existing methods still have some significant shortcomings. First, current methods lack the ability to dynamically adapt to channel conditions and are usually trained and applied under fixed channel conditions. When the channel quality changes, the system performance will degrade rapidly, making it difficult to meet the needs of complex wireless environments. Second, semantic communication systems rely on deep neural networks for encoding and decoding. These models are usually large in scale, which not only increases storage overhead but also limits their application in resource-constrained devices, especially in edge computing and low-power scenarios. In addition, traditional methods are usually optimized for specific tasks or channel conditions, lack versatility in cross-channel and cross-task scenarios, and have difficulty in fully utilizing communication resources.

[0006] In response to the above challenges, the present invention proposes a semantic communication method (Semantic Sparse Coding, SSC) based on sparse sharing, which provides an effective technical solution to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to provide a semantic communication method based on sparse sharing, which introduces a dynamic masking mechanism so that a single model can efficiently adapt to different channel conditions while significantly reducing computational and storage overheads, thus providing new possibilities for the practical deployment of semantic communication.

[0008] To achieve the above object, the present invention provides a semantic communication method based on sparse sharing, comprising the following steps:

[0009] Step S1, the semantic communication system maps the source data into a semantic embedding representation based on a semantically-aware encoding mechanism;

[0010] Step S2: introduce a lightweight dynamic mask strategy, dynamically adjust the model weight through the mask generation module, and achieve efficient adaptation of a single model under different channel conditions;

[0011] Step S3: A task partitioning method based on channel conditions divides the channel conditions according to the signal-to-noise ratio range, and generates an independent sparse mask for each area to achieve adaptive optimization.

[0012] Preferably, in step S1, the semantic communication system is composed of a sending end, a receiving end and a wireless channel, and its working process is as follows:

[0013] Step S11, encoding at the transmitting end;

[0014] The sender takes the original input x∈R N Mapped to the K-dimensional semantic embedding z as follows:

[0015] z=E(x,γ,CR);

[0016] Among them, CR = K / N is the compression ratio; γ is the signal-to-noise ratio; E is the encoder network;

[0017] Step S12: channel transmission;

[0018] The encoded semantic information z is transmitted through the channel and is affected by attenuation and additive Gaussian white noise during transmission, as shown below:

[0019]

[0020] Where, h is the channel gain; represents Gaussian noise; σ 2 is the average noise power; I is the unit matrix;

[0021] Step S13, receiving end encoding;

[0022] The receiver uses decoder D to recover the original input as follows:

[0023]

[0024] Where γ is the signal-to-noise ratio, γ=P|h| 2 / σ 2 , P represents the transmission power.

[0025] Preferably, in step S2, based on the dynamic sparse mask mechanism, the activation parameters of the neural network are adjusted according to the channel environment to generate a sub-network adapted to the specific channel conditions. The specific process is as follows:

[0026] Step S21, mask definition;

[0027] Mask M E ∈{0,1}| θ | and M D ∈{0,1}| β | are used for the sparseness of the transmitter encoder and the receiver decoder, respectively, where θ and β are model parameters; a mask of 1 means retaining the corresponding parameters, and a mask of 0 means pruning, as shown below:

[0028]

[0029]

[0030] Among them, ⊙ represents the element-by-element product operation;

[0031] Step S22, mask generation;

[0032] Use the iterative amplitude pruning algorithm to generate sparse masks and find the optimal sub-network structure in different signal-to-noise ratio ranges; remove the α% parameter with the smallest weight amplitude each iteration until the target sparsity is reached; the calculation formula for sparsity is as follows:

[0033]

[0034] Among them, S represents sparsity;

[0035] Step S23, loss function design;

[0036] During training, the mean square error is used as the loss function as follows:

[0037]

[0038] Where d(·,·) is the mean square error between samples; x iand They correspond to x and The intensity of the color component of each pixel; N is the image size.

[0039] Preferably, in step S3, the task partitioning method based on the channel condition divides the channel condition according to the signal-to-noise ratio range, and generates an independent sparse mask for each area to achieve adaptive optimization. The specific process is as follows:

[0040] Step S31: for each signal-to-noise ratio region, perform mask training using an iterative amplitude pruning algorithm to generate a corresponding sparse mask;

[0041] Step S32: Apply the region mask on the shared model to perform joint optimization training.

[0042] Therefore, the present invention adopts the above-mentioned semantic communication method based on sparse sharing and introduces a dynamic mask mechanism, so that a single model can efficiently adapt to different channel conditions, while significantly reducing computing and storage overheads, providing new possibilities for the actual deployment of semantic communication.

[0043] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a model diagram of a point-to-point image transmission system with signal-to-noise ratio feedback;

[0045] Figure 2 They are the feature learning (FL) module and the attention feature (AF) module;

[0046] Figure 3 As the basic model structure diagram;

[0047] Figure 4 It is a schematic diagram of the system structure based on the semantic mask mechanism of the present invention;

[0048] Figure 5 The experimental results comparison diagram of the influence of channel conditions on model performance; (a) is the experimental results comparison diagram of the influence of AWGN channel on model performance; (b) is the experimental results comparison diagram of the influence of Rayleigh channel on model performance;

[0049] Figure 6 This is a flow chart of a semantic communication method based on sparse sharing of the present invention. DETAILED DESCRIPTION

[0050] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0051] like Figure 6As shown, the present invention provides a semantic communication method based on sparse sharing, comprising the steps of:

[0052] Step S1: The semantic communication system is based on a semantically-aware encoding mechanism, which significantly reduces the transmission of redundant information by mapping source data into semantically embedded representations.

[0053] The semantic communication system consists of a transmitter, a receiver, and a wireless channel. Its main workflow includes the following three stages:

[0054] Step S11, encoding at the transmitting end;

[0055] The sender takes the original input x∈R N Mapped to the K-dimensional semantic embedding z as follows:

[0056] z=E(x,γ,CR);

[0057] Among them, CR = K / N is the compression ratio; γ is the signal-to-noise ratio; E is the encoder network.

[0058] Step S12: channel transmission;

[0059] The encoded semantic information z is transmitted through the channel and is affected by attenuation and additive white Gaussian noise n during transmission, as shown below:

[0060]

[0061] Where, h is the channel gain; represents Gaussian noise; σ 2 is the average noise power; I is the unit matrix.

[0062] Step S13, receiving end encoding;

[0063] The receiver uses decoder D to recover the original input as follows:

[0064]

[0065] Where γ is the signal-to-noise ratio, γ=P|h| 2 / σ 2 , P represents the transmission power.

[0066] Step S2: introduce a lightweight dynamic mask strategy, dynamically adjust the model weights through the mask generation module, and achieve efficient adaptation of a single model under different channel conditions.

[0067] Specifically, the dynamic sparse mask mechanism can flexibly adjust the activation parameters of the neural network according to the channel environment, thereby generating a sub-network that adapts to specific channel conditions.

[0068] Step S21, mask definition;

[0069] Mask M E ∈{0,1}| θ | and M D ∈{0,1}| β | are used for sparseness of the encoder at the transmitter and the decoder at the receiver, respectively, where θ and β are model parameters. A mask of 1 means retaining the corresponding parameters, and a mask of 0 means pruning, as shown below:

[0070]

[0071] Where ⊙ represents the element-wise product operation.

[0072] Step S22, mask generation;

[0073] The sparse mask is generated using the Iterative Magnitude Pruning (IMP) algorithm, and the optimal subnetwork structure is found in different signal-to-noise ratio ranges. The α% parameter with the smallest weight amplitude is removed in each iteration until the target sparsity is reached; the sparsity calculation formula is as follows:

[0074]

[0075] Here, S represents sparsity.

[0076] Step S23, loss function design;

[0077] During training, the mean square error (MSE) is used as the loss function as shown below:

[0078]

[0079] Where d(·,·) is the mean square error between samples; x i and They correspond to x and The intensity of the color component of each pixel; N is the image size.

[0080] Step S3, a task partitioning method based on channel conditions, significantly improves the robustness and storage efficiency of the system through a regional optimization strategy.

[0081] The channel conditions are divided according to the SNR range, such as low SNR area: 0–10dB, high SNR area: 10–20dB, and independent sparse masks are generated for each area to achieve adaptive optimization.

[0082] Step S31, mask training;

[0083] For each SNR region, the corresponding sparse mask is generated using IMP.

[0084] Step S32: model training;

[0085] Apply region masks on shared models for joint optimization.

[0086] Through the above technical solution, the present invention can reduce storage and computing overhead while ensuring effective transmission of semantic information, thereby meeting the needs of the next generation communication system.

[0087] Example 1

[0088] This embodiment analyzes and studies the semantic information encoding mechanism.

[0089] like Figure 1 As shown, this embodiment considers a point-to-point image transmission system with signal-to-noise ratio feedback. The channel signal-to-noise ratio is known at both the joint source channel encoder and the joint source channel decoder. N Represents an input image of size H (height) × W (width) × C (channels), where N = H × W × C, and R represents a set of real numbers.

[0090] The joint source channel encoder encodes x and the feedback signal-to-noise ratio γ, and the encoding function E θ :R N ×R→C K Generate a complex-valued channel input symbol vector z∈C K The encoding process is as follows:

[0091] z=E θ( x,γ,CR ) ∈C K ;

[0092] Wherein, K is the size of the channel input symbol; θ is the parameter set of the joint source channel encoder; CR=K / N is the compression ratio; γ∈R is the channel signal-to-noise ratio that can be estimated at the joint source channel decoder and fed back to the joint source channel encoder; C represents a set of complex numbers.

[0093] The encoder maps an N-dimensional vector of real-valued images x to a K-dimensional vector of complex-valued channel input samples z. In order to satisfy the average power constraint of the joint source channel encoder, where z * represents the conjugate transpose of z. The encoding symbol z is obtained by the function η:C K →C K Represents the noisy channel transmission.

[0094] This embodiment considers the AWGN channel, and the method can also be applied to other differentiable channels. Fading channels are as follows:

[0095]

[0096] where h∈C is the channel gain.

[0097] By applying equalization at the receiver, the noise has a different distribution.

[0098] The joint source-channel decoder uses the decoding function D β :C K ×R→R N Will and γ are mapped as follows:

[0099]

[0100] in, is the estimate of the original image x; β is the parameter set of the joint source-channel decoder.

[0101] The original image x and the reconstructed image The distortion between is as follows:

[0102]

[0103] Among them, x i and They correspond to x and The intensity of each color component of a pixel.

[0104] The image size N, channel input size K and They are called source bandwidth, channel bandwidth and bandwidth ratio respectively. Under a certain CR, the encoder and decoder parameters θ are determined * and β * , to minimize the expected distortion, as follows:

[0105]

[0106] Among them, θ * is the optimal encoder parameter; β * are the optimal decoder parameters; Represents the original image x and the reconstructed image The joint probability distribution of ; p(γ) represents the probability distribution of the signal-to-noise ratio.

[0107] The encoder and decoder are modeled using deep neural networks.

[0108] like Figure 2 As shown in Figure 1, the structure of the encoder and decoder consists of two types of core modules: feature learning (FL) module and attention feature (AF) module. These two types of modules are stacked alternately to achieve efficient processing of input signals and dynamic adaptation to channel conditions.

[0109] Feature Learning (FL) module: Its main function is to extract semantic features from the input. This module is based on a convolutional neural network and can capture the local features and global structural information of the input signal. An FL module consists of a convolutional layer, a parametric rectified linear activation function (PReLU), and a generalized divisive normalization layer (GDN).

[0110] Attention Feature (AF) module: Using the signal-to-noise ratio information and the output features of the FL module, a set of scaling parameters is generated to dynamically adjust the feature representation. Through the scaling operation, the AF module is able to filter and optimize the features according to the current channel conditions. Through the collaborative work of the FL module and the AF module, the system can effectively cope with different channel conditions and achieve high-quality semantic information transmission.

[0111] The core of the AF module is the soft attention mechanism based on channel conditions, and its design includes the following three parts:

[0112] (1) Context extraction;

[0113] The extracted context information includes the signal-to-noise ratio γ and the input feature information IG. First, the global feature IG is extracted by global average pooling G(·) i ; Then, it is connected with the signal-to-noise ratio γ to generate context information I. The context information integrates the global semantic information of the channel state and feature map, providing a basis for subsequent factor prediction.

[0114] (2) Factor prediction;

[0115] Using the factor prediction neural network P ω (·) Generate a scaling factor κ based on the context information I. ω The structure of (·) consists of two fully connected (FC) layers, where the first layer uses ReLU activation and the second layer uses Sigmoid activation to limit the output to the interval (0, 1). The generated scaling factor κ is used to adjust the weights of different components of the input features.

[0116] (3) Feature recalibration;

[0117] The scaling factor κ is used to adjust the input feature map FG, and each channel of FG is element-wise multiplied by the corresponding scaling factor to obtain the recalibrated feature map FA. This process can highlight important features while suppressing redundant information, thereby improving the efficiency and robustness of feature representation.

[0118] By combining the feature learning module with the attention feature module, the encoder and decoder can flexibly adjust the feature representation under changing channel conditions. This design not only improves the transmission efficiency of semantic information, but also enhances the adaptability and robustness of the system, providing key technical support for achieving efficient semantic communication.

[0119] The overall architecture of the network is as follows Figure 3 As shown in Figure 1, there are five FL modules in the encoder, and each module is followed by an AF module except the last FL module. The output of the previous FL module is one input to the current AF module, and the current AF module output is the input to the next FL module. Another input to the AF module is the SNR level associated with the communication channel. The construction of the decoder is similar. Different bandwidth ratios can be obtained by changing the output channel size c in the last convolutional layer of the encoder.

[0120] Example 2

[0121] This embodiment analyzes and studies the dynamic mask mechanism.

[0122] The mask generation algorithm of the present invention aims to dynamically generate sparse masks that meet the requirements of different tasks. It uses pruning technology and dynamic adjustment strategies for the encoder and decoder parts of the model to reduce computational and storage overhead while ensuring the robustness of model performance. The core logic of mask generation is based on the iterative amplitude pruning (IMP) algorithm, and the model's parameter optimization is achieved through sparsification. The following are the specific steps of the mask generation algorithm:

[0123] (1) Input and initialization;

[0124] The inputs to the mask generation algorithm include:

[0125] A. Encoder and decoder model parameters: denoted as θ and β respectively, representing the weight vector of the model.

[0126] θ={θ1,θ2,…,θ s},β={β1,β2,…,β t};

[0127] B. Sparsity target S: It is used to control the sparsity of the generated mask and indicates the proportion of retained parameters.

[0128] Initially, the mask M E and M D Set to a matrix of all 1s, as follows:

[0129] M E =1,M D =1;

[0130] (2) Iterative amplitude pruning (IMP);

[0131] Dynamic sparse mask generation uses the iterative amplitude pruning (IMP) algorithm, which gradually prunes parameters with smaller weight amplitudes through multiple rounds of iterations. The specific process is as follows:

[0132] A. Parameter amplitude sorting: Sort the absolute values ​​of the model parameters θ and β of the encoder and decoder to obtain the amplitude of each weight, as shown below:

[0133] Rank ( θ i ) = | θ i | ,Rank ( β j ) = | β j | ;

[0134] Among them, θ i is the i-th parameter in the encoder parameter θ; β j is the jth parameter in the encoder parameter.

[0135] B. Pruning ratio calculation: Set the ratio of parameters to be removed in each iteration to α%. According to the amplitude sorting results, select the α% parameter with the smallest weight amplitude for pruning.

[0136] Update mask: The mask value corresponding to the pruned weight is set to 0, and the unpruned weight remains 1.

[0137] C. Iterative execution: Repeat the above process, removing the α% parameter with the smallest amplitude each time, until the overall mask reaches the set sparsity target S, as shown below:

[0138]

[0139] Each iteration of IMP can gradually approach the final sparsity target while ensuring that the retained parameters have high importance, thereby maximizing the effective performance of the model.

[0140] Example 3

[0141] This embodiment analyzes and studies the semantic communication system architecture.

[0142] like Figure 4 As shown in the figure, the task partitioning method based on channel conditions of the present invention aims to significantly improve the robustness, adaptability and storage efficiency of the system through a regional optimization strategy. Specifically, the method divides the channel conditions according to the signal-to-noise ratio (SNR) range (for example, low SNR area: 0-10dB; high SNR area: 10-20dB), and generates an independent sparse mask for each channel area to achieve adaptive optimization of the task.

[0143] This method optimizes the tasks in different channel areas separately, which can significantly reduce the system's computing and storage overhead while ensuring the effective transmission of semantic information. The following is the detailed implementation process of this method:

[0144] (1) Model training;

[0145] After the mask training phase is complete, the region mask is applied on the shared base model for joint optimization. The encoder and decoder are optimized using dynamically generated sparse masks. By selectively activating different parts of the model through masks, redundant computations can be significantly reduced and storage space can be saved without the need to train models separately for each channel region.

[0146] During the training process, the network switches the corresponding mask for training according to different channel conditions. This not only improves the efficiency of the training process, but also avoids the computational and storage overhead caused by the traditional method of training multiple models separately for each channel condition.

[0147] (2) Experimental verification;

[0148] In order to verify the effectiveness of the method of the present invention, multiple rounds of experiments were conducted, using the CIFAR-10 dataset and tested under AWGN channel and Rayleigh fast fading channel. In the experimental setting, it was compared with two existing semantic communication methods: deepJSCC and ADJSCC. This embodiment uses peak signal-to-noise ratio (PSNR), a widely used image quality indicator, to evaluate the performance of the method and benchmark scheme proposed in the present invention. A higher PSNR indicates better image quality.

[0149] (3) Experimental results;

[0150] The proposed method is compared with the ADJSCC and deepJSCC methods under AWGN and Rayleigh channels. Figure 5 As shown, the experimental results show that:

[0151] ADJSCC uses a uniform SNR distribution (0dB to 20dB) during training, while the deepJSCC model is trained at specific SNR test values ​​(such as 7dB and 13dB). The ADJSCC model outperforms any deepJSCC model trained at a specific SNR under all test SNRs. The proposed method outperforms ADJSCC and deepJSCC under all test SNR conditions, demonstrating the advantage of dynamically adjusting channel adaptability through sparse masks.

[0152] Therefore, the present invention adopts the above-mentioned semantic communication method based on sparse sharing and introduces a dynamic mask mechanism, so that a single model can efficiently adapt to different channel conditions, while significantly reducing computing and storage overheads, providing new possibilities for the actual deployment of semantic communication.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A semantic communication method based on sparse sharing, characterized in that: The following steps are involved: Step S1, the semantic communication system maps the source data into a semantic embedding representation based on a semantically-aware encoding mechanism; Step S2: introduce a lightweight dynamic mask strategy, dynamically adjust the model weight through the mask generation module, and achieve efficient adaptation of a single model under different channel conditions; Step S3: A task partitioning method based on channel conditions divides the channel conditions according to the signal-to-noise ratio range, and generates an independent sparse mask for each area to achieve adaptive optimization.

2. A semantic communication method based on sparse sharing according to claim 1), characterized in that: In step S1, the semantic communication system consists of a transmitter, a receiver, and a wireless channel, and its workflow is as follows: Step S11, encoding at the transmitting end; The sender takes the original input x∈R N Mapped to the K-dimensional semantic embedding z as follows: z=E(x,γ,CR); Among them, CR = K / N is the compression ratio; γ is the signal-to-noise ratio; E is the encoder network; Step S12: channel transmission; The encoded semantic information z is transmitted through the channel and is affected by attenuation and additive Gaussian white noise during transmission, as shown below: Where, h is the channel gain; represents Gaussian noise; σ 2 is the average noise power; I is the unit matrix; Step S13, receiving end encoding; The receiver uses decoder D to recover the original input as follows: Where γ is the signal-to-noise ratio, γ=P|h| 2 / σ 2 , P represents the transmission power.

3. According to claim 1, a semantic communication method based on sparse sharing) is characterized in that: In step S2, based on the dynamic sparse mask mechanism, the activation parameters of the neural network are adjusted according to the channel environment to generate a sub-network adapted to the specific channel conditions. The specific process is as follows: Step S21, mask definition; Mask M E ∈{0,1} |θ| and M D ∈{0,1} |β| They are used for the sparseness of the encoder at the transmitter and the decoder at the receiver, respectively, where θ and β are model parameters; a mask of 1 indicates that the corresponding parameters are retained, and a mask of 0 indicates pruning, as shown below: Among them, ⊙ represents the element-by-element product operation; Step S22, mask generation; Use the iterative amplitude pruning algorithm to generate sparse masks and find the optimal sub-network structure in different signal-to-noise ratio ranges; remove the α% parameter with the smallest weight amplitude each iteration until the target sparsity is reached; the calculation formula for sparsity is as follows: Among them, S represents sparsity; Step S23, loss function design; During training, the mean square error is used as the loss function as follows: Where d(·,·) is the mean square error between samples; x i and They correspond to x and The intensity of the color component of each pixel; N is the image size.

4. The semantic communication method based on sparse sharing according to claim 1, characterized in that: In step S3, the task partitioning method based on channel conditions divides the channel conditions according to the signal-to-noise ratio range, and generates an independent sparse mask for each area to achieve adaptive optimization. The specific process is as follows: Step S31: for each signal-to-noise ratio region, perform mask training using an iterative amplitude pruning algorithm to generate a corresponding sparse mask; Step S32: Apply the region mask on the shared model to perform joint optimization training.

Images