Image semantic joint communication method based on saliency guide chaotic encryption

Abstract

The invention relates to an image semantic joint communication method based on significance guide chaotic encryption, and belongs to the field of semantic communication. The method comprises the following steps: firstly, inputting an image into an encoder, extracting a multi-scale semantic feature map of the image, fusing the multi-scale feature maps to generate a saliency map for evaluating the importance of each region of the image, generating a chaotic disturbance mask according to a shared chaotic key seed, and performing element-by-element disturbance and encrypting original semantic features in combination with the saliency map. Then, the scrambled features are flattened, discretized and digitally modulated, and signals suitable for transmission are generated. Then, the signal is transmitted to a receiving end through an additive Gaussian white noise channel; a receiving end reconstructs a disturbance mask according to the same chaotic key, and executes descrambling and feature reduction operation; and finally, respectively inputting the restored features into an image reconstruction decoder and a semantic classification decoder, and outputting image reconstruction and image semantic labels. According to the method, image reconstruction and image semantic recognition tasks can be effectively completed.

Description

Technical Field

[0001] The invention belongs to the field of semantic communication and relates to an image semantic joint communication method based on saliency-guided chaotic encryption. Background Art

[0002] With continuous breakthroughs in deep learning and wireless communication technologies, traditional communication systems are undergoing a paradigm shift from "symbol transmission" to "semantic transfer." In classical communication theory, the design goal of a communication system is typically to ensure that the bit stream transmitted by the transmitter can be accurately reconstructed by the receiver in a noisy channel. Performance metrics are often measured in terms of bit error rate (BER) or symbol error rate (SER). However, with the development of 5G, the Internet of Things, and edge intelligence, traditional bit-level communication is gradually facing bottlenecks in efficiency and adaptability in the face of demands for large-scale connectivity, low latency, and high reliability.

[0003] Semantic communication, as an emerging communication model, aims to optimize the expression and understanding of information during transmission, using semantic correctness as its core metric. By using deep learning models to perform high-level abstraction of raw data, semantic communication systems can extract and transmit only the semantic content that is relevant to the task, effectively reducing transmission redundancy and enhancing the system's robustness in low signal-to-noise ratio conditions. This concept is being widely explored and is gradually being extended to multiple data types, such as text, speech, and images.

[0004] Among numerous research areas, image semantic communication is one of the most challenging. Compared to structured data such as text, images are high-dimensional, dense, and highly redundant. Furthermore, semantic information is often implicit in images in the form of spatial distribution. In recent years, end-to-end image semantic communication systems have emerged, attempting to extract image semantic features through deep encoders, compress and modulate them, and transmit them to the receiver for image reconstruction or semantic recognition.

[0005] Although existing work has preliminarily verified the feasibility of this direction, current image semantic communication still faces the following key issues: First, existing methods usually process the entire image in an integrated manner, and encoding and perturbation processing are generally unified strategies, which fail to fully identify and protect semantic key areas; second, facing an open channel environment, the system generally lacks an effective security protection mechanism. Once the transmitted content is intercepted, it can be easily directly decoded or restored; finally, image semantic communication systems usually focus on one direction of image reconstruction or semantic recognition in task modeling, and lack an integrated mechanism to simultaneously encode and uniformly transmit image and semantic information, which cannot fully tap the potential for feature sharing and fusion.

[0006] Therefore, there is an urgent need for a new image semantic communication system that can overcome the problems of lack of semantic region modeling, insufficient security of the communication process, and separation of task expression in existing image semantic communication. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide a digital communication method that combines saliency-guided perturbation, chaotic key perturbation mechanism and joint semantic image decoding, while having regional semantic recognition capability, security perturbation capability and task collaborative decoding capability, and can provide effective support for future intelligent communication scenarios.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] Solution 1: A joint image semantic communication method based on saliency-guided chaotic encryption, specifically including the following steps:

[0010] S1: The image is input into the encoder, and the encoder's multi-scale feature extraction module extracts the multi-scale semantic features of the image;

[0011] S2: The semantic features of different scales are fused through the saliency map generation module to obtain the saliency map S;

[0012] S3: Input the feature map and the saliency map into the chaotic perturbation encryption module, generate the chaotic perturbation mask M according to the shared chaotic key, and obtain the encrypted feature;

[0013] S4: Flatten, discretize, encode and modulate the encrypted features to convert them into a signal Z suitable for channel transmission;

[0014] S5: Transmit the signal Z through the channel to the receiver, while being interfered by white Gaussian noise (AWGN);

[0015] S6: The receiver reconstructs the chaotic perturbation mask based on the same shared chaotic key and completes the inverse perturbation transform in combination with the local saliency map to restore the original semantic feature estimate.

[0016] S7: Input the recovered features into the image reconstruction decoder and the image classification decoder respectively to obtain the reconstructed image and the semantic category prediction label;

[0017] S8: Use a joint loss function to jointly optimize the image reconstruction error and semantic classification error to complete end-to-end training.

[0018] Scheme 2: An image semantic joint communication system based on saliency-guided chaotic encryption, including a transmitter, a channel transmission module and a receiver.

[0019] (1) The transmitter includes an encoder and a modulation and coding module.

[0020] 1) The encoder includes a multi-scale feature extraction module, a feature upsampling and alignment module, a saliency map generation module, a saliency normalization module, a chaotic perturbation encryption module, and a saliency guided perturbation module.

[0021] Multi-scale feature extraction module: Receives the input image X and extracts shallow structural information and deep semantic information through four levels of residual convolutional layers. The intermediate features obtained are named Z3, Z4, and Z5, with decreasing spatial resolution and increasing number of channels to cover multi-scale semantics.

[0022] Feature upsampling and alignment module: Since the spatial sizes of Z3, Z4, and Z5 are inconsistent, they are aligned to a unified scale through upsampling (bilinear interpolation) to facilitate subsequent fusion operations;

[0023] The saliency map generation module first performs saliency estimation on semantic feature maps of three different scales. All saliency maps are then resized and concatenated. A fused saliency map is then output through a 1×1 convolution and sigmoid activation. Each saliency estimator integrates multi-scale convolution, channel attention, and threshold prediction mechanisms to improve saliency modeling accuracy and semantic focus.

[0024] Saliency normalization module: The original saliency map S is normalized to the final saliency map S′∈(0,1) using the Sigmoid activation function for subsequent perturbation intensity adjustment; locations with high responses represent important areas in the image, and stronger perturbations will be applied.

[0025] Chaotic perturbation encryption module: Let the key seed seed∈(0,1) be the shared input to initialize the Logistic chaotic perturbation mask. The chaotic perturbation mask M ultimately has the same spatial size and number of channels as Z5. The perturbation value of each pixel fluctuates between (0,1) and has high entropy, high sensitivity and unpredictability. The generating function of M is as follows:

[0026]

[0027] Among them, M0 is the initial value of the iterative sequence (the result of the 0th iteration), M n is the result value obtained after the nth iteration, seed is the shared chaotic key seed, r is the chaotic parameter of the Logistic map, and r is set to 3.99

[0028] Saliency-guided perturbation module: The normalized saliency map S′ is multiplied by the chaotic perturbation mask M to form a joint perturbation factor α, and then the deep feature map Z5 is perturbed in a position-controlled manner. The operation is as follows:

[0029] α=M×S′

[0030] X′=Z5×(1+α)

[0031] Among them, X′ is the feature map after perturbation.

[0032] 2) The modulation and coding module flattens the perturbed feature map X′ and inputs it into the linear layer, mapping it into a continuous feature vector. It then performs hard sampling using Gumbel Softmax to obtain discrete symbols. The discrete symbols are encoded into signal Z through the modulation and coding module (BPSK modulation, 4-QAM modulation, 16-QAM modulation, or 64-QAM modulation).

[0033] (2) Channel transmission module: The signal Z is sent to the channel transmission module, during which additive white Gaussian noise (AWGN) interference is simulated:

[0034] Z′=Z+n

[0035] Among them, n~N(0,σ 2 ), indicating that the source of n is zero mean and variance is σ 2 Gaussian process, σ 2 The signal-to-noise ratio n is set to be consistent with the actual wireless communication environment.

[0036] (3) The receiving end includes a shared descrambling module, an image reconstruction decoder, and an image classification decoder.

[0037] 1) The shared descrambling module specifically uses the shared chaotic key seed seed that is consistent with the transmitter, generates the chaotic perturbation mask M through the same chaotic mapping function (Logistic mapping), and calculates the perturbation factor α′ by element-wise multiplication with the normalized saliency map S″ calculated by the shared descrambling module. The calculation formula is:

[0038] α′=M×S″

[0039] It is used for descrambling operation, and the input feature vector Z′ is subjected to inverse perturbation transformation according to the following formula to preliminarily restore the original feature estimate Z5′:

[0040]

[0041] This step is equivalent to the inverse operation of the scrambling operation, which can initially restore the original semantic features. Considering that the nonlinear loss introduced by the perturbation process cannot be fully recovered by simple division, the system further designs a compensation sub-network to perform fine-grained repair on the descrambled features:

[0042] Z out =Z5′+fcomp(Z′)

[0043] Among them, Z outis the descrambled feature vector, fcomp(·) is the residual compensation subnetwork at the receiving end, which consists of two layers of 3×3 convolution and ReLU activation function. Its function is to predict the distribution of perturbation error based on the input feature vector Z′ and perform additive compensation to improve the descrambling accuracy.

[0044] 2) The image reconstruction decoder is used to descramble the perturbed feature vector Z out Restore the original image to its pixel-level representation output, including:

[0045] Feature reshaping module: the input one-dimensional vector Z out Mapped to the initial feature map tensor Z0 through the linear layer, its shape is B×C×H×W, where B, C, H, and W are the batch size, number of channels, height of the current feature map after downsampling, and width of the current feature map after downsampling, respectively;

[0046] Upsampling module 1: Use the DepthToSpace module to upsample the feature map spatial size to a medium resolution;

[0047] Channel Attention Module: This module is used to perform weighted adjustments between channels in the upsampled feature map to highlight important semantic channels.

[0048] Upsampling module 2: Use the DepthToSpace module again to restore the feature map to the target image space size;

[0049] Output layer: Use 3×3 convolution and Tanh activation function to compress the number of channels to 3 and output the final RGB reconstructed image.

[0050] 3) The image reconstruction decoder is used to descramble the perturbed feature vector Z out Perform global representation extraction and output the final semantic category prediction result Y∈R Class , specifically including:

[0051] Multilayer Perceptron architecture: Consists of four fully connected submodules, each of which includes a linear layer (Linear), batch normalization (BatchNorm1d), and a learnable activation function (PReLU), and is equipped with a Dropout layer to enhance generalization capabilities;

[0052] Alternating fusion mechanism: Each layer's input consists of the first or second half of the feature vector concatenated with the output of the previous layer to enhance contextual semantic modeling capabilities, unlike the standard residual connection structure.

[0053] Classification output layer: The final output is concatenated from multiple intermediate representations and input into a linear layer. The output dimension is the number of categories C = 10, which is used as logits for CrossEntropyLoss.

[0054] The entire system network is trained in an end-to-end manner, using a joint loss function to simultaneously optimize the two tasks of image reconstruction and semantic classification.

[0055] The beneficial effects of the present invention are as follows: This technical solution constructs an end-to-end image semantic communication system framework, integrating a saliency modeling mechanism, a chaotic perturbation encryption mechanism, and a multi-task joint optimization strategy. While ensuring the semantic expression capability of the image, a lightweight security perturbation mechanism is introduced to achieve basic information scrambling protection. Unlike traditional unified perturbation processing, this technical solution uses a saliency map to achieve "on-demand perturbation" of semantically critical areas, and combines a chaotic perturbation mask to achieve "controllable perturbation intensity", which not only enhances semantic effectiveness but also improves system robustness and anti-interception capabilities.

[0056] Furthermore, this technical solution utilizes a unified feature representation to complete both image reconstruction and image semantic recognition tasks, rather than the traditional system structure of independently trained task branches. This facilitates feature sharing and collaborative optimization, simplifying system complexity while improving the efficiency of inter-task collaboration. Compared to existing systems that rely on explicit transmission of perturbation information, this system's perturbation process is driven entirely by a shared key, with the perturbation strength implicitly encoded in the signal itself. This eliminates the need for additional perturbation maps and effectively reduces communication redundancy and the risk of leakage.

[0057] The specific outstanding advantages and innovations of this technical solution are:

[0058] (1) For the first time, a dual perturbation mechanism of "saliency map guidance + chaos perturbation control" is proposed to impose strong perturbations on semantically critical areas and achieve on-demand allocation of communication resources;

[0059] (2) The chaotic perturbation generation method driven by a shared key is unpredictable and highly sensitive, effectively improving the anti-eavesdropping and anti-reconstruction capabilities of the communication system;

[0060] (3) By implicitly encoding the perturbation parameters in the feature representation, the receiver can complete the inverse perturbation transform without transmitting the perturbation map, thus improving the system compactness and information security;

[0061] (4) Introducing a unified feature representation and multi-task joint training structure to significantly improve the synergy between image reconstruction and semantic recognition tasks;

[0062] (5) The overall structure supports end-to-end optimization training and can adapt to multiple modulation modes (BPSK, 4QAM, 16QAM, 64QAM, etc.), with strong system adaptability and practical deployment potential.

[0063] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0065] Figure 1 This is a schematic diagram of the overall structure of the image semantic joint communication system based on saliency-guided chaotic encryption of the present invention;

[0066] Figure 2 Schematic diagram of the encoder processing flow;

[0067] Figure 3 Schematic diagram of the processing flow of the shared descrambling module;

[0068] Figure 4 Schematic diagram of the visualization results of the significance guidance mechanism. DETAILED DESCRIPTION

[0069] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0070] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0071] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0072] See also Figures 1 to 4 ,This paper proposes an image semantic joint communication method based on ,saliency-guided chaotic encryption, which aims to realize the semantic recognition of ,images and the reconstruction of image content, and introduces a lightweight scrambling ,mechanism in the communication process to improve the security and robustness of the ,system under low signal-to-noise ratio and open channel.

[0073] In the implementation of this invention, the experimental data was selected from the public dataset CIFAR-10, which contains 10 image classification categories: airplane, car, bird, cat, deer, dog, frog, horse, boat, and truck. Each image has a resolution of 32×32 pixels and three color channels (RGB). This dataset was used as the basis for training and testing, verifying the quality of image reconstruction and evaluating the accuracy of image semantic classification.

[0074] The known parameters and variables involved in this embodiment are defined as follows:

[0075] n: the signal-to-noise ratio parameter value set;

[0076] seed: shared key seed;

[0077] Class: the number of classification categories, which is 10 in CIFAR-10;

[0078] r: Chaos parameter of the Logistic map, set r = 3.99;

[0079] X: input image;

[0080] Z3, Z4, Z5: semantic feature maps of different layers output by the encoder;

[0081] S: original saliency map;

[0082] B: batch size;

[0083] C: number of channels;

[0084] H: the height of the current feature map after downsampling;

[0085] W: the width of the current feature map after downsampling;

[0086] S′: final saliency map;

[0087] α′: perturbation factor calculated by the encoder;

[0088] M: chaos perturbation mask;

[0089] X′: perturbation feature map guided by saliency;

[0090] Z: signal obtained after modulation module;

[0091] Z′: signal after adding Gaussian white noise;

[0092] S″: normalized saliency map calculated by the shared descrambling module;

[0093] α′: the perturbation factor calculated by the shared descrambling module;

[0094] Z5′: preliminary recovery of original feature estimation;

[0095] fcomp(·): residual compensation subnetwork at the receiving end;

[0096] Z out : shared feature vector after descrambling;

[0097] L class : Cross entropy loss for semantic classification tasks;

[0098] L recon : Mean square error (MSE) loss for image reconstruction tasks;

[0099] λ: An adjustable hyperparameter that controls the weight of the two tasks and the optimization weight between the two tasks;

[0100] L total : Total loss function.

[0101] Based on the above scenario settings and parameter definitions, the implementation process of the present invention can be described as follows.

[0102] like Figure 1 As shown, the overall structure of the system of the present invention includes three parts: a transmitter, a channel transmission module, and a receiver. The transmitter includes an encoder (including a multi-scale feature extraction module, a feature alignment and upsampling module, a saliency map generation module, a saliency normalization module, a chaotic perturbation generation module, and a saliency guided perturbation module) and a modulation and coding module. The channel transmission module is a simulated additive white Gaussian noise channel (AWGN). The receiver includes a shared descrambling module, an image reconstruction decoder, and an image classification decoder.

[0103] After an image is input into the system, the encoder first extracts multi-scale features and assesses the importance of each region using a saliency mechanism. A shared chaotic key seed is then used to perform a saliency-guided perturbation on the feature map, achieving lightweight encryption. The scrambled features are then discretized, modulated, and mapped to channel symbols for transmission. The receiver then descrambles the image using the shared key, restoring the semantic feature representation for further image reconstruction and semantic classification.

[0104] like Figure 2 As shown in the figure, the structure of the encoder module is further refined into the following submodules:

[0105] 1) Multi-scale Feature Extraction Module: This module receives the input image X and extracts shallow structural information and deep semantic information through four levels of residual convolutional layers. The resulting intermediate features are named Z3, Z4, and Z5, with decreasing spatial resolution and increasing number of channels to cover multi-scale semantics.

[0106] 2) Feature alignment and upsampling module: Since the spatial sizes of Z3, Z4, and Z5 are inconsistent, they are aligned to a unified scale through upsampling (bilinear interpolation) to facilitate subsequent fusion operations.

[0107] 3) Saliency Map Generation Module: First, saliency is estimated for each of the three semantic feature maps at different scales. All saliency maps are then resized and concatenated. A fused saliency map is then output via 1×1 convolution and sigmoid activation. Each saliency estimator integrates multi-scale convolution, channel attention, and threshold prediction mechanisms to improve saliency modeling accuracy and semantic focus.

[0108] 4) Saliency Normalization Module: The original saliency map S is normalized to the final saliency map S′∈(0,1) using the Sigmoid activation function for subsequent perturbation intensity adjustment. Locations with high responses represent important areas in the image and will receive stronger perturbations.

[0109] 5) Chaotic perturbation encryption module: Let the shared chaotic key seed seed∈(0,1) be the shared input to initialize the Logistic chaotic perturbation mask. The chaotic perturbation mask M ultimately has the same spatial size and number of channels as Z5. The perturbation value of each pixel fluctuates between (0,1) and has high entropy, high sensitivity, and unpredictability. The generating function of M is as follows:

[0110]

[0111] 6) Saliency-guided perturbation module: The normalized saliency map S′ is multiplied by the chaotic perturbation mask M to form a joint perturbation factor α, and then the deep feature map Z5 is perturbed in a position-controlled manner as follows:

[0112] α=M×S′ (2)

[0113] X′=Z5×(1+α) (3)

[0114] The perturbed feature map X′ is not directly used as the transmission signal, but is further processed as an intermediate representation. First, X′ is flattened and input into a linear layer, mapped into a continuous feature vector. Hard sampling is performed using Gumbel Softmax to obtain discrete symbols. The discrete symbols are then encoded into signal Z through a modulation module (BPSK modulation, 4-QAM modulation, 16-QAM modulation, or 64-QAM modulation). Signal Z is then sent to the channel transmission module, where additive white Gaussian noise (AWGN) interference is simulated:

[0115] Z′=Z+n (4)

[0116] Among them, n~N(0,σ 2 ), indicating that the source of n is zero mean and variance is σ 2 Gaussian process, σ 2 The signal-to-noise ratio n is set to be consistent with the actual wireless communication environment.

[0117] At the receiving end, Figure 3 As shown in FIG, according to the shared chaotic key seed seed consistent with the transmitter, the perturbation mask M is generated by the same chaotic mapping function (Logistic mapping), and the perturbation factor α′ is calculated by element-wise multiplication with the normalized saliency map S″ calculated by the shared descrambling module:

[0118] α′=M×S″ (5)

[0119] It is used for descrambling operation, and the input feature vector Z′ is subjected to inverse perturbation transformation according to the following formula to preliminarily restore the original feature estimate:

[0120]

[0121] This step is equivalent to the inverse operation of the scrambling operation, which can initially restore the original semantic features. Considering that the nonlinear loss introduced by the perturbation process cannot be fully recovered by simple division, the system further designs a compensation sub-network to perform fine-grained repair on the descrambled features:

[0122] Z out =Z5′+fcomp(Z′) (7)

[0123] Among them, fcomp(·) is the residual compensation subnetwork at the receiving end. Its structure consists of two layers of 3×3 convolution and ReLU activation function. Its function is to predict the distribution of perturbation error based on the perturbation input Z′ and perform additive compensation to improve the descrambling accuracy.

[0124] Then the descrambled eigenvector Z out The two decoder modules are input to complete the two tasks of image reconstruction and semantic classification. The two decoder structures are independent of each other, but both rely on the shared feature vector Z out , realizing feature sharing and collaborative optimization between tasks.

[0125] The image reconstruction decoder (Decoder_Recon) is used to descramble the perturbation feature vector Z out Restore the original image to its pixel-level representation output. Its structure consists of the following submodules:

[0126] 1) Feature Restructuring Module: The input one-dimensional vector Z out Mapped to the initial feature map tensor Z0 through the linear layer, its shape is B×C×H×W;

[0127] 2) Upsampling module 1: Use the DepthToSpace module to upsample the feature map spatial size to a medium resolution;

[0128] 3) Channel Attention Module: This module is used to perform weighted adjustments between channels in the upsampled feature map to highlight important semantic channels.

[0129] 4) Upsampling module 2: Use the DepthToSpace module again to restore the feature map to the target image space size;

[0130] 5) Output layer: Use 3×3 convolution and Tanh activation function to compress the number of channels to 3 and output the final RGB reconstructed image.

[0131] The image classification decoder (Decoder_Class) is used to restore the semantic features Z after disturbance out Perform global representation extraction and output the final semantic category prediction result Y∈R Class The structure is as follows:

[0132] 1) Multilayer Perceptron Architecture: Consists of four fully connected submodules, each of which includes a linear layer (Linear), batch normalization (BatchNorm1d), a learnable activation function (PReLU), and a dropout layer to enhance generalization capabilities;

[0133] 2) Alternating Fusion Mechanism: Each layer's input consists of the concatenation of the first and second halves of the feature vector and the output of the previous layer to enhance contextual semantic modeling capabilities. This differs from the standard residual connection structure.

[0134] 3) Classification output layer: The final output is concatenated from multiple intermediate representations and input into a linear layer. The output dimension is the number of categories C = 10, which is used as logits for CrossEntropyLoss.

[0135] The training of the entire network is performed in an end-to-end manner, using the joint loss function L total Simultaneously optimize the two tasks of image reconstruction and semantic classification:

[0136] L total =L class +λ×L recon (8)

[0137] Therefore, this joint optimization strategy encourages the model to maximize feature reuse and optimize communication resource efficiency while simultaneously improving image fidelity and semantic accuracy. Experiments have shown that the setting of λ has a regulatory effect on task emphasis: when λ is small, the model prioritizes semantic classification accuracy; when λ is large, it prioritizes image fidelity. Furthermore, to improve convergence stability and training efficiency, the system employs a cosine annealing scheduler to dynamically adjust the learning rate and periodically saves the optimal model during training for easy restoration during testing.

[0138] Figure 4 This demonstrates the visualization effect of the saliency mechanism of the present method. To verify the effectiveness of the saliency map generation module, this example visualizes the saliency map output by the system during the test phase. The figure shows 16 randomly selected image samples from the CIFAR-10 test set, arranged in 4 rows and 4 columns, forming a total of 16 image pairs.

[0139] Each set of images contains two sub-images, namely:

[0140] Left image: original input image;

[0141] Right image: The normalized saliency response map S′ output by the encoder is bilinearly interpolated to the original image size and visualized as a heat map.

[0142] Visualization results show that the saliency map accurately captures key semantic regions within an image. For example, in images of "plane," "frog," "boat," and "dog," saliency values ​​are generally concentrated in the main body of the object, while background regions such as sky, grass, and water are less responsive. The system's saliency map is highly activated in the outline of the target subject, while the background is less responsive, demonstrating good semantic focus.

[0143] In addition, the saliency map has spatial selectivity and category consistency, which proves that the saliency guidance mechanism of the present invention can indeed impose stronger perturbations on the "important areas" in image features rather than uniform perturbations, thereby realizing on-demand allocation of communication resources and effectively improving the efficiency of semantic expression.

[0144] This invention, by introducing a saliency guidance mechanism and a chaotic perturbation encryption mechanism, achieves selective scrambling of important semantic regions in an image semantic communication system. The perturbation mask generated based on a shared key is controllable and unpredictable, effectively enhancing data security during channel transmission. Furthermore, the system uses unified coding features to achieve both image reconstruction and semantic recognition, balancing fidelity and semantic accuracy.

[0145] The system of the present invention has the ability to jointly express image pixel information and semantic labels, effectively completing image reconstruction and image semantic recognition tasks. The method described in this invention can effectively improve the communication system's expressiveness, security, and anti-interference capabilities without adding additional transmission burden. It is suitable for image semantic transmission scenarios with low signal-to-noise ratios and high security requirements.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for joint image semantic communication based on saliency-guided chaotic encryption, characterized by: The method specifically comprises the following steps: S1: The image is input into the encoder, and the encoder's multi-scale feature extraction module extracts the multi-scale semantic features of the image; S2: The semantic features of different scales are fused through the saliency map generation module to obtain the saliency map S; S3: Input the feature map and the saliency map into the chaotic perturbation encryption module, generate the chaotic perturbation mask M according to the shared chaotic key seed, and obtain the encrypted feature; S4: Flatten, discretize, encode and modulate the encrypted features to convert them into a signal Z suitable for channel transmission; S5: Transmit the signal Z to the receiving end through the channel while being interfered by Gaussian white noise; S6: The receiver reconstructs the chaotic perturbation mask based on the same shared chaotic key seed and completes the inverse perturbation transform in combination with the local saliency map to restore the original semantic feature estimate; S7: The recovered features are input into the image reconstruction decoder and the image classification decoder respectively to obtain the reconstructed image and semantic category prediction label.

2. The image semantic joint communication method according to claim 1, characterized in that: A communication system suitable for this method includes a transmitter, a channel transmission module and a receiver; the transmitter includes an encoder and a modulation coding module; the encoder includes a multi-scale feature extraction module, a feature alignment and upsampling module, a saliency map generation module, a saliency normalization module, a chaotic perturbation generation module and a saliency guided perturbation module; the channel transmission module is a simulated additive white Gaussian noise channel; the receiver includes a shared descrambling module, an image reconstruction decoder and an image classification decoder.

3. The image semantic joint communication method according to claim 2, characterized in that: The multi-scale feature extraction module receives the input image X and extracts shallow structural information and deep semantic information through four levels of residual convolution layers; the obtained intermediate features are named Z3, Z4, and Z5, with decreasing spatial resolution and increasing number of channels to cover multi-scale semantics; The feature alignment and upsampling module: Since the spatial sizes of Z3, Z4, and Z5 are inconsistent, they are aligned to a unified scale through upsampling; The saliency map generation module first performs saliency estimation on semantic feature maps of three different scales, then unifies the scale of all saliency maps and concatenates them, and then outputs the fused saliency map through 1×1 convolution and Sigmoid activation. The saliency normalization module: normalizes the original saliency map S to the final saliency map S′∈(0,1) through the Sigmoid activation function; The chaotic perturbation encryption module: set the key seed seed∈(0,1) as a shared input to initialize the Logistic chaotic perturbation mask. The chaotic perturbation mask M ultimately has the same spatial size and number of channels as Z5, and the perturbation value of each pixel fluctuates between (0,1); The saliency-guided perturbation module multiplies the normalized saliency map S′ with the chaotic perturbation mask M to form a joint perturbation factor α, and then performs position-controlled perturbation on the deep feature map Z5. The operation is as follows: α=M×S′ X′=Z5×(1+α) Among them, X′ is the feature map after perturbation.

4. The image semantic joint communication method according to claim 3, characterized in that: In the saliency map generation module, each saliency estimator integrates multi-scale convolution, channel attention and threshold prediction mechanisms.

5. The image semantic joint communication method according to claim 3, characterized in that: In the chaotic perturbation encryption module, the generation function of the chaotic perturbation mask M is as follows: Among them, M0 is the initial value of the iterative sequence, M n is the result value obtained after the nth iteration, seed is the shared chaotic key seed, and r is the chaotic parameter of the Logistic mapping.

6. The image semantic joint communication method according to claim 3, characterized in that: The modulation coding module specifically flattens the perturbed feature map X′ and inputs it into the linear layer, maps it into a continuous feature vector, performs hard sampling using Gumbel Softmax, and obtains discrete symbols; the discrete symbols are encoded into signal Z through the modulation coding module.

7. The image semantic joint communication method according to claim 2, characterized in that: The shared descrambling module specifically uses the shared chaotic key seed seed that is consistent with the transmitting end, generates a chaotic perturbation mask M through the same chaotic mapping function, and calculates the perturbation factor α′ by element-wise multiplication with the normalized saliency map S″ calculated by the shared descrambling module. The calculation formula is: α′=M×S″ It is used for descrambling operation, and the input feature vector Z′ is subjected to inverse perturbation transformation according to the following formula to preliminarily restore the original feature estimate Z5′: Perform fine-grained repair on the descrambled features: From out =Z5′+fcomp(Z′) Among them, Z out is the descrambled feature vector, fcomp(·) is the residual compensation subnetwork at the receiving end, which consists of two layers of 3×3 convolution and ReLU activation function. Its function is to predict the distribution of perturbation error based on the input feature vector Z′ and perform additive compensation to improve the descrambling accuracy.

8. The image semantic joint communication method according to claim 7, characterized in that: The image reconstruction decoder is used to descramble the perturbation feature vector Z out Restore the original image to its pixel-level representation output, including: Feature reshaping module: the input one-dimensional vector Z out Mapped to the initial feature map tensor Z0 through the linear layer, its shape is B×C×H×W, where B, C, H, and W are the batch size, number of channels, height of the current feature map after downsampling, and width of the current feature map after downsampling, respectively; Upsampling module 1: Use the depth-to-spatial conversion module to upsample the feature map spatial size to a medium resolution; Channel attention module: The channel attention module is introduced to perform weighted adjustment between channels of the upsampled feature map; Upsampling module 2: Use the depth-to-spatial conversion module again to restore the feature map to the target image space size; Output layer: Use 3×3 convolution and Tanh activation function to compress the number of channels to 3 and output the final RGB reconstructed image.

9. The image semantic joint communication method according to claim 7, characterized in that: The image reconstruction decoder is used to descramble the perturbed feature vector Z out Perform global representation extraction and output the final semantic category prediction result Y∈R Class , specifically including: Multilayer Perceptron architecture: Consists of four fully connected submodules, each of which contains a linear layer, batch normalization, and a learnable activation function, and is equipped with a Dropout layer to enhance generalization capabilities; Alternating fusion mechanism: The input of each layer is composed of the first or second half of the feature vector concatenated with the output of the previous layer; Classification output layer: The final output is concatenated from multiple intermediate representations and then input into a linear layer. The output dimension is the number of categories C.

10. The image semantic joint communication method according to claim 2, characterized in that: The entire system network is trained in an end-to-end manner, using a joint loss function to simultaneously optimize the two tasks of image reconstruction and semantic classification.

Images