A method and system for protecting image semantic communication security based on semantic noise

Abstract

The application discloses a kind of based on image semantic communication security protection method of semantic noise, which includes the following steps: original input image is encoded after obtaining original semantic information by semantic encoder;Obtain noisy semantic information, construct and train generative adversarial network, input noisy semantic information into trained generative adversarial network to generate false image;Construct feature enhancement network and train, input false image into trained feature enhancement network, generate enhanced generated image;Based on enhanced generated image, construct adversarial disturbance, map adversarial disturbance to semantic space and carry out power normalization, obtain semantic noise;Noisy semantic information is mixed with semantic noise, to obtain mixed transmission information.The application can independently generate semantic noise with active camouflage ability under the condition of only having the right to use semantic encoder, to realize the effective protection to transmission semantic information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image semantic wireless communication, and particularly relates to an image semantic communication security protection method and system based on semantic noise. BACKGROUND

[0002] A semantic communication system realizes extraction and reconstruction of key semantic content in original data through a semantic encoder and a decoder constructed by a deep neural network. However, the current semantic communication system based on deep learning faces serious information security threats when transmitting the extracted semantic representation in an open wireless channel. Although the semantic encoding process has highly abstracted the original data, making it difficult to directly restore the original content, studies have shown that a malicious eavesdropper can still train an independent semantic inversion classifier through a publicly available dataset to infer the corresponding semantic category or high-level semantic attribute from the intercepted semantic representation, thereby causing privacy leakage. Therefore, it is crucial to protect the semantic representation transmitted in an open channel.

[0003] In existing secure communication schemes, such as a semantic camouflage communication method, system, power grid and electronic device disclosed in patent No. CN119697327A, the method protects the transmitted semantic information by introducing semantic noise, so that the legitimate receiving party (i.e. the target decoder) can correctly restore the original semantic content, while potential eavesdroppers will obtain incorrect or misleading semantic information when attempting to decode, thereby achieving communication security. However, this technical solution has significant limitations in practical application. The generation of semantic noise depends on accurate semantic understanding of the sent content (such as images), including specific content and its label information, and also requires joint retraining of the encoder and decoder in the deployed semantic communication system to ensure that the legitimate receiving end can still accurately decode, which poses multiple challenges in real-world scenarios: first, users usually have high requirements for the privacy of the transmitted content and are unwilling or unable to provide detailed semantic information of the original data to the noise generation module, thereby limiting the applicability of the method; second, in many actual deployments of semantic communication systems, especially in systems with distributed architecture (such as the encoder and decoder located in different geographical locations), joint retraining of the encoder and decoder not only requires a large amount of communication overhead and computing resources, but also may cause system service interruption, resulting in significant increase in time and economic costs, and even completely unfeasible in some cases.

[0004] As a kind of anti-frequency analysis searchable encryption method and system disclosed in patent No. CN120567407A, the method adopts a certificateless encryption system, which takes into account the application requirements of limited computing resources while ensuring security. Although this method can effectively prevent encrypted data from leaking sensitive information after being intercepted, the additional computational overhead caused by the introduction of ciphertext expansion and encryption / decryption process makes it difficult to meet the real-time transmission requirements of image semantic wireless communication systems for low latency and high efficiency. Therefore, traditional encryption mechanisms have limitations in semantic communication scenarios, and directly encrypting semantic content is not an ideal security solution.

[0005] In particular, in the current mainstream Terminal-Edge-Cloud distributed semantic communication architecture, the above problems are particularly prominent: semantic encoders are usually deployed on user terminal sides (such as mobile phones, cameras), while semantic decoders are located on edge servers or clouds, and the two are controlled by different subjects, making it impossible to share model parameters or jointly train them. At the same time, users are reluctant or unable to provide original image content and its label information to third parties due to privacy concerns. Therefore, existing security solutions are not applicable or completely infeasible under this typical architecture.

[0006] In summary, there is an urgent need for a lightweight security protection mechanism that does not require original data, decoder prior knowledge or joint training to promote the safe landing of semantic communication technology under the Terminal-Edge-Cloud architecture. SUMMARY

[0007] To overcome the deficiencies and shortcomings of the prior art, the present application provides an image semantic communication security protection method and system based on semantic noise, which only needs to access the deployed semantic encoder (without original image, without label, without decoder model or parameter) to generate strong misleading semantic noise autonomously, which can protect the transmission of semantic representation. Under the condition of only having the right to use the semantic encoder, the present application can autonomously generate semantic noise with active camouflage capability to effectively protect the transmission of semantic information and promote the safe landing of semantic communication technology under the Terminal-Edge-Cloud architecture.

[0008] To achieve the above purpose, the present application adopts the following technical solutions:

[0009] The present application provides an image semantic communication security protection method based on semantic noise, comprising the following steps:

[0010] The original input image is encoded by a semantic encoder to obtain original semantic information;

[0011] Obtain noisy semantic information, construct a generative adversarial network and train it, and input the noisy semantic information into the trained generative adversarial network to generate a fake image;

[0012] constructing a feature enhancement network and training the same, inputting the false image into the trained feature enhancement network to generate an enhanced generated image;

[0013] constructing an adversarial perturbation based on the enhanced generated image, mapping the adversarial perturbation to a semantic space and performing power normalization to obtain semantic noise;

[0014] superimposing the noisy semantic information and the semantic noise to obtain superimposed transmission information.

[0015] As a preferred technical solution, the adversarial generative network comprises a generator and a discriminator;

[0016] dimensionally splicing the noisy semantic information and a random noise vector to obtain a one-dimensional vector, the generator being provided with a plurality of cascaded convolutional blocks, the one-dimensional vector being expanded into a spatialized image tensor, the image tensor being mapped from a low-dimensional joint representation space to a high-dimensional image space, and the generator outputting a false image;

[0017] The discriminator is provided with a plurality of linear layers and activation functions for evaluating the consistency of the high-dimensional image space and the original semantic representation.

[0018] As a preferred technical solution, when training the adversarial generative network, a total loss function is constructed based on a self-supervised adversarial loss and a mutual information adversarial loss, and the adversarial generative network is trained based on the total loss function.

[0019] As a preferred technical solution, the feature enhancement network is constructed and trained, specifically including:

[0020] constructing an adjacent sampler for sampling two similar but different sub-images from the same false image, and taking the obtained sub-images as a training data set of the feature enhancement network;

[0021] The feature enhancement network adopts an encoder-decoder structure, the encoder performs feature transformation through convolution and nonlinear activation, the decoder is connected to the encoder through a skip connection, and the output features are mapped back to the original image channel space through a convolution layer;

[0022] The feature enhancement network is trained based on a self-supervised denoising learning paradigm.

[0023] As a preferred technical solution, when training the feature enhancement network, a composite loss function is constructed based on a similarity reconstruction loss and an ideal denoising loss, and the feature enhancement network is trained based on the composite loss function.

[0024] As a preferred technical solution, the adversarial perturbation is constructed based on the enhanced generated image, specifically including:

[0025] assigning pseudo labels to the enhanced generated image to obtain a training set;

[0026] An alternative classifier is trained based on the training set, and adversarial perturbations are calculated.

[0027] As a preferred technical solution, the alternative classifier employs a classification network with multiple stacked convolutional layers and ReLU function as the activation layer.

[0028] As a preferred technical solution, the method for countering disturbances is represented as follows:

[0029] ;

[0030] in, This indicates resistance to disturbances. Indicates the step size of the adversarial example. Represents a symbolic function. Indicates the enhanced generated image gradient operator, Indicates an alternative classifier. Represents the cross-entropy loss function. This indicates a pseudo-tag.

[0031] The present invention also provides an image semantic communication security protection system based on semantic noise, used to implement the above-mentioned image semantic communication security protection method based on semantic noise. The system includes: an original semantic information generation module, a noisy semantic information acquisition module, a fake image generation module, a fake image enhancement module, an anti-disturbance construction module, a semantic noise generation module, and a transmission information generation module.

[0032] The original semantic information generation module is used to encode the original input image using a semantic encoder to obtain the original semantic information;

[0033] The noisy semantic information acquisition module is used to acquire noisy semantic information;

[0034] The fake image generation module is used to input noisy semantic information into the trained adversarial generative network to generate fake images.

[0035] The fake image enhancement module is used to input fake images into the trained feature enhancement network to generate enhanced generated images.

[0036] The adversarial perturbation construction module is used to construct adversarial perturbations based on the enhanced generated image;

[0037] The semantic noise generation module is used to map adversarial perturbations to the semantic space and normalize their power to obtain semantic noise;

[0038] The transmission information generation module is used to mix noisy semantic information with semantic noise to obtain mixed transmission information.

[0039] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0040] (1) The present application has black box compatibility without semantic decoder prior knowledge. Under the condition of completely unknown legal semantic decoder structure and parameters, without the user submitting the original image or semantic label, the present application can still generate adaptive adversarial perturbation for the current transmission content, effectively generate semantic noise with protection ability, realize the security enhancement of semantic information, without modifying the original semantic encoder or semantic decoder, and without introducing additional encryption and decryption process, thus having little influence on transmission delay and image reconstruction performance, and having good practicability and deployment compatibility.

[0041] (2) For the problem of not being able to obtain the original image, the present application constructs a joint framework based on a self-supervised generative adversarial network and a feature enhancement network, reverses the noisy semantic representation into a false image, iteratively optimizes the semantic noise, and realizes accurate modeling of the semantic space disturbance. This process does not depend on any real image supervision, and protects the user's privacy.

[0042] (3) The semantic noise of the present application directly acts on the semantic feature space and is superimposed and transmitted with the original semantic information in the channel. The semantic noise can effectively interfere with the attack behavior of the eavesdropper using semantic representation for classification or semantic inference while maintaining the image reconstruction quality of the legal receiving end. Unlike the method of content pre-protection at the original input end, the present application does not depend on the specific structure of the semantic encoder, has good generalization ability for various network architectures and semantic encoding methods, is suitable for heterogeneous, dynamic or deployed semantic communication systems, and significantly improves the strong generalization and active defense ability. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 The flowchart of the image semantic communication security protection method based on semantic noise of the present application is shown in the figure;

[0044] Figure 2 The network architecture diagram of the self-supervised generative adversarial network and the feature enhancement network of the present application is shown in the figure;

[0045] Figure 3 The application scenario diagram of the image semantic communication security protection method based on semantic noise of the present application is shown in the figure;

[0046] Figure 4 The comparison diagram of the reconstructed visual effect of the present application is shown in the figure; DETAILED DESCRIPTION

[0047] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0048] Embodiment 1

[0049] As Figure 1 shown, the embodiment provides a semantic noise-based image semantic communication security protection method, including the following steps:

[0050] S1: The original input image is encoded by a semantic encoder to obtain original semantic information, and the original semantic information is transmitted in a wireless open channel, which is specifically represented as:

[0051] ;

[0052] Among them, denotes the original input image, , , , denote the length, width and channel number of the image respectively, denotes a pre-trained and frozen parameter semantic encoder, denotes semantic information extracted by the semantic encoder, , is the feature dimension number;

[0053] S2: Obtain the noisy semantic information of the wireless open channel, construct a self-supervised generative adversarial network and train it, input the noisy semantic information into the trained generative adversarial network to generate a false image, which specifically includes:

[0054] S21: In this embodiment, due to the requirement of privacy protection, the relevant information of the original input image cannot be obtained, only the semantic information in the wireless open channel can be obtained, and the obtained noisy semantic information is affected by the wireless channel noise, which is represented as:

[0055] ;

[0056] Among them, denotes the original semantic information, denotes channel additive white Gaussian noise;

[0057] S22: As Figure 2 shown, the self-supervised generative adversarial network includes a generator G and a discriminator D, wherein the generator G is not used for image reconstruction or image generation task, but as a pre-modeling tool for semantic space disturbance, its goal is to generate a semantic equivalent image which is distributed in the encoder output space with the original semantic representation, but completely irrelevant in the pixel space. This image is not used for visual display, but as an intermediate agent data for subsequent semantic noise construction, so as to avoid direct dependence on the original input image or the decoder for disturbance generation.

[0058] In this embodiment, the noisy semantic information is sampled from a Gaussian distribution with zero mean and high variance , and is concatenated with the dimension to obtain a one-dimensional vector , where is the number of dimensions, and the one-dimensional vector is input to the generator G, and the generator network structure is defined as a mapping function: , i.e., mapping from a low-dimensional joint representation space to a high-dimensional image space, outputting a fake image;

[0059] In this embodiment, the network is stacked by three cascaded convolutional blocks (k = 1, 2, 3), forming a deep nonlinear decoding network, which gradually expands the one-dimensional vector into a spatialized image tensor, and the internal structure of each convolutional block is as follows:

[0060] ;

[0061] where is the initial input vector, represents a 4x4 transposed convolutional layer with a stride of 2 and a padding of 1, Linear is a linear mapping layer (fully connected layer) used to adjust the number of channels or feature dimensions, LN is a layer normalization (Layer Normalization) used to stabilize the training process, especially suitable for small batch or single sample input scenarios, and ReLU is an activation function that introduces nonlinear expression capability;

[0062] Specifically, the output feature maps of each convolutional block change as follows:

[0063] First block: map the initial feature map to through a fully connected layer ;

[0064] Second block: perform transposed convolution upsampling on , output ;

[0065] Third block: continue upsampling, output , i.e., the final fake image ;

[0066] In this embodiment, the fake image There is no direct pixel-level similarity with the original input image in visual appearance (i.e. no leakage of original content), but through subsequent adversarial loss and mutual information constraint, it is ensured that it is highly consistent with the original semantic representation in the high-level semantic space, thereby serving as a semantic equivalent proxy data in the semantic noise generation process.

[0067] S23: In this embodiment, in order to realize effective supervision of the generation process, a double learning mechanism of adversarial distribution alignment and variational mutual information maximization is introduced, and a total loss function is constructed based on self-supervised adversarial loss and mutual information adversarial loss, specifically including:

[0068] Unlike the function of the discriminator in traditional GAN to distinguish true and false images, the discriminator in this embodiment is composed of multiple linear layers and Sigmoid activation function, and its function is to evaluate the distribution similarity of two semantic vectors, therefore, the self-supervised adversarial loss constructed is:

[0069] ;

[0070] Wherein, E represents mathematical expectation, the self-supervised adversarial loss drives the generator to generate output that is difficult to distinguish from the original noisy semantics in the semantic encoding space, thereby realizing the alignment in the semantic distribution level rather than the real feeling in the image level;

[0071] The mutual information adversarial loss is represented as:

[0072] ;

[0073] Wherein, LMI represents the lower bound of mutual information, Ladv represents the standard adversarial loss, λ represents a hyperparameter, D represents the discriminator, G represents the generator, MI represents the mutual information estimator, which is used to balance the adversarial loss and the mutual information regularization term. It forces the generator to retain the key structure of the input semantic information and prevent semantic drift;

[0074] The generator and the discriminator are trained, and based on the trained generator, a false image is generated:

[0075] ;

[0076] S3: As shown in Figure 2 , a feature enhancement network is constructed and trained, and the false image is input into the trained feature enhancement network to generate an enhanced generated image;

[0077] The fake image generated in step S2 maintains high consistency with the original input image in the semantic space, but due to the complete dependence of the generation process on the inversion reconstruction of the noisy semantic information, the lack of local texture and structure prior of the real image, there are problems such as blur, distortion or loss of high-frequency details in the pixel space, therefore, if the semantic noise is directly constructed based on the fake image, the disturbance quality may be reduced and the misleading effect on the eavesdropper may be affected, in order to improve the visual structural integrity and local consistency of the fake image, feature enhancement needs to be performed on it, however, since the original input image belongs to user privacy content, it cannot be used for supervised training, and traditional methods based on pixel-level reconstruction loss (such as L1 / L2 loss) or perceptual loss are not applicable. Therefore, the embodiment constructs a completely self-supervised feature enhancement network , the training process of which does not depend on any real image label or external supervision signal, and only uses the local structural consistency of the fake image itself to construct a pseudo-supervised task, specifically including:

[0078] S31: Constructing a self-supervised training data set;

[0079] In the embodiment, a neighboring sampler is constructed, where, , are two similar samplers, respectively, for sampling two similar but different subgraphs from the same fake image, and the obtained subgraphs are used as the training data set of the feature enhancement network , , the specific steps are as follows:

[0080] (1) The input fake image is divided into non-overlapping windows;

[0081] (2) Randomly select two adjacent pixel positions , in each window, which satisfy , respectively, as the sampling points of the two subgraphs;

[0082] (3) Scan the entire image with a step size to generate two down-sampled subgraphs , , which constitute an unlabeled image pair sample, this strategy is based on the local smoothness assumption, that is, adjacent pixels in the same small area have highly similar semantic and texture features, therefore, , can be regarded as noise versions of each other, which are suitable for constructing a denoising type self-supervised task;

[0083] S32: Constructing a feature enhancement network , adopts an encoder-decoder structure and a skip connection mechanism. In the nth layer of the encoder, the feature map is transformed through convolution and nonlinear activation, represented as:

[0084] ;

[0085] wherein, is the input image, is the output feature map of the nth layer, with the spatial resolution halved layer by layer ( , ) and the number of channels increasing, represents a 3x3 convolution operation, represents batch normalization, represents an activation function;

[0086] Each layer is followed by a 2x2 max-pooling operation to reduce the resolution:

[0087] ;

[0088] The deepest layer semantic feature is obtained;

[0089] The decoder starts from the deepest layer feature , gradually restores the spatial resolution through upsampling. In the nth layer of the decoder, the output of the previous layer is first upsampled:

[0090] ;

[0091] Then, it is concatenated with the feature map of the nth layer of the encoder through a skip connection:

[0092] ;

[0093] wherein, represents concatenation in the channel dimension, represents the fused feature;

[0094] Subsequently, the fused feature is extracted through a convolution layer:

[0095] ;

[0096] This process preserves the high-frequency spatial information (such as edges and textures) in the encoder and reuses it in the decoding process, effectively alleviating the loss of details caused by multiple downsampling;

[0097] Finally, the decoder outputs the feature ( is the number of channels in the last layer), which is mapped back to the original image channel space through a 1x1 convolution layer:

[0098] = ;

[0099] i.e.: ;

[0100] S33: training the feature enhancement network;

[0101] In this embodiment, a self-supervised denoising learning paradigm is adopted, and a composite loss function is represented as:

[0102] ;

[0103] wherein, is a balance hyperparameter, ;

[0104] The similarity reconstruction loss is represented as:

[0105] ;

[0106] This term forces the network to recover the corresponding version of its spatial neighborhood from one subgraph to the other , achieving local structure enhancement;

[0107] The ideal denoising loss is represented as:

[0108] ;

[0109] This term introduces a sampling consistency constraint, ensuring that the network's enhanced output still maintains local structure stability under adjacent sampling, preventing the enhancement process from introducing artifacts or destroying semantic consistency;

[0110] In this embodiment, by minimizing the composite loss on a large number of , generated by adjacent samplers, the feature enhancement network learns to recover enhanced images with reasonable texture and edge structure from blurred, distorted, and false images without accessing any real original input images;

[0111] S34: After the feature enhancement network is trained, the feature enhancement capability is obtained, the noise part of the input image is suppressed to obtain a clearer image, and an enhanced generated image is output, which is specifically represented as:

[0112] ;

[0113] wherein, retains the global semantic information of the false image , while significantly improving the local visual quality, providing a high-quality data basis for subsequent semantic noise generation;

[0114] S4: To further enhance the ability of semantic noise to mislead eavesdroppers, a transferable adversarial perturbation is constructed based on the enhanced generated image. Its generation process is completely separate from the original input image, thus protecting user privacy.

[0115] In this embodiment, the alternative classification network is trained using a generated image overlay fake label technique, specifically including:

[0116] S41: Constructing the training set ;

[0117] Since real image labels are unavailable, a pseudo-label self-supervised strategy is employed to assign semantic categories to the augmented generated images. Specifically, a pre-trained classifier is used. Assign pseudo-labels to the enhanced generated image, represented as:

[0118] ;

[0119] in, Pseudo-labels, classifier The preferred classifier is ResNet50.

[0120] S42: Construct an alternative classifier. The alternative classifier structure uses a classification network with four stacked convolutional layers and ReLU function as the activation layer, based on the training set. Train an alternative classifier to approximate the classification behavior in a real semantic communication system;

[0121] S43: Calculate adversarial perturbations based on alternative classifiers, expressed as:

[0122] ;

[0123] in, This indicates resistance to disturbances. Indicates the enhanced generated image gradient operator, Indicates an alternative classifier. Represents a symbolic function. Represents the cross-entropy loss function. Indicates the step size of the adversarial example;

[0124] In this embodiment, to ensure that the disturbance conforms to the transmission characteristics of the semantic communication system, the following processing is further performed to counteract the disturbance:

[0125] Semantic space projection: Mapping adversarial perturbations to the semantic space.

[0126] ;

[0127] Power normalization is performed, and semantic noise generation must satisfy power constraints, expressed as:

[0128] ;

[0129] wherein, Pmaxdenotes the maximum allowed injection power, is a variance function, denotes the injection power;

[0130] In this embodiment, the training strategy of the self-supervised generative adversarial network and the feature enhancement network adopts a three-stage optimization:

[0131] First stage: training the self-supervised generative adversarial network to convergence;

[0132] Second stage: freezing the parameters of the self-supervised generative adversarial network, and independently training the feature enhancement network;

[0133] Third stage: jointly optimizing the generator of the self-supervised generative adversarial network and the feature enhancement network;

[0134] In this embodiment, the Adam trainer is used in the training process, and the learning rate is ;

[0135] S5: semantic noise injection and semantic decoding;

[0136] After the generation of the semantic noise is completed, the content to be transmitted can be directly protected on the channel, the semantic noise is superimposed with the noisy semantic information to obtain the superimposed transmission information, denoted as:

[0137] ;

[0138] wherein, denotes the superimposed transmission information, denotes the noisy semantic information, denotes the semantic noise;

[0139] Since the semantic noise is specially designed for the eavesdropper to obtain the semantic information content by using an illegal classifier, the reconstruction picture task of the legal decoder has no obvious influence, and the decoded image can be directly decoded:

[0140] ;

[0141] wherein, denotes the legal decoder, denotes the decoded image;

[0142] As Figure 3As shown, the semantic information is calculated to obtain the semantic noise through the self-supervised adversarial neural network and the feature enhancement network, the pre-calculated semantic noise is injected into the channel when the data transmission is detected, the semantic noise is mixed with the noisy semantic information, and after the eavesdropper eavesdrops the mixed information, the illegal classification network of the eavesdropper cannot obtain the effective message, because the task of the semantic decoder is not a classification task but an image recovery, and the task of the illegal classification network of the eavesdropper is a classification task, for example, if the actually transmitted is an image of a horse, the eavesdropper eavesdrops an image of an airplane.

[0143] As shown in the following table 1, for the reconstruction effect and the effect of misleading the eavesdropper's classifier, the present application gives specific numerical comparison: Figure 4 As shown in the following table 1, for the reconstruction effect and the effect of misleading the eavesdropper's classifier, the present application gives specific numerical comparison:

[0144] As shown in the following table 1, for the reconstruction effect and the effect of misleading the eavesdropper's classifier, the present application gives specific numerical comparison:

[0145] Table 1 Effect index comparison data table

[0146]

[0147] Among them, the first column from the left represents the method of generating semantic noise, the second, third and fourth columns represent the image indicators recovered by the legal receiver for the features added with noise, wherein the higher the peak signal-to-noise ratio and the structural similarity index, the smaller the mean square error, the better the quality of the recovered picture, and the fifth column represents the accuracy rate of the eavesdropper for classifying the features protected by the semantic noise after eavesdropping, the lower the accuracy rate, the better the protection effect, it can be seen that the method of the embodiment achieves the best performance in the image quality recovered by the legal user and the accuracy rate of the eavesdropper for classification, and is superior to the other methods.

[0148] Embodiment 2

[0149] The embodiment provides an image semantic communication security protection system based on semantic noise, which is used for the image semantic communication security protection method based on semantic noise in the above embodiment 1, and the system comprises an original semantic information generation module, a noisy semantic information acquisition module, a false image generation module, a false image enhancement module, an adversarial disturbance construction module, a semantic noise generation module and a transmission information generation module.

[0150] In the embodiment, the original semantic information generation module is used to obtain original semantic information by encoding the original input image through a semantic encoder.

[0151] In the embodiment, the noisy semantic information acquisition module is used to acquire noisy semantic information.

[0152] In the embodiment, the false image generation module is configured to input the noisy semantic information into the trained generative adversarial network to generate a false image;

[0153] In the embodiment, the false image enhancement module is configured to input the false image into the trained feature enhancement network to generate an enhanced generated image;

[0154] In the embodiment, the adversarial perturbation construction module is configured to construct an adversarial perturbation based on the enhanced generated image;

[0155] In the embodiment, the semantic noise generation module is configured to map the adversarial perturbation to a semantic space and perform power normalization to obtain semantic noise;

[0156] In the embodiment, the transmission information generation module is configured to superimpose the noisy semantic information and the semantic noise to obtain superimposed transmission information.

[0157] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application should be equivalent replacement methods, and are all included in the protection scope of the present application.

Claims

1. A method for ensuring secure image semantic communication based on semantic noise, characterized in that, Includes the following steps: The original input image is encoded by a semantic encoder to obtain the original semantic information; Noisy semantic information is obtained, an adversarial generative network is constructed and trained, and the noisy semantic information is input into the trained adversarial generative network to generate fake images. When training the Generative Adversarial Network (GAN), a total loss function is constructed based on self-supervised adversarial loss and mutual information adversarial loss, and the GAN is trained based on the total loss function. The constructed self-supervised adversarial loss is: ； in, Represents the mathematical expectation. Indicates the discriminator, Represents a generator. Represents noisy semantic information. Represents the original semantic information. This represents a pre-trained semantic encoder with its parameters frozen. Indicates standard resistance to loss; Mutual information adversarial loss is represented as: ； in, Indicates the lower bound of mutual information. Indicates hyperparameters, Represents a mutual information estimator; A feature enhancement network is constructed and trained. A fake image is input into the trained feature enhancement network to generate an enhanced generated image. Adversarial perturbations are constructed based on the enhanced generated images, and the adversarial perturbations are mapped to the semantic space and their power is normalized to obtain semantic noise. Adversarial perturbations are constructed based on the enhanced generated images, specifically including: Pseudo-labels are assigned to the enhanced generated images to obtain the training set; Train an alternative classifier based on the training set and calculate adversarial perturbations; The noisy semantic information is mixed with semantic noise to obtain the mixed transmission information.

2. The image semantic communication security protection method based on semantic noise according to claim 1, characterized in that, The adversarial generative network includes a generator and a discriminator; The generator is equipped with multiple cascaded convolutional blocks to expand the one-dimensional vector into a spatialized image tensor, which maps from a low-dimensional joint representation space to a high-dimensional image space and outputs a fake image. The discriminator has multiple linear layers and activation functions to evaluate the consistency between the high-dimensional image space and the original semantic representation.

3. The image semantic communication security protection method based on semantic noise according to claim 1, characterized in that, Constructing and training a feature enhancement network specifically includes: Construct an adjacent sampler to sample two similar but different sub-images from the same fake image, and use the resulting sub-images as the training dataset for the feature enhancement network; The feature enhancement network adopts an encoder-decoder structure. The encoder performs feature transformation through convolution and non-linear activation. The decoder and encoder are spliced ​​together by skip connections. The output features are mapped back to the original image channel space through convolutional layers. The feature enhancement network is trained based on a self-supervised denoising learning paradigm.

4. The image semantic communication security protection method based on semantic noise according to claim 3, characterized in that, When training the feature enhancement network, a composite loss function is constructed based on similarity reconstruction loss and ideal denoising loss, and the feature enhancement network is trained based on the composite loss function.

5. The image semantic communication security protection method based on semantic noise according to claim 1, characterized in that, The alternative classifier is a classification network that uses multiple stacked convolutional layers and ReLU functions as activation layers.

6. The image semantic communication security protection method based on semantic noise according to claim 1, characterized in that, Adversarial perturbations are represented as: ； in, This indicates resistance to disturbances. Indicates the step size of the adversarial example. Represents a symbolic function. Indicates the enhanced generated image gradient operator, Indicates an alternative classifier. Represents the cross-entropy loss function. This indicates a pseudo-tag.

7. A security protection system for image semantic communication based on semantic noise, characterized in that, The system is used to implement the image semantic communication security protection method based on semantic noise according to any one of claims 1-6. The system includes: an original semantic information generation module, a noisy semantic information acquisition module, a fake image generation module, a fake image enhancement module, an anti-disturbance construction module, a semantic noise generation module, and a transmission information generation module. The original semantic information generation module is used to encode the original input image using a semantic encoder to obtain the original semantic information; The noisy semantic information acquisition module is used to acquire noisy semantic information; The fake image generation module is used to input noisy semantic information into the trained adversarial generative network to generate fake images. The fake image enhancement module is used to input fake images into the trained feature enhancement network to generate enhanced generated images. The adversarial perturbation construction module is used to construct adversarial perturbations based on the enhanced generated image; The semantic noise generation module is used to map adversarial perturbations to the semantic space and normalize their power to obtain semantic noise; The transmission information generation module is used to mix noisy semantic information with semantic noise to obtain mixed transmission information.

Images