High-robust image semantic perception adaptive mask transmission method for internet of vehicles

Abstract

This invention relates to a robust image semantic perception adaptive masking transmission method for vehicle-to-everything (V2X) networks, comprising: Step 1: The transmitting end performs semantic analysis on the original input image, extracts high-precision semantic features, and identifies the regions of interest (ROI) and non-ROI regions of interest (NROI); Step 2: A Boolean mask is dynamically generated based on the semantic segmentation result and SNR; Step 3: The ROI features are directly masked based on the Boolean mask; Subsequently, the processed feature map is fed into a semantic encoder to extract multi-scale semantic features; Step 4: Power normalization is performed, dynamic feature encoding is performed, and then the encoded signal is transmitted through a typical analog wireless channel, and noise is superimposed before reception and recovery; Step 5: The receiving end receives the noisy feature signal, performs channel fading compensation and preliminary denoising, and sends the decoded features into a semantic decoder; Through progressive upsampling and multi-level semantic reconstruction, multi-scale features are fused, and finally, a high-quality reconstructed image is recovered and output.

Description

Technical Field

[0001] This invention relates to the field of intelligent semantic communication technology, specifically to a highly robust deep joint source-channel coding transmission method based on image semantic perception masking for bandwidth-constrained vehicle networking environments. Background Technology

[0002] With the deep integration of 5G / 6G communication technology and the Internet of Vehicles (IoV), the amount of data generated by various in-vehicle high-definition cameras, sensors, and autonomous driving systems is exploding, making the demand for efficient and reliable transmission between vehicles (V2V) and between vehicles and roadside infrastructure (V2I) increasingly urgent. In traditional image transmission systems, the Separate Source Channel Coding (SSCC) method is typically used. This separation design is based on Shannon's theorem and can achieve optimal performance under ideal conditions. However, in real-world vehicle-to-everything (V2X) scenarios, this method faces severe challenges: First, "performance collapse" under highly dynamic channels. Due to the high speed of vehicles, signal transmission is more susceptible to obstruction by buildings or other vehicles, leading to severe signal fading. The receiver cannot decode correctly, causing the autonomous driving system to lose critical and reliable perception input, seriously threatening driving safety. Second, there is a disconnect between resource allocation and semantic importance. Traditional compression algorithms distribute limited bandwidth evenly across all image content when processing traffic images. However, in autonomous driving scenarios, the semantic value of different regions of an image varies greatly. Fixed encoding mechanisms cannot dynamically adjust bandwidth resources according to the importance of traffic semantics. As a result, when the channel is congested, core safety information is damaged along with background information, severely affecting subsequent assisted driving decisions. Therefore, designing and developing a robust communication transmission method that can perceive image traffic semantics and adaptively adjust the masking strategy according to channel changes is of significant research and application value for ensuring safe, efficient, and reliable data interaction in V2X.

[0003] In recent years, the rise of deep learning technology has driven the rapid development of the emerging architecture of Deep Joint Source-Channel Coding (Deep JSCC). As a pioneering study in this field, the paper (Eirina Bourtsoulatze, David Burth Kurka, and Deniz G¨und¨uz. Deep joint source-channel coding for wireless image transmission. IEEE Transactions on Cognitive Communications and Networking, 5(3):567–579, 2019.) proposed a classic end-to-end image transmission model (i.e., the DeepJSCC model). This model uses deep neural networks to directly map the image pixel space to the channel symbol space, avoiding the suboptimal nature of traditional separate designs. Existing research shows that this type of Deep JSCC architecture performs well in wireless image transmission, achieving higher peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) under the same bandwidth. For example, the WITT model based on the DeepJSCC architecture improves transmission robustness by introducing an attention mechanism and outperforms traditional methods in complex scenarios such as variable signal-to-noise ratio (SNR). However, these methods still have significant limitations:

[0004] First, there's the issue of feature homogeneity. Existing Deep JSCC architectures treat all regions of an image equally during feature extraction, resulting in extracted features lacking fine-grained differentiation of semantic content. For example, foreground objects (such as vehicles and other key targets) typically carry the core semantic information of an image, requiring higher fidelity and bandwidth resources, while background regions (such as sky, grass, or static textures) are relatively less important, allowing for lower fidelity and resource requirements. However, both are given the same weight during encoding. This homogenization not only leads to more computational processing and transmission bandwidth but also amplifies the noise impact of non-critical regions during transmission, causing distortion of important semantic elements in the reconstructed image.

[0005] Secondly, its resistance to channel fading is insufficient. Under strong fading channels, the features extracted by existing Deep JSCC architectures are easily affected because they typically use pixel-level or low-level representations, lacking protection for deep semantics. This leads to severe signal fading during transmission resulting in low SNR, which in turn distorts semantic feature vectors and amplifies errors in image feature reconstruction, causing semantic distortion in the output image. For example, under low SNR conditions, traditional Deep JSCC architectures cannot effectively recover the structural information of the image, causing visual artifacts or information loss, thus affecting the accuracy of subsequent tasks. In vehicle-to-everything (V2X) and mobile IoT scenarios facing strong fading, this has become a core obstacle restricting the system from achieving efficient and reliable transmission.

[0006] Third, it lacks the adaptive capability to cope with dynamic channel changes. In mobile wireless networks such as vehicular networks where channels change dynamically, the fixed coding strategies used in existing research cannot adjust the feature compression rate in a timely manner according to the actual channel conditions, resulting in wasted transmission resources or deterioration of image reconstruction quality. For example, when channel conditions suddenly deteriorate, the fixed coding strategy may forcibly discard some data, causing irreversible semantic loss; conversely, when the channel improves, it cannot fully utilize the additional bandwidth to improve transmission quality. This fixed coding strategy is difficult to cope with the variable channel environment in actual dynamic vehicular networks, thus limiting the flexibility and efficiency of image transmission. There is an urgent need to design an adaptive coding mechanism to achieve dynamic resource allocation and optimization.

[0007] To address the aforementioned issues, Semantic Communication (SemCom) technology can be introduced into the field of image transmission in vehicle-to-everything (V2X) networks. Semantic Communication emphasizes transmitting the semantic information of images rather than raw pixels, achieving efficient compression and robust transmission by extracting high-level semantic features from images, such as object contours and texture semantics. Existing semantic communication often employs semantic-guided masking strategies, prioritizing bandwidth allocation to regions of interest (ROIs) to improve overall transmission quality. Furthermore, to further optimize system performance in noisy environments, adaptive masking mechanisms can be incorporated into semantic communication systems. However, existing semantic communication frameworks still face the following challenges: first, insufficient semantic perception accuracy, as traditional mask generation relies on fixed thresholds and cannot dynamically respond to image complexity and channel conditions in real time; second, mask design lacks robustness, and under noise interference, the reconstruction of masked regions is prone to artifacts or semantic inconsistencies. These shortcomings make it difficult for existing semantic communication technologies to achieve efficient and robust semantic transmission in mobile networks or IoT scenarios.

[0008] Therefore, it is necessary to develop a novel image semantic transmission method for vehicle-to-everything (V2X) networks, combining semantic awareness and mask adaptive mechanisms to better adapt to dynamic fading and bandwidth-constrained environments. This invention aims to overcome the limitations of existing technologies and provide an efficient, reliable, and robust solution to improve the semantic fidelity and efficiency of image transmission in V2X networks. Summary of the Invention

[0009] The purpose of this invention is to provide a robust image semantic transmission method based on semantic awareness and adaptive masking, so as to solve the problems of feature homogeneity, insufficient anti-fading ability and limitations of fixed coding strategies in the prior art.

[0010] By integrating a deep learning-driven semantic perception module and an adaptive masking mechanism, this invention achieves fine-grained differentiation of image content, robust reconstruction under channel fading, and adaptive optimization of channel states, thereby improving the semantic fidelity and overall efficiency of image transmission in bandwidth-constrained and channel-fading environments. This invention aims to enhance the robustness and semantic fidelity of image data transmission for autonomous driving in connected vehicles under adverse conditions.

[0011] Terminology Explanation: 1. Semantic Communication: This refers to a communication paradigm that emphasizes the semantic information of transmitted data rather than the raw bit stream. It achieves efficient and robust communication by extracting and encoding deep semantic features of images, reducing redundant information and improving transmission quality in bandwidth-constrained environments.

[0012] 2. SSCC: Refers to the traditional split communication system design, which is based on Shannon's theorem and optimizes source coding and channel coding as two independent physical modules.

[0013] 3. Deep JSCC: This refers to the end-to-end joint optimization of source coding and channel coding using deep neural networks. It directly maps the input image data to the channel symbol space, avoiding the suboptimal results caused by traditional separate designs, and making the image more suitable for transmission in wireless fading channel environments.

[0014] 4. Semantic-aware masking: A masking mechanism based on image semantic content generation. It dynamically adjusts the masking probability through a high-density semantic recognition network and a semantic segmentation network, so that semantically high-density regions can obtain a lower masking rate, thereby achieving resource allocation optimization guided by semantic importance.

[0015] 5. Adaptive masking: A mechanism that dynamically adjusts the mask ratio and applies it based on the real-time channel status when the SNR or bandwidth changes, enabling the transmission system to flexibly adapt to the dynamic environment.

[0016] 6. PSNR: This refers to an image quality assessment metric used to quantify the pixel-level differences between a reconstructed image and the original image. The calculation formula is as follows: ,in The maximum possible value of an image pixel. MSE The mean squared error is the average error. A higher PSNR value indicates less image distortion and is often used to evaluate the image fidelity of a transmission system.

[0017] 7. SSIM: This is an image quality assessment index that simulates the human visual system. It comprehensively considers brightness, contrast and structural similarity, and ranges from [0, 1]. The closer the value is to 1, the more similar the reconstructed image is to the original image in terms of human perceptual quality. It is used to evaluate perceptual quality and structural integrity in semantic transmission.

[0018] The technical solution of the present invention is as follows: A robust image semantic-aware adaptive masking transmission method for vehicle-to-everything (V2X) networks is proposed, employing the Deep JSCC framework. The Deep JSCC framework includes a transmitter and a receiver, comprising: Step 1: The sending end performs semantic analysis on the original input image, extracts high-precision semantic features, and automatically identifies the regions of interest and non-regions of interest in the image; Step 2: Combine the semantic segmentation results from Step 1 with the SNR obtained based on the real-time channel state to dynamically generate a Boolean mask; Step 3: Apply the Boolean mask generated in Step 2 to directly clear the masked RONI features; then, feed the processed feature map into the semantic encoder based on the ConvNeXt architecture to extract multi-scale semantic features. Step 4: Normalize the power of the extracted multi-scale semantic features, generate a scaling factor based on the real-time SNR, perform dynamic feature encoding, and then transmit the encoded signal through a typical analog wireless channel, and perform reception recovery after adding noise. Step 5: The receiver receives the noisy feature signal, performs channel fading compensation and preliminary denoising, and sends the decoded features to the semantic decoder based on the ConvNeXt architecture; through progressive upsampling and multi-level semantic reconstruction, multi-scale features are fused, and finally the reconstructed high-quality image is recovered and output.

[0019] According to a preferred embodiment of the present invention, the transmitting end includes a semantic perception module, a mask adaptation module, a semantic feature extraction module, and a channel coding module; the receiving end includes a channel decoding module, a semantic feature recovery module, and an image reconstruction module.

[0020] According to a preferred embodiment of the present invention, the specific implementation process of step 1 includes: The sending end uses a semantic awareness module to define the foreground region with important semantics as the region of interest (ROI) and classify low semantic density regions such as the image background as non-ROI regions. Specifically, this includes: A semantic perception module is used as a semantic segmentation network to preprocess the input image, and then high-precision semantic features are extracted through an encoder-decoder architecture; the semantic perception module includes dilated convolution, dilated spatial pyramid pooling and a decoding unit; Dilated convolutions expand the receptive field of the convolutional kernel by introducing a dilation rate without increasing the number of parameters. Dilated spatial pyramid pooling captures context at different scales through multiple parallel branches. Dilated spatial pyramid pooling includes a 1x1 convolution, three dilated convolutions with different dilation rates, and a global average pooling branch. All outputs are then fused through a concatenation operation. Represented as: ; in This indicates the concatenation operation. It is a 1x1 convolution operation. Indicates the expansion rate i The dilated convolution operation, This indicates a global average pooling operation; Subsequently, the decoding unit generates refined semantic segmentation through 4x upsampling and low-level feature fusion, outputting foreground and background binary codes, i.e., high-precision semantic features, where 0 represents the background region and 1 represents the foreground region; based on this binary code, the image is divided into... N The nth image patch and calculate the nth k Foreground percentage of each patch : ; in This indicates the number of foreground pixels in each patch. and These represent the height and width of the patch, respectively. If the value exceeds the set threshold, the patch is classified as a region of interest; otherwise, it is classified as a region of non-interest.

[0021] According to a preferred embodiment of the present invention, step 2 includes the following specific implementation process: The mask adaptation module dynamically generates an adaptive mask (Boolean mask) based on the semantic segmentation results generated by the semantic awareness module and the real-time SNR; specifically, it includes: First, calculate the mask probability for each patch to achieve semantically guided resource allocation; the formula is: ; in For the first k The mask probability of each patch This is a preset base mask ratio used to control the overall mask strength of the image; The foreground percentage calculated for the semantic awareness module. Given the current signal-to-noise ratio, The preset maximum signal-to-noise ratio threshold; Subsequently, a Boolean mask tensor is generated using random sampling. for: ; ; in Indicates the first k The mask of each patch, random variables This represents sampling from a uniform distribution within the interval (0, 1); when the generated random number is greater than the mask probability, , indicates that the patch is not masked; otherwise, This indicates that the patch is masked.

[0022] According to a preferred embodiment of the present invention, step 3 specifically includes the following steps: The semantic encoder based on the ConvNeXt architecture includes several ConvNeXt blocks, normalization layers, and two-dimensional convolutional layers. A ConvNeXt block includes a depthwise separable convolutional layer, a normalization layer, a first 1×1 convolutional layer, an activation function, a global response normalization layer, and a second 1×1 convolutional layer. In the semantic extraction process, firstly, the input image is block-embedded and projected onto the initial embedding dimension via convolution; then, an adaptive mask is applied to the embedded image, as shown in the formula: ; in The feature map after block embedding. This indicates element-wise multiplication. The Boolean mask tensor generated by the mask adaptation module. It is the input feature map after the mask; Then, the masked input feature map is fed into a semantic encoder based on the ConvNeXt architecture to obtain a new semantic feature vector. : ; in Indicates that the training parameters are The semantic extraction function.

[0023] According to a preferred embodiment of the present invention, step 4 includes the following specific implementation process: The channel coding module outputs the feature vector from the semantic feature extraction module. Power normalization and channel adaptation processing are performed; specifically including: Introduce a scaling factor related to SNR And the SNR and scaling factor are mapped to a high-dimensional channel embedding vector through a multilayer perceptron (MLP); Secondly, the generated high-dimensional channel embedding vector and feature vector are combined. Channel-level splicing is performed, and the data is fed into a multilayer perceptron to generate dynamic coding weights. These dynamic coding weights are then used to... Perform feature recalibration; Finally, the encoded feature vectors are subjected to power normalization. Extracted semantic information Channel coding is performed to obtain the coded information to be transmitted. : ; in Indicates that the training parameters are The channel coding function; In a noisy fading environment, the signal received by the receiver for: ; In the formula For wireless channel matrix, This represents the noise vector.

[0024] According to a preferred embodiment of the present invention, step 5 includes the following specific implementation process: The channel decoding module performs channel fading compensation and preliminary denoising on the noisy fading characteristic signal; firstly, it performs channel gain estimation and power recovery on the received signal, and then obtains the recovered semantic feature vector through channel decoding. Represented as: ; in Indicates that the training parameters are Channel decoding function; The decoded feature signal is then transmitted to a semantic decoder based on the ConvNeXt architecture; In the semantic feature recovery stage, the semantic feature vector output by the channel decoding module is recovered in the semantic decoder. Multi-level semantic reconstruction is performed to gradually restore the semantic structure and detailed information of the image. Upsampling operations progressively increase the spatial resolution while reducing the channel dimension to fuse multi-scale semantic features and generate a high-resolution output. The reconstructed image obtained through this process... Represented as: ; in Indicates that the training parameters are The semantic recovery function.

[0025] The beneficial effects of this invention are as follows: 1. Achieving semantically guided, efficient resource allocation and significantly reducing transmission bandwidth pressure: This invention introduces ASCM for accurate identification of ROIs and RONIs in images, and utilizes a mask adaptive module to perform high-probability masking on non-critical RONIs before transmission, thereby prioritizing channel bandwidth allocation to ROIs carrying core semantics. This mechanism enables the system to maintain high semantic fidelity even in limited bandwidth environments, effectively solving the problem of overall image quality degradation caused by the average allocation of resources across the entire image in traditional communication methods when bandwidth is limited.

[0026] 2. Excellent channel environment adaptability, overcoming the "cliff effect": The dynamic masking mechanism with joint semantic density and real-time signal-to-noise ratio designed in this invention can automatically increase the masking rate to compress information and enhance resistance to channel fading under adverse channel conditions, while decreasing the masking rate to transmit more details under favorable channel conditions. Combined with the SNR-dependent power scaling factor in the channel coding module, the transmission strategy can be adjusted in real time according to the channel state, thereby avoiding the "cliff effect" common in traditional communication and achieving smooth and stable transmission under various channels from AWGN to Rayleigh fading.

[0027] 3. Enhanced robustness of semantic feature extraction and transmission: This invention employs an encoder and decoder based on the ConvNeXt architecture, combined with the ASPP structure. This not only expands the receptive field and avoids resolution loss, but also solves the feature homogeneity problem in other deep networks. In particular, the Deep JSCC adaptive mask design at the transmitting end makes the extracted semantic feature vectors have extremely strong anti-fading capabilities. Even under severe fading, the decoder can still effectively recover the structure and texture of the ROI through the learned prior knowledge, which greatly reduces semantic distortion.

[0028] 4. End-to-end optimization reduces system complexity and latency: This invention abandons the independent source coding, channel coding, and equalizer modules in traditional communication systems, and instead employs an end-to-end deep neural network for joint training and optimization. Semantic extraction and encoding at the transmitter and decoding and recovery at the receiver are tightly coupled, simplifying the system architecture and reducing processing latency in intermediate stages. Simultaneously, the multi-SNR training mode ensures that the model only needs to be trained once to adapt to dynamically changing wireless environments, greatly improving system reliability and robustness. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the overall architecture of a highly robust image semantic perception adaptive mask transmission method for vehicle networking proposed in this invention.

[0030] Figure 2 This is a schematic diagram illustrating the specific structure of the semantic feature extraction module and the internal composition of the ConvNeXt module in this invention.

[0031] Figure 3 This is a schematic diagram comparing the performance of the method of this invention with the WITT model and traditional separation methods in an AWGN channel.

[0032] Figure 4 This is a schematic diagram comparing the performance of the method of this invention with the WITT model and traditional separation methods in the Rayleigh channel. Detailed Implementation

[0033] The present invention will be further defined below with reference to the accompanying drawings and embodiments, but is not limited thereto.

[0034] Example 1 A robust image semantic-aware adaptive masking transmission method for vehicle-to-everything (V2X) networks is proposed, employing the Deep JSCC framework. The Deep JSCC framework includes a transmitter and a receiver, comprising: Step 1: The sending end performs semantic analysis on the original input image, extracts high-precision semantic features, and automatically identifies the regions of interest (ROI) and non-ROI of the image; providing semantic guidance for subsequent resource allocation.

[0035] Step 2: Combine the semantic segmentation results from Step 1 with the SNR obtained based on the real-time channel state to dynamically generate a Boolean mask. This step assigns a low mask probability to the ROI in the image and adaptively adjusts the mask probability of the ROI based on the SNR and the proportion of the foreground in the image to achieve semantically guided resource allocation.

[0036] Step 3: Apply the Boolean mask generated in Step 2 to directly clear the masked RONI features; then, feed the processed feature map into the semantic encoder based on the ConvNeXt architecture to efficiently extract multi-scale semantic features. Step 4: Normalize the power of the extracted multi-scale semantic features, generate a scaling factor based on the real-time SNR, perform dynamic feature encoding, and then transmit the encoded signal through a typical analog wireless channel, and perform reception recovery after adding noise. Step 5: The receiver receives the noisy feature signal, performs channel fading compensation and preliminary denoising, and sends the decoded features to the semantic decoder based on the ConvNeXt architecture; through progressive upsampling and multi-level semantic reconstruction, multi-scale features are fused, and finally the reconstructed high-quality image is recovered and output.

[0037] Example 2 The difference between the robust image semantic perception adaptive mask transmission method for vehicle networking described in Example 1 and the one described in Example 1 is that: like Figure 1 As shown, this embodiment proposes a robust image semantic-aware adaptive mask transmission method for vehicle-to-everything (V2X) networks. It mainly consists of three parts: a transmitter, a wireless channel environment, and a receiver. This embodiment uses an RGB image with a transmission resolution of H×W as an example for illustration.

[0038] Step 1: Semantic Perception and Region Segmentation. After receiving the original image X, the sending end introduces a semantic perception module based on a dilated convolutional network to distinguish key information from non-key background in the image. This module adopts an encoder-decoder architecture. First, it extracts features from the input image using a dilated convolutional backbone network with a dilation rate r, effectively expanding the receptive field without increasing the number of parameters. Then, it uses a dilated spatial pyramid pooling (ASPP) module to capture multi-scale contextual information. ASPP concatenates and fuses the outputs of a 1×1 convolution, three 3×3 dilated convolutions with different dilation rates, and a global average pooling branch. The fused features are upsampled by the decoder to generate a binary segmentation mask with foreground pixels of 1 and background pixels of 0. Based on this, the system divides the input image into several patches and calculates the foreground pixel proportion of each patch according to a formula. Finally, a threshold τ is set, and... Patches with a value greater than τ are identified as ROI regions, while the remaining patches are classified as RONI regions, thus providing a semantic basis for subsequent resource allocation.

[0039] Step 2: Adaptive Mask Generation. To maximize transmission efficiency with limited bandwidth, the adaptive mask generation module generates a mask based on the foreground proportion. The system dynamically generates a mask tensor M based on real-time channel state information. Specifically, the system first sets a base mask ratio and a maximum signal-to-noise ratio threshold to calculate the mask probability for each patch. This mechanism ensures that more information is preferentially retained in the ROI region and under poor channel conditions, while the mask ratio is appropriately increased in the ROI region or under good channel conditions to save bandwidth. Subsequently, the system generates random numbers in the interval [0, 1] for each patch. When the generated random number is less than the mask probability, the mask value is set to 1 to indicate occlusion; otherwise, it is set to 0 to indicate retention. The final generated mask tensor M is then smoothed using Softmax and applied to subsequent feature selection.

[0040] Step 3: Semantic feature extraction. For example... Figure 2 As shown, this module uses the ConvNeXt V2 architecture as the encoder. First, it performs patch embedding on the original image X, segmenting and projecting it into an embedded feature map. Then, the eigenvalues ​​of the RONI region are set to zero according to the adaptive masking tensor M, significantly reducing computational data redundancy while preserving key information. Next, the masked feature map... The input is fed into stacked ConvNeXt Blocks for deep feature encoding. Each Block contains, sequentially, 7×7 depthwise separable convolutions for spatial feature extraction, layer normalization, 1×1 convolutions to increase the dimensionality to 4 times the channel width, and a GELU activation function. Notably, the module introduces a Global Response Normalization (GRN) layer as a key step. By calculating the global L2 norm of the feature channels, it effectively prevents feature collapse caused by a large number of masked inputs and enhances the response amplitude of effective features. Finally, the dimensionality is reduced back to the original channel width by 1×1 convolutions, outputting a high-dimensional semantic feature vector. .

[0041] Step 4: Channel Modulation and Transmission. Extracted Semantic Features The signal is fed into the channel coding module (ChannelEncoder). First, the system calculates a scaling factor based on the real-time signal-to-noise ratio (SNR) to dynamically adjust the amplitude of the transmitted signal. Then, the semantic information undergoes power normalization and is mapped using Deep JSCC nonlinearity to generate the symbols to be transmitted. Finally, the signal When transmitted through a wireless channel, it becomes susceptible to noise interference. This embodiment supports simulating AWGN channels and Rayleigh fading channels during the training phase, thereby giving the model adaptability to various channel environments.

[0042] Step 5: Channel Decoding and Semantic Recovery. The receiver, upon receiving the signal... Then the reverse recovery process is initiated. First, the channel decoder uses a neural network. Equalization and decoding are performed on the received signal, and the recovered semantic features are output while implicitly performing channel estimation and denoising. Subsequently, this feature is input into a semantic feature recovery module (Semantic Decoder) that is symmetrical to the encoder structure at the transmitting end. This module consists of multiple inverse ConvNeXt Blocks and upsampling layers, and is processed through an inverse mapping function. By combining prior image knowledge learned during training, the system effectively infers and completes the masked background region from incomplete semantic features, while meticulously reconstructing the texture details of the ROI region, ultimately outputting a high-quality reconstructed image. .

[0043] Example 3 The difference between the robust image semantic perception adaptive mask transmission method for vehicle networking described in Example 1 and the one described in Example 1 is that: The transmitting end includes a semantic awareness module (ASCM), a mask adaptation module, a semantic feature extraction module, and a channel coding module; the receiving end includes a channel decoding module, a semantic feature recovery module, and an image reconstruction module.

[0044] The specific implementation process of step 1 includes: The sending end uses a semantic awareness module to define the foreground region with important semantics as the region of interest (ROI) and classify low semantic density regions such as the image background as non-ROI regions. Specifically, this includes: A semantic awareness module is used as a semantic segmentation network to preprocess the input image, and then high-precision semantic features are extracted through an encoder-decoder architecture. The semantic awareness module includes atrous convolution, atrous spatial pyramid pooling (ASPP), and a decoding unit. Dilated convolutions expand the receptive field of the convolutional kernel by introducing a dilation rate without increasing the number of parameters; this allows the network to capture a wider range of contextual information without downsampling, thus avoiding resolution loss. Dilated spatial pyramid pooling captures context at different scales through multiple parallel branches. Dilated spatial pyramid pooling includes a 1x1 convolution, three dilated convolutions with different dilation rates, and a global average pooling branch. All outputs are then fused through a concatenation operation. Represented as:

[0045] ; in This indicates the concatenation operation. It is a 1x1 convolution operation. Indicates the expansion rate i The dilated convolution operation, This indicates a global average pooling operation; Subsequently, the decoding unit generates refined semantic segmentation through 4x upsampling and low-level feature fusion, outputting foreground and background binary codes, i.e., high-precision semantic features. The encoding unit outputs low-resolution, high-dimensional features, while the decoding unit increases resolution and reduces dimensionality. Increasing resolution is achieved through 4x upsampling using linear interpolation. Reducing dimensionality requires feature fusion using 1x1 convolution. Finally, the output binary code is mapped to a range of 0-1 using the sigmoid function. 0 represents the background region, and 1 represents the foreground region. Based on this binary code, the image is segmented into... N The nth patch and calculate the nth k Foreground percentage of each patch :

[0046] ; in This indicates the number of foreground pixels in each patch. and These represent the height and width of the patch, respectively. If the value exceeds a set threshold (an empirical value, typically 0.3), the patch is classified as a region of interest; otherwise, it is classified as a region of non-interest. This provides semantic guidance for subsequent mask generation.

[0047] The specific implementation process of step 2 includes: The mask adaptation module dynamically generates an adaptive mask (Boolean mask) based on the semantic segmentation results generated by the semantic awareness module and the real-time SNR; specifically, it includes: First, the mask probability of each patch is calculated to achieve semantically guided resource allocation. For regions of interest, a lower mask probability is set to prioritize the protection of key semantic information. For regions of non-interest, the mask probability is adaptively adjusted based on the base mask ratio, foreground proportion, and real-time SNR. The formula is: ; in For the first k The mask probability of each patch This is a preset base mask ratio used to control the overall mask strength of the image; The foreground ratio is calculated for the semantic awareness module. A higher value indicates higher semantic density and lower mask probability. Given the current signal-to-noise ratio, To preset the maximum signal-to-noise ratio threshold, when near When, the mask probability is reduced to make full use of a good channel; when When the mask probability is low, the mask probability is increased to reduce the impact of non-critical information and improve robustness.

[0048] Subsequently, a Boolean mask tensor is generated using random sampling. for: ; ; in Indicates the first k The mask of each patch, random variables This represents sampling from a uniform distribution within the interval (0, 1); when the generated random number is greater than the mask probability, , indicates that the patch is not masked; otherwise, This indicates that the patch is masked. This module also supports multi-SNR training modes, randomly sampling SNR values ​​during training and dynamically updating them. This enables adaptive optimization to change the channel environment. Through this adaptive mechanism, the system can increase the mask rate to above 0.7 under low SNR to reduce transmission load, and reduce it to below 0.3 under high SNR to improve detail fidelity.

[0049] The specific implementation process of step 3 includes: Semantic feature extraction, performed in the Deep JSCC transmitter encoder, involves multi-level semantic feature extraction from the input image and applying adaptive masks to prioritize the preservation of information from the Regions of Interest (ROIs). Specifically, this module employs a deep neural network based on the ConvNeXt architecture as the encoder to achieve efficient semantic feature extraction learning. The core components of ConvNeXt include depthwise separable convolution, layer normalization, and global response normalization, thereby progressively extracting multi-scale features from low-level texture to high-level semantics. Each ConvNeXt layer includes downsampling to progressively reduce the image spatial resolution while increasing the channel dimension, thus better capturing the global context.

[0050] The semantic encoder based on the ConvNeXt architecture includes several ConvNeXt blocks, normalization layers, and two-dimensional convolutional downsampling layers. A ConvNeXt block includes a depthwise separable convolutional layer, a normalization layer, a first 1×1 convolutional layer, an activation function, a global response normalization layer, and a second 1×1 convolutional layer. Extraction process: In the shallow extraction stage, due to the high resolution of the feature map and the small number of downsampling times, the network captures local details such as color, edges, texture and simple geometry.

[0051] Expanded receptive field: The use of 7×7 large convolutional kernels in each layer, combined with the increase in depth, allows each neuron to "see" an increasingly larger area in the original image.

[0052] As the network deepens, the feature maps have undergone multiple downsampling processes. At this point, a single pixel on the feature map represents a large region in the original image. The extracted features are no longer the color of the pixel, but rather higher-level semantics such as "there is a target here" or "this is the background."

[0053] Multiscale semantics is mathematically a set of high-dimensional tensors.

[0054] The low-scale semantics of shallow output are manifested as strong edge responses. This includes the skeleton and contour map of the image, as well as the boundary texture of objects.

[0055] Mid-level feature maps: mostly contain information about local parts of an object; Deep, high-level feature maps: These are highly abstract representations that typically depict the type of target, complex contextual relationships, etc. In the semantic extraction process, firstly, the input image is block-embedded and projected onto the initial embedding dimension via convolution; then, an adaptive mask is applied to the embedded image, as shown in the formula: ; in The feature map after block embedding. This indicates element-wise multiplication. This is a Boolean mask tensor generated by the mask adaptation module; specifically, 1 represents the masked region, and 0 represents the region to be preserved. This is the input feature map after masking; this operation can remove ROI features, thereby prioritizing the processing of ROI features, reducing interference from non-critical semantic features, and thus improving semantic fidelity.

[0056] Then, the masked input feature map is fed into a semantic encoder based on the ConvNeXt architecture to obtain a new semantic feature vector. : ; in Indicates that the training parameters are The semantic extraction function described above effectively solves the problem of semantic feature homogeneity. The extracted semantic feature vector is then processed... Transmitted to the channel coding module.

[0057] The specific implementation process of step 4 includes: The channel coding module outputs the feature vector from the semantic feature extraction module. Power normalization and channel adaptation processing are performed; specifically including: To enable the network to accurately perceive channel conditions, a scaling factor related to SNR is introduced. Furthermore, the SNR and scaling factor are mapped to a high-dimensional channel embedding vector through a multilayer perceptron (MLP). The multilayer perceptron is a simple neural network composed of several fully connected layers. Its input is the value of SNR and scaling factor, and its output is a high-dimensional vector. The dimension of this vector is the same as the dimension of the semantic feature vector output by the semantic extraction network, so concatenation can be performed.

[0058] Secondly, the generated high-dimensional channel embedding vector and feature vector are combined. Channel-level splicing is performed, and the data is fed into a multilayer perceptron to generate dynamic coding weights. These dynamic coding weights are then used to... Feature relabeling is then performed; similarly, it's a simple fully connected neural network where the input is the concatenated vector and the output is the weights of each dimension of the vector. Then, the weights are... The physical meaning of this mechanism is that under low SNR, the system will reduce the weight of minor feature vectors, thereby automatically suppressing minor features and minimizing the impact of bit errors; while under high SNR, the system will amplify the weight of detailed features to make full use of the channel bandwidth.

[0059] Finally, the encoded feature vector is normalized to ensure that the energy distribution of the encoded signal is reasonable, so as to output a highly robust continuous analog symbol to the wireless channel for transmission. Extracted semantic information Channel coding is performed to obtain the coded information to be transmitted. : ; in Indicates that the training parameters are The channel coding function ensures that the feature vectors are better adapted to the channel before transmission. Through an end-to-end Deep JSCC design, this module avoids the suboptimal nature of traditional split coding, improves the overall reliability and robustness of transmission, and can control the compression ratio to below 0.5 bit / pixel in a limited bandwidth environment while maintaining high image quality.

[0060] In vehicle-to-everything (V2X) wireless transmission, two typical channel models are employed to evaluate and optimize the robustness and reliability of the transmission process in order to simulate real-world wireless channels. Specifically, this invention supports multiple typical wireless channel models, including additive white Gaussian noise (AWGN) channels and Rayleigh fading channels, to cover transmission interference from static to dynamic. The purpose of channel simulation is to inject real-world noise interference into end-to-end training, ensuring that the system can adapt to bandwidth-constrained and channel fading conditions, and prioritizing the protection of the semantic information of the Region of Interest (ROI). In noisy fading environments, the signal received by the receiver... for:

[0061] ; In the formula For wireless channel matrix, This represents the noise vector. The noisy fading characteristic signal is then transmitted to the receiver's channel decoding module for subsequent recovery and reconstruction.

[0062] The specific implementation process of step 5 includes: The channel decoding module performs channel fading compensation and preliminary denoising on the noisy fading characteristic signal to extract reliable received features. First, channel gain estimation and power recovery are performed on the received signal to ensure that subsequent modules receive normalized input. The recovered semantic feature vector is obtained through channel decoding. Represented as:

[0063] ; in Indicates that the training parameters are Channel decoding function; The decoded feature signals are then transmitted to a semantic decoder based on the ConvNeXt architecture; it is a symmetric inverse ConvNeXt block structure, similar to the semantic encoder above. It is responsible for restoring the multi-level semantic features extracted by the encoder into a human-recognizable image structure. Thanks to end-to-end joint training, the network can reconstruct damaged semantic features based on prior knowledge of image semantics, thereby ensuring reliability and robustness under low SNR and improving the semantic transmission quality in bandwidth-constrained and channel-fading environments.

[0064] In the semantic feature recovery stage, the semantic feature vector output by the channel decoding module is recovered in the semantic decoder. Multi-level semantic reconstruction is performed to gradually restore the semantic structure and detailed information of the image. This module uses a ConvNeXt architecture, symmetrical to the semantic encoding at the sending end, as the decoder to achieve efficient feature recovery and upsampling operations. The upsampling operation gradually increases the spatial resolution while reducing the channel dimension to fuse multi-scale semantic features and generate a high-resolution output. The reconstructed image obtained from this process... Represented as: ; in Indicates that the training parameters are The semantic recovery function. Since the network can make full use of the global context dependency brought about by multi-scale fusion, this recovery process can effectively solve the semantic distortion problem under noise interference.

[0065] The semantic feature recovery module and the image reconstruction module together form the semantic decoder. The image reconstruction module follows the semantic feature recovery module.

[0066] Semantic recovery module: Extracts pure semantics from noisy fading feature signals. Performs semantic inference on masked regions. And gradually restores the features to a high-resolution structure through upsampling.

[0067] Image reconstruction module: mainly restores the semantic features output by the semantic feature recovery module into an RGB three-dimensional image.

[0068] To verify the effectiveness of the present invention, simulation experiments were conducted on a test set and compared with the existing advanced method WITT.

[0069] Figure 3 and Figure 4 In the graph, the horizontal axis represents the signal-to-noise ratio (SNR). The vertical axes, PSNR and MSSSIM, are evaluation metrics; higher values ​​are better. Figure 3 The comparison is within the AWGN channel. Figure 4 The comparison is within the Rayleigh channel. The solid red line represents the model proposed in this invention, while the other two are comparative models; the dashed blue line represents the WITT model, a semantic communication model based on the swin transformer; the dashed green line represents the traditional split communication model.

[0070] This embodiment employs an end-to-end joint training scheme based on the standard image dataset DIV2k. For the optimization objective, a weighted combination of mean squared error loss and perceptual loss is used as the total loss function, as shown in the formula:

[0071] ; This approach balances pixel-level accuracy in image reconstruction with naturalness in visual perception. Specifically, a multi-SNR training mechanism is introduced during training, where each training batch randomly samples SNR values ​​from the range [-10, 10] dB. This forces the model to learn to dynamically adjust the masking strategy and feature extraction weights under different signal-to-noise ratios, thereby achieving the generalization capability of "adapting to all channel environments with a single training iteration."

[0072] AWGN channel performance verification. For example... Figure 3As shown, in additive white Gaussian noise channels, this invention demonstrates a significant advantage in low signal-to-noise ratio (SNR) environments (less than 4 dB) for PSNR, especially in extreme environments of -10 dB, where the PSNR leads the WITT model by approximately 2 dB. This proves that the mask-based semantic protection mechanism greatly enhances transmission robustness. However, in high SNR (>6 dB) stages, the PSNR performance of this invention is slightly lower than that of WITT. This is because the system ensures the transmission of core semantics under limited bandwidth by employing a mask-dropping strategy for non-interest regions. Consequently, the reconstructed image is slightly weaker in pixel-level alignment in non-critical background areas compared to the WITT model, which transmits all features. This is a necessary resource trade-off for achieving high robustness and low bandwidth consumption. Regarding the comparison of MS-SSIM parameters, since the loss function of the proposed model considers both pixel-level accuracy and the naturalness of visual perception, it can learn more image perception knowledge. Therefore, in channels with SNR less than 8 dB, the MS-SSIM performance of the proposed model significantly outperforms the WITT model. This indicates that the image transmitted by the proposed model is more in line with human perception habits.

[0073] Rayleigh channel performance verification. For example... Figure 4 As shown, in Rayleigh fading channels where signal fluctuations are more severe, this invention achieves full-band superiority over the WITT model in both PSNR and MS-SSIM, from -10dB to 10dB. Thanks to the synergistic effect of the mask adaptive module and the channel modulation mechanism, the system effectively resists deep interference caused by multipath fading. Under the harsh condition of -10dB, the PSNR advantage expands to nearly 2.8dB, and the MS-SSIM curve consistently maintains a significant lead. This fully demonstrates the strong robustness of this invention in complex dynamic environments and effectively avoids the performance collapse phenomenon commonly seen in existing technologies in fading channels.

Claims

1. A robust image semantic-aware adaptive mask transmission method for vehicle-to-everything (V2X) networks, characterized in that, The Deep JSCC framework is used. The Deep JSCC framework includes a sender and a receiver, including: Step 1: The sending end performs semantic analysis on the original input image, extracts high-precision semantic features, and automatically identifies the regions of interest and non-regions of interest in the image; Step 2: Combine the semantic segmentation results from Step 1 with the SNR obtained based on the real-time channel state to dynamically generate a Boolean mask; Step 3: Apply the Boolean mask generated in Step 2 to directly clear the masked RONI features; then, feed the processed feature map into the semantic encoder based on the ConvNeXt architecture to extract multi-scale semantic features. Step 4: Normalize the power of the extracted multi-scale semantic features, generate a scaling factor based on the real-time SNR, perform dynamic feature encoding, and then transmit the encoded signal through a typical analog wireless channel, and perform reception recovery after adding noise. Step 5: The receiver receives the noisy feature signal, performs channel fading compensation and preliminary denoising, and sends the decoded features to the semantic decoder based on the ConvNeXt architecture; through progressive upsampling and multi-level semantic reconstruction, multi-scale features are fused, and finally the reconstructed high-quality image is recovered and output.

2. The robust image semantic-aware adaptive mask transmission method for vehicle-to-everything (V2X) networks according to claim 1, characterized in that, The transmitting end includes a semantic awareness module, a mask adaptation module, a semantic feature extraction module, and a channel coding module; the receiving end includes a channel decoding module, a semantic feature recovery module, and an image reconstruction module.

3. The highly robust image semantic-aware adaptive mask transmission method for vehicle-to-everything (V2X) networks according to claim 2, characterized in that, The specific implementation process of step 1 includes: The sending end uses a semantic awareness module to define the foreground region with important semantics as the region of interest (ROI) and classify low semantic density regions such as the image background as non-ROI regions. Specifically, this includes: A semantic perception module is used as a semantic segmentation network to preprocess the input image, and then high-precision semantic features are extracted through an encoder-decoder architecture; the semantic perception module includes dilated convolution, dilated spatial pyramid pooling and a decoding unit; Dilated convolutions expand the receptive field of the convolutional kernel by introducing a dilation rate without increasing the number of parameters. Dilated spatial pyramid pooling captures context at different scales through multiple parallel branches. Dilated spatial pyramid pooling includes a 1x1 convolution, three dilated convolutions with different dilation rates, and a global average pooling branch. All outputs are then fused through a concatenation operation. Represented as: ； in This indicates the concatenation operation. It is a 1x1 convolution operation. Indicates the expansion rate i The dilated convolution operation, This indicates a global average pooling operation; Subsequently, the decoding unit generates refined semantic segmentation through 4x upsampling and low-level feature fusion, outputting foreground and background binary codes, i.e., high-precision semantic features, where 0 represents the background region and 1 represents the foreground region; based on this binary code, the image is divided into... N The nth patch and calculate the nth k Foreground percentage of each patch : ； in This indicates the number of foreground pixels in each patch. and These represent the height and width of the patch, respectively. If the value exceeds the set threshold, the patch is classified as a region of interest; otherwise, it is classified as a region of non-interest.

4. The highly robust image semantic-aware adaptive mask transmission method for vehicle networking according to claim 2, characterized in that, The specific implementation process of step 2 includes: The mask adaptation module dynamically generates an adaptive mask (Boolean mask) based on the semantic segmentation results generated by the semantic awareness module and the real-time SNR; specifically, it includes: First, calculate the mask probability for each patch to achieve semantically guided resource allocation; the formula is: ； in For the first k The mask probability of each patch This is a preset base mask ratio used to control the overall mask strength of the image; The foreground percentage calculated for the semantic awareness module. Given the current signal-to-noise ratio, The preset maximum signal-to-noise ratio threshold; Subsequently, a Boolean mask tensor is generated using random sampling. for: ； ； in Indicates the first k The mask of each patch, random variables This represents sampling from a uniform distribution within the interval (0, 1); when the generated random number is greater than the mask probability, , indicates that the patch is not masked; otherwise, This indicates that the patch is masked.

5. A robust image semantic-aware adaptive mask transmission method for vehicle-to-everything (V2X) networks according to claim 2, characterized in that, The specific implementation process of step 3 includes: The semantic encoder based on the ConvNeXt architecture includes several ConvNeXt blocks, normalization layers, and two-dimensional convolutional layers. A ConvNeXt block includes a depthwise separable convolutional layer, a normalization layer, a first 1×1 convolutional layer, an activation function, a global response normalization layer, and a second 1×1 convolutional layer. In the semantic extraction process, firstly, the input image is block-embedded and projected onto the initial embedding dimension via convolution; then, an adaptive mask is applied to the embedded image, as shown in the formula: ； in The feature map after block embedding. This indicates element-wise multiplication. The Boolean mask tensor generated by the mask adaptation module. It is the input feature map after the mask; Then, the masked input feature map is fed into a semantic encoder based on the ConvNeXt architecture to obtain a new semantic feature vector. : ； in Indicates that the training parameters are The semantic extraction function.

6. A robust image semantic-aware adaptive mask transmission method for vehicle-to-everything (V2X) networks according to claim 2, characterized in that, The specific implementation process of step 4 includes: The channel coding module outputs the feature vector from the semantic feature extraction module. Power normalization and channel adaptation processing are performed; specifically including: Introduce a scaling factor related to SNR And the SNR and scaling factor are mapped to a high-dimensional channel embedding vector through a multilayer perceptron (MLP); Secondly, the generated high-dimensional channel embedding vector and feature vector are combined. Channel-level splicing is performed, and the data is fed into a multilayer perceptron to generate dynamic coding weights. These dynamic coding weights are then used to... Perform feature recalibration; Finally, the encoded feature vectors are subjected to power normalization. Extracted semantic information Channel coding is performed to obtain the coded information to be transmitted. : ； in Indicates that the training parameters are The channel coding function; In a noisy fading environment, the signal received by the receiver for: ； In the formula For wireless channel matrix, This represents the noise vector.

7. A robust image semantic-aware adaptive mask transmission method for vehicle-to-everything (V2X) networks according to any one of claims 2-6, characterized in that, The specific implementation process of step 5 includes: The channel decoding module performs channel fading compensation and preliminary denoising on the noisy fading characteristic signal; firstly, it performs channel gain estimation and power recovery on the received signal, and then obtains the recovered semantic feature vector through channel decoding. Represented as: ； in Indicates that the training parameters are Channel decoding function; The decoded feature signal is then transmitted to a semantic decoder based on the ConvNeXt architecture; In the semantic feature recovery stage, the semantic feature vector output by the channel decoding module is recovered in the semantic decoder. Multi-level semantic reconstruction is performed to gradually restore the semantic structure and detailed information of the image. Upsampling operations progressively increase the spatial resolution while reducing the channel dimension to fuse multi-scale semantic features and generate a high-resolution output. The reconstructed image obtained through this process... Represented as: ； in Indicates that the training parameters are The semantic recovery function.

Images