Semantic communication method and system for high-resolution image

Abstract

The invention provides a semantic communication method and system for a high-resolution image, and relates to the technical field of semantic communication, and the method comprises the steps: obtaining a semantic feature and super-prior of a to-be-coded image; segmenting the semantic features into four groups of feature elements; based on the four groups of feature elements, the hyper-priori and the channel state, four-step airspace priori and semantic importance are obtained; based on the feature element combination corresponding to the ith estimation index, the ith semantic importance, the semantic feature coding result of each path before the ith path and the channel state, utilizing a semantic feature codec to obtain received semantic features; and based on the received semantic features and channel states, utilizing a semantic information restorer and a denoising enhancement module to reconstruct an image. The semantic features of the image are extracted and recovered by adopting the constructed module, and the semantic features of the image are processed by utilizing a coder-decoder, so that the coding and decoding efficiency is improved, the understanding ability of the model to the image content is enhanced, and the high-quality image content can be better reconstructed.

Description

Technical Field

[0001] The present application relates to the field of semantic communication technology, and in particular to a semantic communication method and system for high-resolution images. Background Art

[0002] The amount of data required to be transmitted for high-resolution image-related services is extremely large, which will bring huge communication overhead. It is necessary to implement a compact coding and transmission scheme to reduce the use of channel bandwidth.

[0003] Traditional image communication systems with a separate design have the following problems when transmitting high-resolution images: when the channel conditions are poor or the bandwidth is limited, once there is a mismatch between the communication transmission code rate and the channel capacity, it will lead to a cliff effect, that is, the communication performance will drop sharply when the channel capacity is lower than the communication transmission code rate, and the receiving end is often unable to recover the image data; the system is only an independent optimization at the symbol level. When the channel environment is poor or the communication bandwidth is limited, the accuracy at the symbol level is difficult to effectively guarantee the accuracy at the source semantic level.

[0004] Based on this, a new semantic communication method for high-resolution image transmission is urgently needed. Summary of the invention

[0005] The present application provides a semantic communication method and system for high-resolution images to solve the above problems.

[0006] In a first aspect of the present application, a semantic communication method for high-resolution images is provided, the method comprising: The image to be encoded is nonlinearly transformed from the pixel domain to the feature domain to obtain semantic features, wherein the image to be encoded is a video I frame; Performing a hyper-prior distribution estimation on the semantic feature to obtain a hyper-prior; Evenly dividing the semantic features into four groups along the channel dimension to obtain four groups of feature elements; Based on the four groups of characteristic elements and the super prior, a quadtree-based spatial entropy model is used to perform step-by-step prior estimation to obtain a four-step spatial prior; Based on the channel state, the four groups of feature elements and the four-step spatial domain prior, four-step semantic importance is estimated; For the i-th path among the four paths, the feature element combination corresponding to the i-th step estimation index, the i-th step semantic importance, the semantic feature encoding results of the paths before the i-th path, and the channel state are input into the i-th path semantic feature encoder, and the i-th path semantic feature encoding result is output, wherein the semantic feature encoder is a four-path semantic feature encoder, corresponding to the four-step prior estimation of the spatial entropy model based on the quadtree, and the value range of i is 0-3; The semantic feature encoding result of each channel is mapped into symbols of different lengths for wireless transmission through the corresponding symbol mapping modules of each channel; The symbol mapping modules of each channel at the receiving end restore the symbols according to their dimensions; For the i-th channel among the four channels, the channel state, the received i-th channel semantic feature encoding result, and the semantic feature decoding results of each channel before the i-th channel are input into the i-th channel semantic feature decoder to obtain the i-th channel semantic feature decoding result, the semantic feature decoder is a four-channel semantic feature decoder, and the four-channel semantic feature decoding results constitute the received semantic feature; Inputting the received semantic features and the channel state into a semantic information restorer, wherein the semantic information restorer is used to restore the received semantic features into noisy features through upsampling; The noisy features and the channel status are input into a denoising and enhancement module to reconstruct an image.

[0007] In an optional embodiment of the present application, a hyper-prior distribution estimation is performed on the semantic feature to obtain a hyper-prior, including: The semantic feature is input into a super-prior estimation module to obtain the super-prior. The super-prior estimation module is only deployed at the transmitting end, and the transmitting end will not transmit the super-prior.

[0008] In an optional embodiment of the present application, when a quadtree-based spatial entropy model is used for step-by-step prior estimation, each group of characteristic elements in each step of the step-by-step prior estimation is different; The prior used by the feature element of step 0 is the initial fusion prior, which is generated based on the super prior; The feature elements in step 1 will use the priors estimated based on the feature elements in step 0 and the initial fusion priors; The feature elements in step 2 will use the priors estimated based on the feature elements in step 0, the feature elements in step 1, and the initial fusion priors; The feature elements in step 3 will use the priors estimated based on the feature elements in step 0, the feature elements in step 1, the feature elements in step 2, and the initial fusion priors; The prior used in each step of the characteristic element is used to estimate the symbol length corresponding to the encoding of the characteristic element of this step.

[0009] In an optional embodiment of the present application, based on the four groups of characteristic elements and the super prior, a quadtree-based spatial entropy model is used to perform step-by-step prior estimation to obtain a four-step spatial prior, including: Generate an initial fusion prior according to the super prior and use it as the spatial domain estimation result of the spatial domain estimation in step 0; In the process of spatial domain estimation from step 1 to step 3, the estimated distribution features obtained in the previous step and the initial fusion prior are used as references, and the estimation is completed using a network with shared weights to obtain the spatial domain prior for each step.

[0010] In an optional embodiment of the present application, each semantic feature encoder includes: a channel perception module and a semantic feature encoding module, and the semantic feature encoding module includes: a first cross attention layer and a second cross attention layer; The feature element combination corresponding to the estimated index of the i-th step, the semantic importance of the i-th step, the semantic feature encoding results of each path before the i-th path, and the channel state are input into the i-th semantic feature encoder, and the i-th semantic feature encoding result is output, including: Inputting the channel state and the characteristic element combination corresponding to the i-th step estimation index into the channel sensing module to obtain a first channel sensing result; Inputting the first channel perception result and the i-th step semantic importance into the first cross attention layer to obtain a first cross attention result; Based on the first cross-attention result and the semantic feature encoding results of each path before the i-th path, the second cross-attention layer is used to obtain the i-th semantic feature encoding result.

[0011] In an optional embodiment of the present application, the channel perception module includes a spatial attention module and a channel attention module; Inputting the channel state and the characteristic element combination corresponding to the i-th step estimation index into the channel sensing module to obtain a first channel sensing result, including: Input the channel state and the characteristic element combination corresponding to the i-th step estimation index into the channel attention module to obtain a channel attention result; Based on the feature element combination corresponding to the estimated index in the i-th step and the channel attention result, the spatial attention result is obtained by using the spatial attention module; Based on the combination of feature elements corresponding to the i-th step estimation index and the spatial attention result, the first channel perception result is obtained.

[0012] In an optional embodiment of the present application, each semantic feature decoder includes: a channel perception module and a semantic feature decoding module, and the semantic feature decoding module includes: a third cross attention layer; Inputting the channel state, the received i-th semantic feature encoding result, and the semantic feature decoding results of each channel before the i-th semantic feature decoder to obtain the i-th semantic feature decoding result, including: Based on the received i-th semantic feature encoding result and the semantic feature decoding results before the i-th path, using the third cross attention layer, a second cross attention result is obtained; Based on the channel state and the second cross-attention result, the channel perception module is utilized to obtain the i-th semantic feature decoding result.

[0013] In an optional embodiment of the present application, the semantic information restorer includes: a first semantic information recovery module, a channel perception module, and a second semantic information recovery module; The step of inputting the received semantic features and the channel state into a semantic information restorer, wherein the semantic information restorer is used to restore the received semantic features into noisy features through upsampling, comprising: Inputting the received semantic features into the first semantic information recovery module to obtain a semantic information recovery result; Inputting the semantic information recovery result and the channel state into the channel sensing module to obtain a second channel sensing result; The second channel perception result is input into the second semantic information recovery module to obtain a noisy feature.

[0014] In an optional embodiment of the present application, the denoising enhancement module includes: a plurality of Unet2 layers, a Conv2d layer and a channel perception module; Inputting the noisy feature and the channel state into a denoising and enhancement module to reconstruct an image includes: Input the noisy features into the first Unet2 layer in the denoising enhancement module to obtain the noise features; Based on the noise characteristics and channel status, multiple Unet2 layers, Conv2d layers and channel perception modules in the denoising enhancement module are used to obtain the noise characteristics based on the channel status; An image is reconstructed based on the noisy feature and the noise feature based on the channel state.

[0015] In a second aspect of the present application, a semantic communication system for high-resolution images is provided, the system comprising: A semantic feature extraction module, used for performing nonlinear transformation of the image to be encoded from the pixel domain to the feature domain to obtain semantic features, wherein the image to be encoded is a video I frame; A super prior distribution estimation module, used for estimating the super prior distribution of the semantic features to obtain a super prior; A feature segmentation module, used for evenly segmenting the semantic features into four groups along the channel dimension to obtain four groups of feature elements; A distribution prior estimation module, used for performing step-by-step prior estimation based on the four groups of characteristic elements and the super prior using a quadtree-based spatial entropy model to obtain a four-step spatial prior; A semantic importance estimation module, used for estimating four-step semantic importance based on the channel state, the four groups of feature elements and the four-step spatial domain prior; A semantic feature encoding module is used for inputting, for the i-th path among the four paths, a combination of feature elements corresponding to the i-th step estimation index, the i-th step semantic importance, the semantic feature encoding results of the paths before the i-th path, and the channel state into the i-th path semantic feature encoder, and outputting the i-th path semantic feature encoding result, wherein the semantic feature encoder is a four-path semantic feature encoder corresponding to the four-step prior estimation of the spatial entropy model based on the quadtree, and the value range of i is 0-3; A transmission module, used to map the semantic feature encoding result of each channel into symbols of different lengths for wireless transmission through the corresponding symbol mapping modules of each channel; A restoration module is used for each symbol mapping module of the receiving end to restore the symbol according to the symbol dimension; A semantic feature decoding module is used for inputting the channel state, the received semantic feature encoding result of the i-th channel, and the semantic feature decoding results of each channel before the i-th channel into the i-th semantic feature decoder for the i-th channel among the four channels, so as to obtain the semantic feature decoding result of the i-th channel, wherein the semantic feature decoder is a four-channel semantic feature decoder, and the four-channel semantic feature decoding results constitute the received semantic feature; A semantic recovery module, used for inputting the received semantic features and the channel state into a semantic information restorer, and the semantic information restorer is used for restoring the received semantic features into noisy features through upsampling; The reconstruction module is used to input the noisy features and the channel status into the denoising and enhancement module to reconstruct the image.

[0016] The present application includes the following advantages: a semantic communication method and system for high-resolution images are provided in the present application, the image to be encoded is transformed from the pixel domain to the feature domain through nonlinear transformation to obtain semantic features; the semantic features are estimated through super-prior distribution to obtain super-prior; the semantic features are evenly divided into four groups along the channel dimension to obtain four groups of feature elements; based on the four groups of feature elements and the super-prior, a quadtree-based spatial entropy model is used to perform step-by-step prior estimation to obtain a four-step spatial prior; based on the channel state, the four groups of feature elements and the four-step spatial prior, the four-step semantic importance is estimated; for the i-th path among the four paths, the feature element combination corresponding to the i-th step estimation index, the i-th step semantic importance, the semantic feature encoding results of the paths before the i-th path and the channel state are input into the i-th path semantic feature encoder, and the i-th path semantic feature encoding result is output, and the semantic feature encoder is a four-path semantic feature encoder , corresponding to the four-step prior estimation of the spatial entropy model based on the quadtree, the value range of i is 0-3; the semantic feature encoding result of each channel is mapped into symbols of different lengths for wireless transmission through the corresponding symbol mapping modules of each channel; the symbol mapping modules of each channel at the receiving end restore the symbols according to the dimensions of the symbols; for the i-th channel among the four channels, the channel state, the received i-th semantic feature encoding result, and the semantic feature decoding results of each channel before the i-th channel are input into the i-th semantic feature decoder to obtain the i-th semantic feature decoding result, the semantic feature decoder is a four-channel semantic feature decoder, and the four-channel semantic feature decoding results constitute the received semantic feature; the received semantic feature and the channel state are input into the semantic information restorer, and the semantic information restorer is used to restore the received semantic feature to a noisy feature after upsampling; the noisy feature and the channel state are input into the denoising enhancement module to reconstruct the image. The module built with CNN is used to extract and restore the semantic features of video frames, and the codec module is used to process the semantic features of video frames. This not only improves the coding and decoding efficiency, but also enhances the model's ability to understand video content, helps to better reconstruct high-quality video content, and control the size of model parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 It is a flowchart of the steps of a semantic communication method of high-resolution images provided in an embodiment of the present application; Figure 2It is a flowchart of a semantic communication method of a high-resolution image provided in an embodiment of the present application; Figure 3 It is a network structure diagram of a channel perception module in a semantic communication method for high-resolution images provided in an embodiment of the present application; Figure 4 It is a network structure diagram of a symbol mapping module in a semantic communication method for high-resolution images provided in an embodiment of the present application; Figure 5 This is an architecture diagram of a semantic communication system for high-resolution images provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] In order to solve the following problems caused by the traditional image communication system with a separate design when transmitting high-resolution images: under poor channel conditions or limited bandwidth, once a mismatch occurs between the communication transmission code rate and the channel capacity, it will lead to a cliff effect, that is, the communication performance will drop sharply when the channel capacity is lower than the communication transmission code rate, and the receiving end is often unable to recover the image data; the system is only an independent optimization at the symbol level. Under poor channel conditions or limited communication bandwidth, the accuracy at the symbol level is difficult to effectively guarantee the accuracy at the semantic level of the source. The present application proposes a semantic communication method for high-resolution images to solve the above problems. The image frames in the high-resolution images transmitted in the present application are I frames of video image frames, and any I frame can be randomly decoded to play the video stream.

[0021] See also Figure 1 , Figure 1 : is a flowchart of a method for semantic communication of high-resolution images provided in an embodiment of the present application, the method comprising the following steps: Step 101: performing nonlinear transformation from a pixel domain to a feature domain to obtain semantic features of an image to be encoded, wherein the image to be encoded is a video I frame; Step 102: Estimating the hyper-prior distribution of the semantic feature to obtain a hyper-prior; Step 103: Evenly divide the semantic features into four groups along the channel dimension to obtain four groups of feature elements; Step 104: Based on the four groups of feature elements and the super prior, a quadtree-based spatial entropy model is used to perform step-by-step prior estimation to obtain a four-step spatial prior; Step 105: estimating four-step semantic importance based on the channel state, the four groups of feature elements and the four-step spatial domain priors; Step 106: for the i-th path among the four paths, the feature element combination corresponding to the i-th step estimation index, the i-th step semantic importance, the semantic feature encoding results of the paths before the i-th path, and the channel state are input into the i-th path semantic feature encoder, and the i-th path semantic feature encoding result is output, wherein the semantic feature encoder is a four-path semantic feature encoder, corresponding to the four-step prior estimation of the spatial entropy model based on the quadtree, and the value range of i is 0-3; Step 107: Mapping the semantic feature encoding result of each channel into symbols of different lengths for wireless transmission through the symbol mapping module corresponding to each channel; Step 108: The symbol mapping modules of each channel at the receiving end restore the symbols according to the dimensions of the symbols; Step 109: for the i-th channel among the four channels, the channel state, the received i-th channel semantic feature encoding result, and the semantic feature decoding results of each channel before the i-th channel are input into the i-th channel semantic feature decoder to obtain the i-th channel semantic feature decoding result, wherein the semantic feature decoder is a four-channel semantic feature decoder, and the four-channel semantic feature decoding results constitute the received semantic feature; Step 110: inputting the received semantic features and the channel state into a semantic information restorer, wherein the semantic information restorer is used to restore the received semantic features to noisy features through upsampling; Step 111: input the noisy features and the channel status into a denoising and enhancement module to reconstruct an image.

[0022] In order to clearly illustrate the semantic communication method of high-resolution images proposed in this application, Figure 2 To explain, Figure 2 It is a flowchart of a semantic communication method for high-resolution images provided in an embodiment of the present application.

[0023] When implementing step 101, first obtain any I frame in the ultra-high-definition video to be transmitted as a high-resolution image to be encoded, wherein the high-resolution image is an image with a large number of pixels (Pixel) or dots (Dot) in the digital image, and these pixels or dots together constitute the details and clarity of the image. High-resolution images can usually provide finer details, clearer edges and less blur, making the objects, scenes or texts in the image more realistic and easy to identify. The image to be encoded is transformed nonlinearly from the pixel domain to the feature domain to obtain semantic features. The image semantic feature extractor includes a first downsampling module, a channel perception module, a second downsampling module and a Conv2d layer, which can transform the image into an implicit feature space through nonlinear transformation. Each downsampling module contains a downsampling layer and a DepthConvBlock2 layer. Specifically, the acquired image to be encoded Input the first downsampling module to obtain the first downsampling result, input the first downsampling result and the channel state (such as SNR) into the channel sensing module to obtain the channel sensing result, and based on the channel sensing result, use the second downsampling module and the Conv2d layer to obtain the semantic feature .

[0024] When implementing step 102, in order to compact the semantic features, a hyper-prior is introduced in the present application to make a preliminary estimate of the potential distribution of the semantic features, and the hyper-prior distribution estimation is performed through a single neural network model. The hyper-prior distribution estimation is performed on the above-mentioned semantic features to obtain the hyper-prior. Specifically, the above-mentioned semantic features are input into the hyper-prior estimation module to obtain the above-mentioned hyper-prior. The hyper-prior estimation module is only deployed at the transmitting end, and the transmitting end will not transmit the hyper-prior.

[0025] In an optional embodiment of the present application, the super prior estimation module includes two Conv2d layers and one Leaky Relu layer. The super prior estimation module is used to estimate the super prior distribution of the semantic features to obtain the super prior. Specifically, the semantic features Input the hyper prior estimation module to obtain the hyper prior. The above hyper prior estimation module is only deployed at the transmitting end, and the transmitting end will not transmit the hyper prior distribution. and letters The meanings represented are all semantic features, but the letters representing the semantic features are different in different processing scenarios.

[0026] When implementing step 103, the entropy model of the quadtree is introduced in the present application to perform subsequent spatial domain prior distribution estimation and semantic importance estimation. First, the extracted semantic features are evenly divided into four groups along the channel dimension to obtain four groups of feature elements, and the four groups of feature elements are used for subsequent spatial domain prior distribution estimation and semantic importance estimation. The channel dimension of the semantic feature is channel, which can be abbreviated as C.

[0027] When step 104 is specifically implemented, a quadtree-based spatial entropy model is used to estimate the spatial prior distribution and semantic importance. The quadtree-based spatial entropy model can fully identify the redundancy of spatial information in semantic features. First, based on four groups of feature elements and the above-mentioned super prior, a quadtree-based spatial entropy model is used to perform step-by-step prior estimation to obtain a four-step spatial prior. According to the step-by-step estimation strategy of the entropy model, a four-way semantic feature coding transmission link with mutual reference is realized; the output result of the entropy model is used to estimate the semantic importance to guide the feature encoder to fully reduce the channel bandwidth occupancy and realize unequal error protection transmission.

[0028] In an optional embodiment of the present application, when a quadtree-based spatial entropy model is used for step-by-step prior estimation, each group of characteristic elements in each step of the step-by-step prior estimation is different; specifically as follows: the prior used by the characteristic element of step 0 is the initial fusion prior, and the initial fusion prior is generated based on the super prior; the prior used by the characteristic element of step 1 is estimated based on the characteristic element of step 0 and the initial fusion prior; the prior used by the characteristic element of step 2 is estimated based on the characteristic element of step 0, the characteristic element of step 1, and the initial fusion prior; the prior used by the characteristic element of step 3 is estimated based on the characteristic element of step 0, the characteristic element of step 1, the characteristic element of step 2, and the initial fusion prior; since the prior used by the characteristic element of each step is different, the characteristic element of each step is also different. The prior used by the characteristic element of each step is used to estimate the symbol length corresponding to the encoding of the characteristic element of that step.

[0029] In an optional embodiment of the present application, based on the four groups of feature elements and the super prior, a quadtree-based spatial entropy model is used to perform step-by-step prior estimation, and the process of obtaining a four-step spatial prior specifically includes: firstly, an initial fusion prior is generated based on the above super prior, and the initial fusion prior is used as the spatial estimation result of the 0th step spatial estimation; then, in the process of the 1st to 3rd steps of the spatial estimation, the estimated distribution characteristics obtained in the previous steps and the initial fusion prior are used as references, and the estimation is completed using a network with shared weights to obtain the spatial prior of each step. The network structure of the quadtree-based spatial entropy model used in the present application is roughly the same as that of the prior art, but the part of the output quantization step length is removed.

[0030] In an optional embodiment of the present application, the spatial domain prior distribution estimation is divided into four steps, which can be referred to Figure 2 , along the channel dimension C, the dimension is The semantic features are evenly divided into four groups of dimensions: Each group will estimate the feature element at the corresponding index at each step. C represents the channel dimension, H represents the height dimension, and W represents the width dimension. For example, step 0 ( Figure 2 Step 0) Each group will Figure 2 The position with index 0 in the image is estimated. The indexes of each group are different, but the indexes of each group can cover the dimension [H, W] when combined. The indexes of each step are also different. The combination of feature elements used in each step of estimation is all feature elements corresponding to the estimated index of each step. Therefore, each step of spatial prior distribution estimation can cover each spatial position of dimension [H, W], and the estimated feature elements can form features of dimension [C / 4, H, W]. In the subsequent first to third steps, the spatial prior distribution estimation will use the feature elements of the estimated distribution in the previous steps for reference, so that the position to be estimated in the same group in each step can refer to its neighbors in the same group and the same positions in different groups as references, thereby guiding the model to fully identify the spatial information redundancy in the semantic features. The output parameters of the entropy model will also be used to guide the semantic feature encoder to achieve unequal error protection. Based on the semantic importance estimation module, the prior distribution parameters estimated by the entropy model and the semantic features are input to calculate the semantic importance of each feature element. The output result will interact with the semantic feature encoding process through the cross-attention layer in the semantic feature encoder.

[0031] When step 105 is specifically implemented, when estimating semantic importance, the channel state, the above four groups of feature elements and the above four-step spatial domain priors are input into a semantic importance estimation module to estimate the four-step semantic importance.

[0032] In an optional embodiment of the present application, the network structure of the semantic importance module used in each step of semantic importance estimation is the same. The following is an exemplary description of the semantic importance estimation of the i-th step. The semantic importance estimation module includes multiple DepthConvBlock layers, multiple ResBlock layers and a channel perception module (AF Module), which converts the above i-th group of feature elements (The four groups of feature elements are obtained by segmenting the semantic features, and the alphabetical interpretations of the semantic features are still used here to represent the feature elements) and the i-th step spatial domain prior (Etropy prior) are input into the first DepthConvBlock layer to obtain the first processing result; based on the first processing result and the channel state (such as SNR), the channel sensing module is used to obtain the channel sensing processing result; based on the channel sensing processing result, the second ResBlock layer is used to obtain the i-th step semantic importance, and the value range of i is 0-3.

[0033] When step 106 is specifically implemented, for the i-th path among the four paths, the feature element combination corresponding to the i-th step estimation index, the i-th step semantic importance, the semantic feature encoding results of the paths before the i-th path, and the channel state are input into the i-th path semantic feature encoder, and the i-th path semantic feature encoding result is output, the above-mentioned semantic feature encoder is a four-path semantic feature encoder, corresponding to the four-step prior estimation of the spatial entropy model based on the quadtree, and the value range of i is 0-3; The feature spatial domain composition dimension estimated according to the spatial domain prior distribution of each step is: The feature elements of the proposed method are used to construct a four-way semantic feature encoder, which encodes the features composed of each step of spatial prior distribution estimation respectively to obtain the four-step semantic feature encoding result. The self-attention layer based on Swin-Transformer and the cross-attention layer based on Swin-Transformer-V2 are used to construct a neural network model to remove the information redundancy of the semantic features identified in the spatial distribution estimation.

[0034] The network structure of each semantic feature encoder is the same. Each semantic feature encoder includes: a channel perception module and a semantic feature encoding module. The above semantic feature encoding module includes: a first cross-attention layer and a second cross-attention layer, and also includes a first self-attention layer, a second self-attention layer and a DepthConvBlock layer.

[0035] The characteristic element combination corresponding to the estimated index of the i-th step, the semantic importance of the i-th step, the semantic feature encoding results of each path before the i-th path, and the channel state are input into the i-th semantic feature encoder, and the i-th semantic feature encoding result is output, which specifically includes: the above channel state (such as SNR) and the characteristic element combination corresponding to the estimated index of the i-th step are Input the above-mentioned channel perception module AF module to obtain the first channel perception result; input the above-mentioned first channel perception result and the semantic importance of the i-th step into the above-mentioned first cross attention layer to obtain the first cross attention result; based on the above-mentioned first cross attention result and the semantic feature encoding results of each path before the i-th path , using the second cross attention layer, the i-th semantic feature encoding result is obtained .

[0036] In an optional embodiment of the present application, the network structure of the channel perception module AF Module in the semantic feature encoder can be found in Figure 3 , Figure 3 1 is a network structure diagram of a channel perception module in a semantic communication method for high-resolution images provided in an embodiment of the present application. The channel perception module includes a spatial attention module and a channel attention module. The channel state and the characteristic element combination corresponding to the i-th step estimation index are input into the channel perception module to obtain a first channel perception result, specifically including: the channel state (for example, SNR) and the characteristic element combination corresponding to the i-th step estimation index are input into the channel perception module. Input the channel attention module to obtain the channel attention result; based on the feature element combination corresponding to the estimated index in the above step i And the above channel attention results, use the above spatial attention module to obtain the spatial attention results; based on the feature element combination corresponding to the estimated index in the above step i The above-mentioned first channel perception result is obtained by combining the above-mentioned spatial attention result. By integrating the channel perception module, the encoding and decoding strategy can be adaptively adjusted according to the real-time channel conditions (channel status), and the important features of the video data can be enhanced with the attention mechanism, thereby improving the overall transmission efficiency and quality.

[0037] When the steps 107 and 108 are implemented, after obtaining the i-th semantic feature coding result, the corresponding symbol mapping module of each channel maps each semantic feature coding result into symbols of different lengths for wireless transmission; then, the symbol mapping module of each channel at the receiving end restores the symbol according to the symbol dimension, and transmits the restored coding result. The network architecture of the symbol mapping module is shown in FIG. Figure 4 , Figure 4 : is a network structure diagram of a symbol mapping module in a semantic communication method for high-resolution images provided in an embodiment of the present application. The dimension of the output result after the semantic feature passes through the feature encoder is , for the dimension Each element of has a length of , according to the hyper-prior distribution estimated by the entropy model, the symbol length required for the element is calculated, and a linear layer matching the symbol length is selected for mapping. At the receiving end, the corresponding linear layer will be used to map it back to length , thereby recovering the dimension from the received symbol The coded transmission result is then transmitted to the semantic feature decoder. Based on the symbol mapping module, a semantic communication method for high-resolution images is implemented, which dynamically adjusts the video codec output bit rate according to the channel state and semantic information.

[0038] When step 109 is specifically implemented, for the i-th channel among the four channels, the channel state, the received i-th channel semantic feature encoding result, and the semantic feature decoding results of each channel before the i-th channel are input into the i-th channel semantic feature decoder to obtain the i-th channel semantic feature decoding result, and the semantic feature decoder is a four-channel semantic feature decoder.

[0039] In an optional embodiment of the present application, each semantic feature decoder includes: a channel perception module and a semantic feature decoding module, and the semantic feature decoding module includes: a third cross-attention layer, and also includes: a third self-attention layer, a fourth self-attention layer and a DepthConvBlock layer.

[0040] The channel state, the received i-th semantic feature encoding result, and the i-th semantic feature decoding results are input into the i-th semantic feature decoder to obtain the i-th semantic feature decoding result, specifically including: based on the i-th semantic feature encoding result received , the decoding results of the semantic features before the above i-th path , using the third cross attention layer, obtain the second cross attention result; based on the channel state (such as SNR) and the second cross attention result, using the channel perception module AF Module, obtain the i-th semantic feature decoding result , four-way semantic feature decoding results Composition of received semantic features .

[0041] When step 110 is specifically implemented, the received semantic features and the channel state are input into a semantic information restorer, and the semantic information restorer is used to restore the received semantic features to noisy features through upsampling.

[0042] In an optional embodiment of the present application, the above-mentioned semantic information recoverer includes: a first semantic information recovery module, a channel perception module and a second semantic information recovery module; the first semantic information recovery module and the second semantic information recovery module both include a DepthConvBlock2 layer and an upsampling layer.

[0043] The received semantic features and the channel state are input into a semantic information restorer, and the semantic information restorer is used to restore the received semantic features to noisy features through upsampling, specifically including: Input the first semantic information recovery module to obtain the semantic information recovery result; input the semantic information recovery result and the channel state (such as SNR) into the channel sensing module to obtain the second channel sensing result; input the second channel sensing result into the second semantic information recovery module to obtain the noisy feature .

[0044] When implementing step 111, in order to further improve the image reconstruction capability of the receiving end, a feature denoising and enhancement module is constructed to further remove the interference of noise on image reconstruction. The above-mentioned noisy features and the above-mentioned channel state are sent to the denoising and enhancement module to reconstruct a high-quality image.

[0045] In an optional embodiment of the present application, the denoising enhancement module includes: multiple Unet2 layers, Conv2d layers and a channel perception module. Input the first Unet2 layer in the I-frame denoising enhancement module to obtain the noise features ; Based on noise characteristics and channel status (such as SNR), using multiple Unet2 layers, Conv2d layers and channel perception module AF Module in the I frame denoising enhancement module to obtain noise features based on the channel status; based on the noise features and the noise characteristics based on the channel state to reconstruct the image .

[0046] In an embodiment of the present application, a semantic communication method for high-resolution images is provided, wherein an image to be encoded is nonlinearly transformed from a pixel domain to a feature domain to obtain semantic features, wherein the image to be encoded is a video I frame; a super-prior distribution estimation is performed on the semantic features to obtain a super-prior; the semantic features are evenly divided into four groups along the channel dimension to obtain four groups of feature elements; based on the four groups of feature elements and the super-prior, a quadtree-based spatial entropy model is used to perform step-by-step prior estimation to obtain a four-step spatial prior; based on the channel state, the four groups of feature elements and the four-step spatial prior, a four-step semantic importance is estimated; for the i-th path among the four paths, the feature element combination corresponding to the i-th step estimation index, the i-th step semantic importance, the semantic feature encoding results of each path before the i-th path and the channel state are input into the i-th path semantic feature encoder, and the i-th path semantic feature encoding result is output, wherein the semantic feature encoder is a four-path semantic feature encoder. The device corresponds to the four-step prior estimation of the spatial entropy model based on the quadtree, and the value range of i is 0-3; the semantic feature encoding result of each channel is mapped into symbols of different lengths for wireless transmission through the symbol mapping modules corresponding to each channel; the symbol mapping modules of each channel at the receiving end restore the symbols according to the dimensions of the symbols; for the i-th channel among the four channels, the channel state, the received i-th channel semantic feature encoding result, and the semantic feature decoding results of each channel before the i-th channel are input into the i-th channel semantic feature decoder to obtain the i-th channel semantic feature decoding result, the semantic feature decoder is a four-channel semantic feature decoder, and the four-channel semantic feature decoding results constitute the received semantic feature; the received semantic feature and the channel state are input into the semantic information restorer, and the semantic information restorer is used to restore the received semantic feature to a noisy feature after upsampling; the noisy feature and the channel state are input into the denoising enhancement module to reconstruct the image. The module built with CNN is used to extract and restore the semantic features of high-resolution image frames, and the codec module is used to process the semantic features of the image. This not only improves the coding and decoding efficiency, but also enhances the model's ability to understand the image content, helps to better reconstruct high-quality image frequency content, and control the size of model parameters.

[0047] In a second aspect of the present application, a semantic communication system for high-resolution images is provided. Figure 5 , Figure 5 : is an architecture diagram of a semantic communication system for high-resolution images provided in an embodiment of the present application, the system comprising: The semantic feature extraction module 501 is used to perform nonlinear transformation from the pixel domain to the feature domain to obtain semantic features, wherein the image to be encoded is a video I frame; A super prior distribution estimation module 502 is used to estimate the super prior distribution of the semantic features to obtain a super prior; A feature segmentation module 503 is used to evenly segment the semantic features into four groups along the channel dimension to obtain four groups of feature elements; A distribution prior estimation module 504 is used to perform step-by-step prior estimation based on the four groups of feature elements and the super prior using a quadtree-based spatial entropy model to obtain a four-step spatial prior; A semantic importance estimation module 505, configured to estimate four-step semantic importance based on the channel state, the four groups of feature elements and the four-step spatial domain priors; The semantic feature encoding module 506 is used to input the feature element combination corresponding to the estimated index of the i-th step, the semantic importance of the i-th step, the semantic feature encoding results of the paths before the i-th path, and the channel state into the i-th path semantic feature encoder, and output the i-th path semantic feature encoding result, wherein the semantic feature encoder is a four-path semantic feature encoder corresponding to the four-step prior estimation of the spatial entropy model based on the quadtree, and the value range of i is 0-3; The transmission module 507 is used to map the semantic feature encoding result of each channel into symbols of different lengths for wireless transmission through the symbol mapping modules corresponding to each channel; Restoration module 508, used for each symbol mapping module of the receiving end, restores the symbol according to the dimension of the symbol; The semantic feature decoding module 509 is used for inputting the channel state, the received semantic feature encoding result of the i-th channel, and the semantic feature decoding results of each channel before the i-th channel into the i-th semantic feature decoder for the i-th channel among the four channels, so as to obtain the semantic feature decoding result of the i-th channel, wherein the semantic feature decoder is a four-channel semantic feature decoder, and the four-channel semantic feature decoding results constitute the received semantic feature; A semantic recovery module 510, configured to input the received semantic features and the channel state into a semantic information restorer, wherein the semantic information restorer is configured to restore the received semantic features into noisy features through upsampling; The reconstruction module 511 is used to input the noisy features and the channel status into a denoising and enhancement module to reconstruct an image.

[0048] Wherein, the hyper-prior distribution estimation module includes: The distribution estimation submodule is used to input the semantic features into the super prior estimation module to obtain the super prior. The super prior estimation module is only deployed at the transmitting end, and the transmitting end will not transmit the super prior.

[0049] Wherein, each group of characteristic elements in each step of the prior estimation of the distribution is different; the prior estimation module of the distribution includes: A 0-step prior generation submodule, in which the prior used by the feature element of the 0-step is an initial fusion prior, and the initial fusion prior is generated according to the super prior; The prior generation submodule of step 1 uses the priors estimated based on the feature elements of step 0 and the initial fusion priors for the feature elements of step 1; The prior generation submodule of step 2, the feature elements of step 2 will use the priors estimated based on the feature elements of step 0, the feature elements of step 1 and the initial fusion priors; The prior generation submodule of step 3, the prior used for the feature elements of step 3 is estimated based on the feature elements of step 0, the feature elements of step 1, the feature elements of step 2 and the initial fusion prior; The symbol estimation submodule is used to estimate the symbol length corresponding to the encoding of the feature element at each step using the prior information used for the feature element at that step.

[0050] Wherein, the distribution prior estimation module includes: An initial estimation module, used for generating an initial fusion prior according to the super prior and using it as a spatial domain estimation result of the spatial domain estimation in step 0; The submodule for estimating the spatial domain prior at each step is used to utilize the estimated distribution features obtained in the previous steps and the initial fusion prior as references during the process of spatial domain estimation in steps 1 to 3, and complete the estimation using a network with shared weights to obtain the spatial domain prior at each step.

[0051] Among them, each semantic feature encoder in the semantic feature encoding module includes: a channel perception module and a semantic feature encoding module, the semantic feature encoding module includes: a first cross attention layer and a second cross attention layer, and the semantic feature encoding module includes: A first channel sensing submodule, configured to input the channel state and the characteristic element combination corresponding to the i-th step estimation index into the channel sensing module to obtain a first channel sensing result; A first cross attention submodule, used for inputting the first channel perception result and the i-th step semantic importance into the first cross attention layer to obtain a first cross attention result; The second cross-attention sub-module is used to obtain the i-th semantic feature encoding result based on the first cross-attention result and the semantic feature encoding results of each path before the i-th path using the second cross-attention layer.

[0052] Among them, the channel perception module in the first channel perception submodule includes a spatial attention module and a channel attention module; the first channel perception submodule includes: A channel attention unit, used for inputting the channel state and the characteristic element combination corresponding to the i-th step estimation index into the channel attention module to obtain a channel attention result; A spatial attention unit, configured to obtain a spatial attention result using the spatial attention module based on the feature element combination corresponding to the estimated index in the i-th step and the channel attention result; A combining unit is used to obtain the first channel perception result based on the combination of feature elements corresponding to the i-th step estimation index and the spatial attention result.

[0053] Among them, each semantic feature decoder in the semantic feature decoding module includes: a channel perception module and a semantic feature decoding module, and the semantic feature decoding module includes: a third cross attention layer; the semantic feature decoding module includes: A first decoding submodule, configured to obtain a second cross attention result by using the third cross attention layer based on the received i-th semantic feature encoding result and the semantic feature decoding results before the i-th path; The second decoding submodule is used to obtain the i-th semantic feature decoding result based on the channel state and the second cross-attention result using the channel perception module.

[0054] The semantic information restorer in the semantic restoration module includes: a first semantic information restoration module, a channel perception module and a second semantic information restoration module; the semantic restoration module includes: A first recovery submodule, used for inputting the received semantic features into the first semantic information recovery module to obtain a semantic information recovery result; A second recovery submodule, configured to input the semantic information recovery result and the channel state into the channel sensing module to obtain a second channel sensing result; The third recovery submodule is used to input the second channel perception result into the second semantic information recovery module to obtain a noisy feature.

[0055] The denoising enhancement module in the reconstruction module includes: multiple Unet2 layers, Conv2d layers and a channel perception module; the reconstruction module includes: A first denoising submodule, configured to input the noisy feature into the first Unet2 layer in the denoising enhancement module to obtain a noise feature; The second denoising submodule is used to obtain the noise characteristics based on the channel state by using the multiple Unet2 layers, Conv2d layers and channel perception modules in the denoising enhancement module based on the noise characteristics and the channel state; The reconstruction submodule is used to reconstruct an image based on the noisy feature and the noise feature based on the channel state.

[0056] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0057] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0058] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0059] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0060] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0061] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0062] The above is a detailed introduction to the semantic communication method and system for high-resolution images provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A semantic communication method for high-resolution images, characterized in that: The method comprises: The image to be encoded is nonlinearly transformed from the pixel domain to the feature domain to obtain semantic features, wherein the image to be encoded is a video I frame; Performing a hyper-prior distribution estimation on the semantic feature to obtain a hyper-prior; Evenly dividing the semantic features into four groups along the channel dimension to obtain four groups of feature elements; Based on the four groups of characteristic elements and the super prior, a quadtree-based spatial entropy model is used to perform step-by-step prior estimation to obtain a four-step spatial prior; Based on the channel state, the four groups of feature elements and the four-step spatial domain prior, four-step semantic importance is estimated; For the i-th path among the four paths, the feature element combination corresponding to the i-th step estimation index, the i-th step semantic importance, the semantic feature encoding results of the paths before the i-th path, and the channel state are input into the i-th path semantic feature encoder, and the i-th path semantic feature encoding result is output, wherein the semantic feature encoder is a four-path semantic feature encoder, corresponding to the four-step prior estimation of the spatial entropy model based on the quadtree, and the value range of i is 0-3; The semantic feature encoding result of each channel is mapped into symbols of different lengths for wireless transmission through the corresponding symbol mapping modules of each channel; The symbol mapping modules of each channel at the receiving end restore the symbols according to their dimensions; For the i-th channel among the four channels, the channel state, the received i-th channel semantic feature encoding result, and the semantic feature decoding results of each channel before the i-th channel are input into the i-th channel semantic feature decoder to obtain the i-th channel semantic feature decoding result, the semantic feature decoder is a four-channel semantic feature decoder, and the four-channel semantic feature decoding results constitute the received semantic feature; Inputting the received semantic features and the channel state into a semantic information restorer, wherein the semantic information restorer is used to restore the received semantic features into noisy features through upsampling; The noisy features and the channel status are input into a denoising and enhancement module to reconstruct an image.

2. The semantic communication method of high-resolution images according to claim 1, characterized in that: The hyper-prior distribution of the semantic feature is estimated to obtain the hyper-prior, including: The semantic feature is input into a super-prior estimation module to obtain the super-prior. The super-prior estimation module is only deployed at the transmitting end, and the transmitting end will not transmit the super-prior.

3. The semantic communication method of high-resolution images according to claim 1, characterized in that: When the quadtree-based spatial entropy model is used for step-by-step prior estimation, each group of characteristic elements in each step of the step-by-step prior estimation is different; The prior used by the feature element of step 0 is the initial fusion prior, which is generated based on the super prior; The feature elements in step 1 will use the priors estimated based on the feature elements in step 0 and the initial fusion priors; The feature elements in step 2 will use the priors estimated based on the feature elements in step 0, the feature elements in step 1, and the initial fusion priors; The feature elements in step 3 will use the priors estimated based on the feature elements in step 0, the feature elements in step 1, the feature elements in step 2, and the initial fusion priors; The prior used in each step of the characteristic element is used to estimate the symbol length corresponding to the encoding of the characteristic element of this step.

4. The semantic communication method of high-resolution images according to claim 1, characterized in that: Based on the four groups of characteristic elements and the super prior, a quadtree-based spatial entropy model is used to perform step-by-step prior estimation to obtain a four-step spatial prior, including: Generate an initial fusion prior according to the super prior and use it as the spatial domain estimation result of the spatial domain estimation in step 0; In the process of spatial domain estimation from step 1 to step 3, the estimated distribution features obtained in the previous step and the initial fusion prior are used as references, and the estimation is completed using a network with shared weights to obtain the spatial domain prior for each step.

5. The semantic communication method of high-resolution images according to claim 1, characterized in that: Each semantic feature encoder includes: a channel perception module and a semantic feature encoding module, and the semantic feature encoding module includes: a first cross attention layer and a second cross attention layer; The feature element combination corresponding to the estimated index of the i-th step, the semantic importance of the i-th step, the semantic feature encoding results of each path before the i-th path, and the channel state are input into the i-th semantic feature encoder, and the i-th semantic feature encoding result is output, including: Inputting the channel state and the characteristic element combination corresponding to the i-th step estimation index into the channel sensing module to obtain a first channel sensing result; Inputting the first channel perception result and the i-th step semantic importance into the first cross attention layer to obtain a first cross attention result; Based on the first cross-attention result and the semantic feature encoding results of each path before the i-th path, the second cross-attention layer is used to obtain the i-th semantic feature encoding result.

6. The semantic communication method of high-resolution images according to claim 5, characterized in that: The channel perception module includes a spatial attention module and a channel attention module; Inputting the channel state and the characteristic element combination corresponding to the i-th step estimation index into the channel sensing module to obtain a first channel sensing result, including: Input the channel state and the characteristic element combination corresponding to the i-th step estimation index into the channel attention module to obtain a channel attention result; Based on the feature element combination corresponding to the estimated index in the i-th step and the channel attention result, the spatial attention result is obtained by using the spatial attention module; Based on the combination of feature elements corresponding to the i-th step estimation index and the spatial attention result, the first channel perception result is obtained.

7. The semantic communication method of high-resolution images according to claim 1, characterized in that: Each semantic feature decoder includes: a channel perception module and a semantic feature decoding module, and the semantic feature decoding module includes: a third cross attention layer; Inputting the channel state, the received i-th semantic feature encoding result, and the semantic feature decoding results of each channel before the i-th semantic feature decoder to obtain the i-th semantic feature decoding result, including: Based on the received i-th semantic feature encoding result and the semantic feature decoding results before the i-th path, using the third cross attention layer, a second cross attention result is obtained; Based on the channel state and the second cross-attention result, the channel perception module is utilized to obtain the i-th semantic feature decoding result.

8. The semantic communication method of high-resolution images according to claim 1, characterized in that: The semantic information restorer comprises: a first semantic information restoration module, a channel perception module and a second semantic information restoration module; Inputting the received semantic features and the channel state into a semantic information restorer, wherein the semantic information restorer is used to restore the received semantic features to noisy features through upsampling, including: Inputting the received semantic features into the first semantic information recovery module to obtain a semantic information recovery result; Inputting the semantic information recovery result and the channel state into the channel sensing module to obtain a second channel sensing result; The second channel perception result is input into the second semantic information recovery module to obtain a noisy feature.

9. The semantic communication method of high-resolution images according to claim 1, characterized in that: The denoising enhancement module includes: multiple Unet2 layers, Conv2d layers and a channel perception module; Inputting the noisy feature and the channel state into a denoising and enhancement module to reconstruct an image includes: Input the noisy features into the first Unet2 layer in the denoising enhancement module to obtain the noise features; Based on the noise characteristics and channel status, multiple Unet2 layers, Conv2d layers and channel perception modules in the denoising enhancement module are used to obtain the noise characteristics based on the channel status; An image is reconstructed based on the noisy feature and the noise feature based on the channel state.

10. A semantic communication system for high-resolution images, characterized in that: The system comprises: A semantic feature extraction module, used for performing nonlinear transformation of the image to be encoded from the pixel domain to the feature domain to obtain semantic features, wherein the image to be encoded is a video I frame; A super prior distribution estimation module, used for estimating the super prior distribution of the semantic features to obtain a super prior; A feature segmentation module, used for evenly segmenting the semantic features into four groups along the channel dimension to obtain four groups of feature elements; A distribution prior estimation module, used for performing step-by-step prior estimation based on the four groups of characteristic elements and the super prior using a quadtree-based spatial entropy model to obtain a four-step spatial prior; A semantic importance estimation module, used for estimating four-step semantic importance based on the channel state, the four groups of feature elements and the four-step spatial domain prior; A semantic feature encoding module is used for inputting the feature element combination corresponding to the estimated index of the i-th step, the semantic importance of the i-th step, the semantic feature encoding results of the paths before the i-th path, and the channel state into the i-th path semantic feature encoder, and outputting the i-th path semantic feature encoding result, wherein the semantic feature encoder is a four-path semantic feature encoder corresponding to the four-step prior estimation of the spatial entropy model based on the quadtree, and the value range of i is 0-3; A transmission module, used to map the semantic feature encoding result of each channel into symbols of different lengths for wireless transmission through the corresponding symbol mapping modules of each channel; A restoration module is used for each symbol mapping module of the receiving end to restore the symbol according to the symbol dimension; A semantic feature decoding module is used for inputting the channel state, the received semantic feature encoding result of the i-th channel, and the semantic feature decoding results of each channel before the i-th channel into the i-th semantic feature decoder for the i-th channel among the four channels, so as to obtain the semantic feature decoding result of the i-th channel, wherein the semantic feature decoder is a four-channel semantic feature decoder, and the four-channel semantic feature decoding results constitute the received semantic feature; A semantic recovery module, used for inputting the received semantic features and the channel state into a semantic information restorer, and the semantic information restorer is used for restoring the received semantic features into noisy features through upsampling; The reconstruction module is used to input the noisy features and the channel status into the denoising and enhancement module to reconstruct the image.

Images