Distributed image coding method based on double-domain alignment and selective enhancement

Abstract

The invention discloses a distributed image coding method based on double-domain alignment and selective enhancement, and relates to the technical field of image signal processing. The invention discloses a distributed image coding method based on double-domain alignment and selective enhancement. The method comprises the following steps: S1, establishing a distributed image coding data set; s2, constructing a network architecture; s3, designing a distributed image coding scheme, and building a distributed image coding model according to the designed scheme; s4, training the model by using a deep learning Pytorch framework; s5, inputting a to-be-transmitted image and a side information image test data pair into the model to obtain a transmission image result after coding reconstruction; according to the method, the performance of distributed information source coding is improved to a new height by utilizing a proposed multi-scale pixel-level alignment architecture, a rough domain and fine domain alignment method and an abnormal region repairing method based on selective enhancement.

Description

Technical Field

[0001] The present invention relates to the technical field of image signal processing, and in particular to a distributed image coding method based on dual-domain alignment and selective enhancement. Background Art

[0002] Distributed image coding aims to restore high-quality transmission images using high-quality side information images at the decoding end. It can be applied to many practical applications that require multi-view image data transmission, such as autonomous driving, AR / VR, and camera arrays. Distributed image coding faces the problem of difficulty in aligning side information images with target images, and uneven quality after feature alignment.

[0003] The key problem of learning-based distributed image coding is the alignment between the transmitted low-quality target image and the high-quality side information image. On the one hand, some researchers directly fuse the common information between the side information image and the target image, while others propose alignment based on cross-attention of image blocks. However, this method achieves cross-domain alignment by calculating the correlation between the high-quality side information image and the degraded target image, which hinders the alignment performance. Therefore, alignment within the same domain is particularly important.

[0004] On the other hand, in order to solve the domain difference problem, some researchers let the high-quality side information image pass through the same encoder and decoder process as the target image, and then perform image block matching on the reconstructed target image (feature) and the reconstructed side information image (feature), and use the matching index result to align the side information image (feature); however, the image block matching process is performed in the degraded domain (compressed domain), which cannot generate satisfactory matching performance; recently, researchers introduced an hourglass-based refinement network to further improve the coarse alignment result generated by image block matching, but the two inputs still have a large domain difference, and performing a second alignment based on the previous alignment result may cause error propagation. Therefore, alignment only in the degraded domain is not accurate enough, and using the displacement information provided by the more accurate fusion domain for alignment is a better alignment method; the alignment result cannot be guaranteed to be completely correct, and some researchers use network models to enhance the alignment result to reduce this problem, but the network cannot selectively process the wrong content, resulting in suboptimal performance, so selective processing of the alignment result quality is a feasible solution. Summary of the invention

[0005] The purpose of the present invention is to restore the transmitted image by using side information only at the decoding end, reduce the amount of transmitted information, realize distributed image coding, and on this basis propose a distributed image coding method based on dual-domain alignment and selective enhancement.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A distributed image coding method based on dual-domain alignment and selective enhancement specifically comprises the following steps:

[0008] S1. Establish a distributed image coding dataset: given an image x to be transmitted from a binocular image pair and another side information image s in the binocular image pair, obtain an input data pair (x, s), and then obtain a distributed image coding dataset;

[0009] S2. Build a network framework: Design a network containing an encoder g a The encoding network includes an encoder g a , side information feature extraction network f e , displacement information decoder o d , Rough Domain Alignment Network and Fine domain alignment network w f and selective enhancement modules e The decoder network of the proposed method is an entropy model consisting of a checkerboard context model and a hyper-prior model. m ; Use displacement information decoder, alignment network and side information feature extraction network to build a pixel-level dual-domain alignment network framework;

[0010] S3, design scheme, build model: Based on dual-domain alignment and selective enhancement, a distributed image coding scheme is designed in combination with the distributed image coding dataset and network framework described in S1-S2, and a distributed image coding model is built according to the designed scheme. The scheme specifically includes the following contents:

[0011] ① Encode the image to be transmitted: Input the image to be transmitted x in the data pair (x, s) obtained in S1 into the encoder g at the encoding end a , get the encoding result of the encoder After quantization, the result to be transmitted is obtained ;

[0012] ② Entropy model: At the encoding end, the encoding result of the encoder in ① is Input entropy model e containing checkerboard context model and hyper-prior model m , obtain the entropy value of the information to be transmitted;

[0013] ③Decoding side information image: Input the side information image s in the data pair (x, s) obtained in S1 into the encoder g with the same structure and parameters as described in ① a , get the features

[0014] ④ Extract side information features at the decoding end: Input the side information image s in the data pair (x, s) obtained in S1 into the side information feature extraction network f e, we get the side information features of three scales and

[0015] ⑤ Rough domain displacement information decoding: Decode the features obtained in ③ Features obtained from entropy decoding of the transport stream After stacking along the channel dimension, it is sent to the displacement information decoder o d Get the rough displacement information of three scales and

[0016] ⑥Dual domain alignment: First, perform coarse domain alignment to align the two scales of edge information features in ④ and and the rough displacement information of the two scales in ⑤ and Input the rough domain alignment network separately and w c2 Get the aligned side information features and Secondly, feature fusion is performed to align the features. and The characteristics of the decoder receiving in ② Perform multi-scale fusion; then perform fine domain alignment to combine the multi-scale fusion results ⑤ The rough domain displacement information and the side information features in ④ Input fine domain alignment network w f , and get the dual domain alignment result

[0017] ⑦Selective enhancement module to repair the error area: align the two domains Supplied to the selective enhancement module e , and the enhanced result is

[0018] ⑧ Loss function module design: jointly optimize the entropy loss output by the entropy model and the reconstruction loss at the decoding end;

[0019] S4, training model: Use the deep learning Pytorch framework to train the model, traverse the distributed image coding dataset constructed in S1 until the network loss function converges, then reduce the learning rate from 0.0001 at the beginning to 0.00001, and continue to traverse the distributed image coding dataset several times to obtain the final stable model;

[0020] S5. Output result: Input the data pairs in the distributed image coding data set obtained in S1 into the stable model to obtain the distributed image coding and decoding results.

[0021] Preferably, the scheme ① further includes the following contents:

[0022] Encoder g at the encoding end a It consists of three groups of encoding processing units and convolutions with a stride of 2. The encoding processing units consist of a residual downsampling module and a residual module. The three groups of encoding processing units reduce the scale of the feature map to 1 / 8 of the original, and then pass a 3×3 convolution layer with a stride of 2 to reduce the scale of the feature map to 1 / 16 of the original; the residual downsampling module consists of a 3×3 convolution with a stride of 1, a LeakyReLU activation function, a generalized split normalization layer GDN, a 3×3 convolution with a stride of 2 and a jump operation; the residual module consists of two 3×3 convolutions with a stride of 1, a LeakyReLU activation function and a jump operation.

[0023] Preferably, the scheme ② further includes the following contents:

[0024] ① The encoding result of the encoder After entropy model m The hyper-prior transformation network h a The prior information z to be transmitted is obtained and quantized into Then it is transmitted to the decoding end; at the same time, at the encoding end, After the hyper-prior transformation network h s and the chessboard context model g cm , get the encoding result The mean μ and variance σ of assisted entropy coding; at the decoding end, After the hyper-prior transformation network h s and the chessboard context model g cm , get the encoding result The mean μ and variance σ of

[0025] Preferably, the scheme ④ further includes the following contents:

[0026] Side information feature extraction network f e The structure is three groups of encoding processing units; the encoding processing unit consists of a residual downsampling module and a residual module; each of the three encoding processing units is connected to a residual module as an output head to output feature maps of three scales and The residual downsampling module consists of a 3×3 convolution with a stride of 1, a LeakyReLU activation function, a generalized split normalization layer GDN, a 3×3 convolution with a stride of 2, and a skip operation; the residual module consists of two 3×3 convolutions with a stride of 1, a LeakyReLU activation function, and a skip operation.

[0027] Preferably, the scheme ⑤ further includes the following contents:

[0028] Displacement Information Decoder d It consists of a channel fusion unit and three groups of displacement decoding units. The displacement decoding unit consists of a residual module and a residual upsampling module. Each of the three displacement decoding units is connected to a residual module as an output head. The residual module consists of two 3×3 convolutions with a step size of 1, a LeakyReLU activation function and a skip operation. The residual upsampling module consists of a PixelShuffle upsampling module, a LeakyReLU activation function, an inverse generalized split normalization layer IGDN, a skip operation and another PixelShuffle module. The PixelShuffle module consists of a 3×3 convolution with an output channel number 4 times the number of input channels and a PixelShuffle operation.

[0029] Preferably, the scheme ⑥ further includes the following contents:

[0030] The alignment module in dual-domain alignment is based on pixel-level alignment and uses deformable convolution to achieve feature shifting. The operation is as follows:

[0031]

[0032] Among them, F represents the original feature map, represents the aligned feature map, w represents the convolution parameter, R represents the fixed convolution shift parameter, Δp n Represents the learned displacement; first, the offset feature is obtained and The offset is then used to distort the features extracted from the side information image:

[0033]

[0034] Based on deformable convolution, the input displacement information is utilized to predict multiple offsets for each pixel and aggregate information from multiple positions.

[0035] Preferably, the scheme ⑦ further includes the following contents:

[0036] Features of dual domain alignment First, a weighted mask is generated through the gate module, which is denoted as M; then Element-wise multiplication is performed on M to suppress non-matching (or mismatching) areas; then, a learnable token is introduced and the token is multiplied by 1-M, and the learnable token is filled with the suppressed feature position to repair the mismatching area; finally, after two Swin-T modules, the correlation with adjacent features is used to enhance the area, and the result is obtained.

[0037] Preferably, the scheme ⑧ further includes the following contents:

[0038] The loss function contains the per-pixel bit value R of the information to be transmitted output by the entropy model bpp And the loss D of decoding reconstruction, the specific function is expressed as:

[0039] L=R bpp +λ·D

[0040] Among them, λ is a hyperparameter used to adjust the model rate-distortion performance.

[0041] Compared with the prior art, the present invention provides a distributed image coding method based on dual-domain alignment and selective enhancement, which has the following beneficial effects:

[0042] (1) The present invention proposes a dual-domain alignment method to align high-quality side information features with low-quality target image features, which can achieve pixel-level coarse domain and fine domain offset generation.

[0043] (2) The present invention can observe that the fusion result of the coarse alignment edge information feature (including high-frequency details) and the target feature contains implicit offset information, and can extract fine-scale offset features from the fusion result, and then combine it with the coarse-scale offset features to generate fine domain offset.

[0044] (3) The present invention proposes a selective enhancement module, which further improves the alignment result by enhancing and suppressing mismatching features and repairing them through enhancement. The experiments conducted based on the present invention show that the proposed distributed image coding method achieves the most advanced performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The overall framework diagram of the distributed image coding method based on dual-domain alignment and selective enhancement proposed by the present invention;

[0046] Figure 2 This is a structural diagram of three feature extraction modules proposed in Example 1 of the present invention;

[0047] Figure 3 This is a structural diagram of the dual-domain alignment module proposed in Example 1 of the present invention;

[0048] Figure 4 This is a diagram of the selective enhancement structure proposed in Example 1 of the present invention;

[0049] Figure 5 Schematic diagram of the performance characterization of each method proposed in Example 2 of the present invention on different data sets. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0051] Embodiment 1:

[0052] See also Figure 1 The present invention proposes a distributed image coding method based on dual-domain alignment and selective enhancement, comprising the following steps:

[0053] S1. Establish a distributed image coding dataset: given an image x to be transmitted from a binocular image pair and another side information image s in the binocular image pair, obtain an input data pair (x, s), and then obtain a distributed image coding dataset;

[0054] S2. Build a network framework: Design a network containing an encoder g a The encoding network includes an encoder g a , side information feature extraction network f e , displacement information decoder o d , Rough Domain Alignment Network and Fine domain alignment network w f and selective enhancement modules e The decoder network of the proposed method is an entropy model consisting of a checkerboard context model and a hyper-prior model. m ; Use displacement information decoder, alignment network and side information feature extraction network to build a pixel-level dual-domain alignment network framework;

[0055] S3, design scheme and build model: Based on dual-domain alignment and selective enhancement, a distributed image coding scheme is designed in combination with the distributed image coding dataset and network framework described in S1-S2, and a distributed image coding model is built according to the designed scheme;

[0056] S4, training model: Use the deep learning Pytorch framework to train the model, traverse the distributed image coding dataset constructed in S1 until the network loss function converges, then reduce the learning rate from 0.0001 at the beginning to 0.00001, and continue to traverse the distributed image coding dataset several times to obtain the final stable model;

[0057] S5. Output result: Input the data pairs in the distributed image coding data set obtained in S1 into the stable model to obtain the distributed image coding and decoding results.

[0058] The above method further includes the following contents:

[0059] 3.1 Feature Extraction Module

[0060] The method framework proposed in the present invention has four feature extractors, including two encoders for extracting coarse-scale features from the image to be transmitted x and the side information image s, and a side information feature extractor f e , and an offset decoder o d To extract the offset features. Two encoders (g a ) have the same structure and share the same weights. It consists of four cascaded residual blocks and downsampling operations. For the target image x, only the latent code generated by the last layer of features is transmitted. Then, during the decoding process, the latent code of the transmitted code stream can be Refactoring In order to generate side information features with the same degradation degree, the present invention also uses the encoder to extract the last layer features of s and reconstructs them at the decoder side. and In the same domain (i.e., intra-domain features), they are directly connected along the channel dimension and input into the offset decoder o d The offset decoder consists of three cascaded units, each of which is constructed by a residual block and a PixelShuffle module. Then, the offset features of each level are generated by applying a residual module after each unit. and Side information feature extractor f e Use with encoder g a The same backbone architecture. For each scale, the present invention applies a residual to extract high-quality side information features at different scales.

[0061] 3.2 Rough Domain Alignment

[0062] The main problem of DIC is how to separate fine details from high-quality features. Move to lower quality features Since there is a parallax between s and x, directly fusing these two features cannot produce satisfactory results. Previous methods usually use an alignment method based on image block matching, that is, using the matching index obtained by the rough reconstruction feature, and then rearranging the high-quality side information image features according to the index. However, the alignment based on image block matching is not consistent with the pixel-level parallax between the two views. Therefore, the present invention achieves pixel-level alignment through deformable convolution (DConv).

[0063] The purpose of Deformable Convolution (DConv) is to predict multiple offsets for each pixel and aggregate information from multiple locations to generate an alignment result. The above process can be expressed as:

[0064]

[0065] Among them, F represents the original feature map, represents the alignment result. w represents the convolution weight, R represents the fixed offset of the original convolution, Δp n represents the learnable offset generated by the nonlinear mapping function, which is usually implemented by several convolutional layers. In the present invention, an offset is first generated from the obtained offset feature, and then the offset is used to align the features extracted from the side information image. The above process is expressed as:

[0066]

[0067] Wherein, f is a nonlinear mapping function implemented by the convolution layer, which maps the offset feature to the offset. After deformable convolution (DConv), the present invention obtains The alignment result It should be noted that since the offset feature comes from the compressed domain feature, the predicted offset is also at a coarse level. Therefore, the present invention only considers the features of the second and third scales, i.e. and The present invention would like to point out that although the offset is inaccurate, the rough alignment result The details in are much richer than those in the transmission features because Only decode from compressed information. Therefore, the present invention proposes to fuse the alignment result with the decoded target feature. Figure 1 As shown in Figure 2, an upsampling module (including a 2-layer residual module and a Pixelshuffle block) is used to Upsample to Then, join in Then after the upsampling module, the generated

[0068] 3.3 Fine Domain Alignment

[0069] As described in section 3.2 of rough domain alignment, the present invention obtains the fusion result after cascading the rough domain alignment process. A direct approach is also to use a coarse domain alignment module and then However, due to the offset feature Very rough, direct use Alignment does not produce satisfactory results. It is observed that due to the merging process with high-quality side information image features, The detail ratio There are much more details in . However, since the aligned side information image features cannot be accurately aligned with the target image features. Therefore, the merged result It implicitly contains the scale offset information of the fine domain. Considering that the offset feature can be extracted from the concatenation of two different features by channel, the offset feature can also be extracted from the addition of two different features. Inspired by this, the present invention Two SwinTransformer blocks (which can also be replaced by ordinary convolution blocks) are applied to extract fine-scale offset features, which are then combined with the coarse offset features. Then, after a residual block, fine-scale offsets are further extracted. Note that if only to generate the offset, it cannot capture the large displacement between the side information image and the target image. Similarly, if only To generate the offset, it cannot capture the fine-scale displacement. It helps to refine the fine-scale edge information in the coarse domain alignment. Then, DConv is applied to the features A fine alignment result was obtained

[0070] 3.4 Selective Enhancement Module

[0071] Although the fine alignment process can help to obtain the corresponding fine-scale details, it cannot handle the case where there is no matching content between the side information and the target image. This situation exists in stereo images due to parallax. In addition, in There are still unmatched areas in the image. Therefore, the present invention further proposes a selective enhancement module to repair the non-matching (or mismatching) areas and further enhance the matching areas. Considering that the repair task is similar to image completion, a gated convolution operation is introduced in the selective enhancement module. The gate module is constructed by a residual block and a sigmoid function, and its output value ranges from 0 to 1, which is a mask for dynamic feature selection. Aligned features First, a weighted mask (denoted as M) is generated through the gate module. Then and M perform element-wise multiplication. In this way, non-matching (or mismatching) regions can be suppressed. Then, these regions need to be repaired. The present invention introduces some learnable tokens and multiplies these tokens by 1-M to obtain mismatching regions. The suppressed features are added to the result, such as Figure 4 As shown in Figure 2. In this way, the suppressed regions are filled with these learnable tokens. Then, after passing through the Swin-Transformer layer, these mismatched regions are filled by exploiting the correlation with neighboring features.

[0072] 3.5 Loss Function

[0073] The loss function contains the per-pixel bit value R of the information to be transmitted output by the entropy model bpp And the decoding reconstruction loss D:

[0074] L=R bpp +λ·D

[0075] Among them, λ is a hyperparameter used to adjust the model rate-distortion performance.

[0076] Embodiment 2:

[0077] Based on Example 1, but with the following differences: Figure 5 , the performance of the distributed image compression method based on dual-domain alignment proposed in the present invention is experimented using three common data sets in distributed image coding; wherein, the first data set is KITTI Stereo, the second is Resized KITTI Stereo, and the third is Resized Cityscapes. Since the distribution of pixels has changed after scaling, the Resized KITTI Stereo data set is not the same data set as the original KITTI Stereo. The present invention is compared with the most relevant and advanced asymmetric learning distributed image coding methods MSFDPM, NDIC, ATN, DSIN and the advanced single image compression method VVC-intra. Due to its special network structure, ATN cannot process large-size images, so the present invention is not compared with ATN on the KITTI Stereo data set. For the Resized KITTI Stereo data set, the present invention is not compared with MSFDPM and DSIN, because MSFDPM has no open source code, and DSIN cannot reproduce the corresponding performance using the open source code. For NDIC, the present invention reproduced their results using its open source code by retraining the three data sets. For ease of comparison, the present invention quotes other results from the corresponding papers. Such as Figure 5As shown, the proposed method achieves significant improvements on all BPP (bits per pixel) levels for all three datasets compared to the latest state-of-the-art methods MSFDPM or ATN. The performance of VVC-intra is worse because it does not use side information. It should be noted that the image pairs in the Resized KITTI Stereo dataset have smaller displacements compared to the original KITTI Stereo dataset. Therefore, under the same bpp condition, all DIC methods have better reconstruction performance on the Resized KITTI Stereo dataset. Table 1. Comparison of BD-rate (distortion measured by MS-SSIM) on three datasets. The baseline is NDIC and the best results are highlighted in bold

[0078]

[0079] Please refer to Table 1, which gives the rate-distortion performance in terms of BD-rate. NDIC is selected as the anchor point. The proposed method achieves the largest bd-rate reduction of 78.66%, while the second best method only achieves 54.61%. In other words, the proposed method is 24% better than MSFDPM on the KITTI Stereo dataset. On both adjusted datasets, the bd-rate reduction of the proposed method is more than 45% higher than that of ATN.

[0080] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A distributed image coding method based on dual-domain alignment and selective enhancement, characterized in that: The specific steps include: S1. Establish a distributed image coding dataset: given an image x to be transmitted from a binocular image pair and another side information image s in the binocular image pair, obtain an input data pair (x, s), and then obtain a distributed image coding dataset; S2. Build a network framework: Design a network containing an encoder g a The encoding network includes an encoder g a , side information feature extraction network f e , displacement information decoder o d , Rough Domain Alignment Network and Fine domain alignment network w f and selective enhancement modules e The decoder network of the proposed method is an entropy model consisting of a checkerboard context model and a hyper-prior model. m ; Use displacement information decoder, alignment network and side information feature extraction network to build a pixel-level dual-domain alignment network framework; S3, design scheme, build model: Based on dual-domain alignment and selective enhancement, a distributed image coding scheme is designed in combination with the distributed image coding dataset and network framework described in S1-S2, and a distributed image coding model is built according to the designed scheme. The scheme specifically includes the following contents: ① Encode the image to be transmitted: Input the image to be transmitted x in the data pair (x, s) obtained in S1 into the encoder g at the encoding end a , get the encoding result of the encoder After quantization, the result to be transmitted is obtained ② Entropy model: At the encoding end, the encoding result of the encoder in ① is Input entropy model e containing checkerboard context model and hyper-prior model m , obtain the entropy value of the information to be transmitted; ③Decoding side information image: Input the side information image s in the data pair (x, s) obtained in S1 into the encoder g with the same structure and parameters as described in ① a , get the features ④ Extract side information features at the decoding end: Input the side information image s in the data pair (x, s) obtained in S1 into the side information feature extraction network f e , we get the side information features of three scales and ⑤ Rough domain displacement information decoding: Decode the features obtained in ③ Features obtained from entropy decoding of the transport stream After stacking along the channel dimension, it is sent to the displacement information decoder o d Get the rough displacement information of three scales and ⑥Dual domain alignment: First, perform coarse domain alignment to align the two scales of edge information features in ④ and and the rough displacement information of the two scales in ⑤ and Input the rough domain alignment network separately and Get the aligned side information features and Secondly, feature fusion is performed to align the features. and The characteristics of the decoder receiving in ② Perform multi-scale fusion; then perform fine domain alignment to align the multi-scale fusion results ⑤ The rough domain displacement information and the side information features in ④ Input fine domain alignment network w f , and get the dual domain alignment result ⑦Selective enhancement module to repair the error area: align the two domains Supplied to the selective enhancement module e , and the enhanced result is ⑧ Loss function module design: jointly optimize the entropy loss output by the entropy model and the reconstruction loss at the decoding end; S4, training model: Use the deep learning Pytorch framework to train the model, traverse the distributed image coding dataset constructed in S1 until the network loss function converges, then reduce the learning rate from 0.0001 at the beginning to 0.00001, and continue to traverse the distributed image coding dataset several times to obtain the final stable model; S5. Output result: Input the data pairs in the distributed image coding data set obtained in S1 into the stable model to obtain the distributed image coding and decoding results.

2. The distributed image coding method based on dual-domain alignment and selective enhancement according to claim 1, characterized in that: The scheme ① further includes the following contents: Encoder g at the encoding end a It consists of three groups of encoding processing units and convolutions with a stride of 2. The encoding processing units consist of a residual downsampling module and a residual module. The three groups of encoding processing units reduce the scale of the feature map to 1 / 8 of the original, and then pass a 3×3 convolution layer with a stride of 2 to reduce the scale of the feature map to 1 / 16 of the original; the residual downsampling module consists of a 3×3 convolution with a stride of 1, a LeakyReLU activation function, a generalized split normalization layer GDN, a 3×3 convolution with a stride of 2 and a jump operation; the residual module consists of two 3×3 convolutions with a stride of 1, a LeakyReLU activation function and a jump operation.

3. The distributed image coding method based on dual-domain alignment and selective enhancement according to claim 1, characterized in that: The scheme ② further includes the following contents: ① The encoding result of the encoder After the entropy model m The hyper-prior transformation network h a The prior information z to be transmitted is obtained and quantized into Then it is transmitted to the decoding end; at the same time, at the encoding end, After the hyper-prior transformation network h s and the chessboard context model g cm , get the encoding result The mean μ and variance σ of assisted entropy coding; at the decoding end, After the hyper-prior transformation network h s and the chessboard context model g cm , get the encoding result The mean μ and variance σ of 4. The distributed image coding method based on dual-domain alignment and selective enhancement according to claim 1, characterized in that: The scheme ④ further includes the following contents: Side information feature extraction network f e The structure is three groups of encoding processing units; the encoding processing unit consists of a residual downsampling module and a residual module; each of the three encoding processing units is connected to a residual module as an output head to output feature maps of three scales and The residual downsampling module consists of a 3×3 convolution with a stride of 1, a LeakyReLU activation function, a generalized split normalization layer GDN, a 3×3 convolution with a stride of 2, and a skip operation; the residual module consists of two 3×3 convolutions with a stride of 1, a LeakyReLU activation function, and a skip operation.

5. The distributed image coding method based on dual-domain alignment and selective enhancement according to claim 1, characterized in that: The scheme ⑤ further includes the following contents: Displacement Information Decoder d It consists of a channel fusion unit and three groups of displacement decoding units. The displacement decoding unit consists of a residual module and a residual upsampling module. Each of the three displacement decoding units is connected to a residual module as an output head. The residual module consists of two 3×3 convolutions with a step size of 1, a LeakyReLU activation function and a skip operation. The residual upsampling module consists of a PixelShuffle upsampling module, a LeakyReLU activation function, an inverse generalized split normalization layer IGDN, a skip operation and another PixelShuffle module. The PixelShuffle module consists of a 3×3 convolution with an output channel number 4 times the number of input channels and a PixelShuffle operation.

6. The distributed image coding method based on dual-domain alignment and selective enhancement according to claim 1, characterized in that: The scheme ⑥ further includes the following contents: The alignment module in dual-domain alignment is based on pixel-level alignment and uses deformable convolution to achieve feature shifting. The operation is as follows: Among them, F represents the original feature map, represents the aligned feature map, w represents the convolution parameter, R represents the fixed convolution shift parameter, Δp n Represents the learned displacement; first, the offset feature is obtained and The offset is then used to distort the features extracted from the side information image: Based on deformable convolution, the input displacement information is utilized to predict multiple offsets for each pixel and aggregate information from multiple positions.

7. The distributed image coding method based on dual-domain alignment and selective enhancement according to claim 1, characterized in that: The scheme ⑦ further includes the following contents: Features of dual domain alignment First, a weighted mask is generated through the gate module, which is denoted as M; then Element-wise multiplication is performed on M to suppress the mismatched areas. Then, a learnable token is introduced and multiplied by 1-M. The learnable token is filled with the suppressed feature positions to repair the mismatched areas. Finally, after two Swin-T modules, the correlation with adjacent features is used to enhance the area, and the result is obtained.

8. The distributed image coding method based on dual-domain alignment and selective enhancement according to claim 1, characterized in that: The scheme ⑧ further includes the following contents: The loss function contains the per-pixel bit value R of the information to be transmitted output by the entropy model bpp And the loss D of decoding reconstruction, the specific function is expressed as: L=R bpp +λ·D Among them, λ is a hyperparameter used to adjust the model rate-distortion performance.

Images