Adaptive frequency-domain filtering assisted spatial conversion distributed source coding method

Abstract

The present application belongs to the technical field of distributed source coding, and particularly relates to a distributed source coding method of adaptive frequency domain filter assisted spatial conversion. The present application comprises the following steps: at the encoding end, converting the input image to the frequency domain, generating a filter according to the amplitude spectrum and the phase spectrum by using a neural network, and performing filtering; encoding the frequency domain components filtered by using an auto-encoder; encoding the features by using a trainable auto-encoder; at the decoding end, calculating the affine coefficients by using the same frequency domain components in the input image and the side information, performing affine transformation on the side information to eliminate the distortion caused by the parallax between images, and realizing alignment; and reconstructing the input image in combination with the aligned side information. Compared with the prior art, the present application shifts the design focus to the encoding end, effectively screens out the most important frequency domain components at each compression rate through active adaptive frequency domain filtering, improves the compression efficiency, improves the matching degree of the side information by using affine transformation at the decoding end, and improves the reconstruction effect.

Description

Technical Field

[0001] This invention belongs to the field of distributed source coding technology, specifically relating to a distributed source coding method with adaptive frequency domain filtering-assisted spatial transformation. Background Technology

[0002] With the continuous development of digital communication, coding technology plays a crucial role in the efficient transmission and storage of data. Among these technologies, distributed source coding can independently encode multiple related sources and allows these related sources to be jointly decoded.

[0003] Distributed source coding is suitable for scenarios where multiple sources are highly correlated but inter-source communication is limited or costly, such as wireless sensor networks and multi-camera video systems. Despite the support of the Slepian-Wolf theorem and Wyner-Ziv theory, the practical application of distributed source coding still faces significant challenges, primarily due to the difficulty in accurately capturing and utilizing the correlation between sources at the decoder end, while simultaneously maintaining independent distributed operation at the encoder end. This invention focuses on a special scenario in distributed source coding: compressing the transmitted input image when correlated images exist as side information at the decoder end.

[0004] Recent advances in deep learning have driven innovation in this field. Current methods mainly fall into two categories: one utilizes neural networks for block matching at the image level to handle spatial redundancy and mine image similarities; the other leverages side information at the feature level. These methods significantly improve reconstruction results by optimizing the use of side information. However, these advances primarily optimize the encoder by designing the decoder, without specifically designing the encoder itself, generally employing autoencoders. Furthermore, when utilizing side information, they fail to eliminate distortions between images from two different viewpoints, thus not accurately utilizing the side information. Summary of the Invention

[0005] The purpose of this invention is to overcome the technical defects in the prior art and propose a distributed source coding method for adaptive frequency domain filtering-assisted spatial transformation.

[0006] The distributed source coding method with adaptive frequency domain filtering-assisted spatial transformation provided by this invention includes: at the encoding end, active optimization is performed, specifically by adding an adaptive, self-learning frequency domain filtering module to actively select the frequency domain components to be encoded, thereby improving coding efficiency and achieving high-efficiency compression; at the decoding end, affine coefficients are learned using the frequency domain components of the input image that are identical to the side information, and then distortions caused by different viewpoints are eliminated through affine transformation, thereby making more accurate use of side information and achieving high-quality reconstruction. The specific steps are as follows:

[0007] (1) At the encoding end, a two-dimensional image is input, and its frequency domain representation is generated using Fast Fourier Transform. The amplitude spectrum and phase spectrum are calculated, and a filter generation module is used to generate a filter based on the amplitude spectrum and phase spectrum. This filter generation module contains two convolutional layers and two residual layers. Finally, a sigmoid activation function is added as a regularization term to ensure that the generated filter has physical meaning. See [link to documentation]. Figure 2 As shown;

[0008] (2) The generated filter is used to filter the input image, and the image is returned to the spatial domain by inverse fast Fourier transform after filtering.

[0009] (3) The filtered image is extracted and compressed using an autoencoder. The encoder contains a four-layer convolutional structure, with a generalized division normalization (GDN) activation function added between every two convolutional layers for normalization. Of course, other autoencoder structures can also be used for encoding.

[0010] (4) The compressed features are encoded with entropy using a trainable entropy coding module; the entropy coding module is added to facilitate the estimation of the compression ratio.

[0011] (5) At the decoding end, the compressed features of the input image are obtained through entropy decoding;

[0012] (6) The decoding end performs steps (2) and (3) on the edge information in sequence to obtain the filtered and encoded edge information features;

[0013] (7) Concatenate the features of the input image obtained in step (5) and the edge information features obtained in step (6), transform the feature dimension into a one-dimensional vector, perform dimensionality reduction using a linear layer, and then feed it into the affine coefficient learning module to learn the affine coefficients; the affine coefficient calculation module is specifically implemented using a Transformer structure, and the output is dimensionality reduced using a linear layer, finally obtaining a... affine matrix;

[0014] (8) Perform affine transformation on the edge information based on the affine coefficients obtained in step (7) (specifically including rotation, translation, scaling, lateral cutting, etc.) to achieve edge information alignment without losing information.

[0015] (9) The aligned edge information from step 8) is fed into the neural network autoencoder to extract common features. The encoder has the same structure as the encoder in step (3), but the parameters are independent.

[0016] (10) The common features in the compressed features of the input image obtained in step (5) and the side information obtained in step (9) are concatenated and sent to the decoder to obtain the reconstruction result;

[0017] (11) During the training phase, the input image is fed into the autoencoder in step (9) to extract common features as affine transformation labels. The affine loss is calculated with the common features obtained from the edge information in step (9) to constrain the affine process. The loss is calculated under the MSE metric.

[0018] (12) During the training phase, the input image is used as the label, the loss between the label and the reconstruction result is calculated as the reconstruction loss, and the reconstruction loss and the affine loss in step (11) are combined to form the loss function of the entire network, and the neural network is trained iteratively.

[0019] In step (1), an adaptive learning filter based on the amplitude spectrum and phase spectrum is used.

[0020] In step (10), the affine loss is calculated by comparing the features of the affine result with the features of the input image.

[0021] In step (11), the edge information features extracted after affine transformation are used to assist in reconstruction.

[0022] Compared with existing technologies, this invention shifts the design focus to the encoding end, effectively filtering out the most important frequency domain components at each compression ratio through active adaptive frequency domain filtering, thereby improving compression efficiency; at the decoding end, affine transformation is used to improve the matching degree of side information, thereby improving the reconstruction effect. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall framework of the present invention.

[0024] Figure 2 The left image shows the filter generation module of this invention. Figure 2 The right side shows the residual module in the filter generation module.

[0025] Figure 3 This is the frequency domain filtering process of the present invention. Detailed Implementation

[0026] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0027] See Figure 1 This paper describes a distributed source coding method based on adaptive frequency domain filtering-assisted spatial transformation. First, the input image is transformed to the frequency domain, and the amplitude and phase spectra are calculated to generate filters. After filtering, the input image is compressed using a neural network, followed by entropy coding, and then sent to the decoder. The decoder first performs entropy decoding to obtain the features of the input image. Then, related images at the decoder undergo the same filtering and compression as the input image to obtain features. Affine coefficients are learned using the similar features of the two images, and affine transformations are performed on the related images to eliminate distortions caused by different viewpoints. Features are extracted from the affine-transformed related images to assist in the reconstruction of the input image.

[0028] The present invention will now be described using the KITTI Stereo dataset as an example. This dataset contains 1576 pairs of stereo images as the training set and another 790 pairs as the test set. The original data with an image size of 375×1242 is cropped to a 370×740 local image, which is then downsampled to 128×256.

[0029] (1) Input image Perform a Fast Fourier Transform (FFT) to obtain the frequency domain representation. Calculate the amplitude spectrum A and the phase spectrum φ.

[0030]

[0031] φ=arctan2(Im(X f ),Re(X f ))

[0032] (2) Concatenate A and φ along the first dimension and input them into the filter generation module to generate a filter. This module contains two convolutional layers and two residual layers, as shown in the attached diagram. Figure 2 As shown. Then X f Filtering is performed by multiplying the spectrum graph by the filter to generate... The filtering process is as follows: Figure 3 As shown.

[0033] (3) The system is transformed back to the spatial domain using the Inverse Fast Fourier Transform (IFFT) and then encoded using the E encoder. x Encode to obtain encoded features This encoder consists of four convolutional layers, with a Generalized Division Normalization (GDN) activation function added between each pair of convolutional layers for normalization. The encoder structure is not unique; downsampling modules from other structures can also serve as the encoder. The number of output feature channels is also not unique and can be reduced or increased.

[0034] (4) After quantizing the encoded features, entropy encoding is performed. Since the non-differentiability of quantization affects gradient backpropagation during network training, the quantization process is transformed into superimposed uniformly sampled random noise to simulate the loss caused by quantization during the training phase. The superimposed noise ranges within the interval [-0.5, 0.5). During the testing phase, quantization is achieved by directly rounding the encoded features.

[0035] (5) A channel code of the form LDPC is used to protect the quantization bits and ensure their secure passage through the channel. Other channel codes can also be used as channel codes here. The decoder obtains the coding feature x through entropy decoding. enc .

[0036] (6) Using similar components from two images can more accurately find the correlation between them, thus obtaining more accurate affine coefficients. Therefore, we can use the relevant images y t Perform the same processing as the input image sequentially, first using FFT to transform y t The signal is converted to the frequency domain, then filtered using the filter obtained in step (2), and then converted back to the spatial domain using IFFT. Finally, it is encoded using the encoder in step (3) to obtain compressed features.

[0037] (7) Take the x obtained in step (5) enc and y obtained in step (6) xenc After concatenating the first dimension to form a one-dimensional vector, a linear layer is used to reduce it to 256 dimensions before feeding it into the affine coefficient calculation module. This module, implemented using a Transformer architecture, reduces the Transformer's output to 6 dimensions through a linear layer, outputting a single... The vector is transformed in dimension to obtain the affine coefficient matrix.

[0038] (8) Using affine matrices to analyze the relevant image y t Perform an affine transformation to obtain

[0039] (9) Utilizing the neural network encoder E y Extracting affine transformations y affine Features This encoder is used to assist in the reconstruction of the input image. The encoder has the same structure as the encoder in step (3), but the parameters are independent.

[0040] (10) Utilizing the neural network encoder E y Extract input image x t The characteristics were obtained. And the y obtained in (9) enc Calculate the mean squared error loss to obtain the affine loss.

[0041]

[0042] (11) Take the x obtained in step (5) enc The y obtained in step (9) enc The first dimension is concatenated using decoder D. x Decode the code to obtain the reconstruction result. Calculate x and The mean squared error loss between the two sides is used to obtain the reconstruction loss. This decoder D... xIt consists of four deconvolutional layers, with an inverse generalized division normalization (GDN) activation function added between every two deconvolutional layers. Other upsampling structures can also be used for the decoder here.

[0043]

[0044] (12) Calculate the overall loss of the network:

[0045]

[0046] Where R is x enc The bitrate is calculated using bpp, and this loss is used to iteratively optimize the network.

Claims

1. A distributed source coding method for adaptive frequency domain filtering-assisted spatial transformation, characterized by: include: At the encoding end, active optimization is performed by adding an adaptive, self-learning frequency domain filtering module to actively select the frequency domain components to be encoded, thereby improving encoding efficiency and achieving high-efficiency compression. At the decoding end, affine coefficients are learned using the frequency domain components of the input image that are identical to the side information. Then, affine transformation is used to eliminate distortions caused by different viewpoints, thus making more accurate use of side information and achieving high-quality reconstruction. The specific steps are as follows: (1) At the encoding end, input a two-dimensional image, use fast Fourier transform to generate its frequency domain representation, and calculate the amplitude spectrum and phase spectrum. Use the filter generation module to generate a filter based on the amplitude spectrum and phase spectrum. The filter generation module contains two convolutional layers and two residual layers; (2) The generated filter is used to filter the input image, and the image is returned to the spatial domain by inverse fast Fourier transform after filtering. (3) Use an autoencoder to extract features and compress the filtered image; (4) The compressed features are entropy encoded using a trainable entropy coding module; (5) At the decoding end, the compressed features of the input image are obtained through entropy decoding; (6) The decoding end performs steps (2) and (3) on the edge information in sequence to obtain the filtered and encoded edge information features; (7) Using the affine coefficient calculation module, based on the features of the input image obtained in step (5) and the side information features obtained in step (6), the affine coefficients are learned; the affine coefficient calculation module is specifically implemented using a converter structure; (8) Perform an affine transformation on the edge information based on the affine coefficients obtained in step (7) to achieve edge information alignment; (9) Using a neural network encoder, extract the features of the aligned information in step (8); (10) Use the neural network encoder in step (9) to extract the features of the input image and calculate the affine loss with the side information features obtained in step (9); (11) The features obtained in steps (5) and (9) are spliced ​​together and sent to the decoder to obtain the reconstruction result; (12) Calculate the image reconstruction loss and affine transformation loss, construct the loss function, and iteratively train the neural network.

2. The distributed source coding method with adaptive frequency domain filtering-assisted spatial transformation as described in claim 1, characterized in that, In step (1), an adaptive learning filter based on the amplitude spectrum and phase spectrum is used.

3. The distributed source coding method for adaptive frequency domain filtering-assisted spatial transformation as described in claim 1, characterized in that, In step 10), the affine loss is calculated by comparing the features of the affine result with the features of the input image.

4. The distributed source coding method for adaptive frequency domain filtering-assisted spatial transformation as described in claim 1, characterized in that, In step (11), the edge information features extracted after affine transformation are used to assist in reconstruction.

Images