Lightweight scalable bit-rate multi-view image compression method and model

Abstract

The application provides a lightweight variable bit rate multi-view image compression method and system, and relates to the technical field of image compression. The method comprises: downsampling, feature extraction and feature scaling of a single-view image to obtain a latent representation corresponding to a target bit rate; quantization and lossless entropy coding of the latent representation to obtain a final compressed bit stream; subsequent lossless entropy decoding and inverse scaling to restore the latent representation; feature fusion and upsampling of the restored latent representations of different views to generate a reconstructed compressed image. The model comprises a main encoder, a feature scaling module, a quantization module, an autoregressive entropy model, an arithmetic encoder, an arithmetic decoder, a feature inverse scaling module and a decoder. The application can efficiently compress image data, reduce computational complexity and storage space occupation while preserving image details and quality, thereby providing faster speed and lower bandwidth requirement for image transmission and storage.

Description

Technical Field

[0001] This invention belongs to the field of image compression technology, specifically relating to a lightweight variable bit rate multi-view image compression method and model. Background Technology

[0002] Image compression is a core task in many popular fields such as digital media transmission and storage, drone and satellite imagery, video conferencing, and video streaming. Its goal is to reduce image size while maintaining visual quality. It is widely used in digital image processing, transmission, and storage because image compression can effectively reduce storage space, lower bandwidth requirements, and accelerate image transmission.

[0003] Traditional image compression methods suffer from limitations such as information loss and fixed compression rates in these application scenarios, while deep learning-based image compression methods show greater advantages.

[0004] Deep learning-based image compression methods employ two key components: an encoder and a decoder. The encoding stage uses models such as convolutional neural networks to extract and encode features from the input image. The decoding stage uses deconvolutional neural networks for feature decoding and reconstruction. This end-to-end training approach allows the model to automatically learn the correlations present in image data and achieve efficient data representation and reconstruction. The combination of the encoder and decoder is achieved through joint training, aiming to minimize reconstruction error at a fixed compression ratio. To address the needs of image data transmission and processing in distributed scenarios, many excellent image compression algorithms have been applied in real-world production environments. Leveraging GPU parallel computing hardware, more complex and effective deep learning neural network models have also been introduced into compression methods for distributed scenarios. These neural networks can automatically learn internal image information and the interrelationships between transmission channels from the data, and the combination of semi-supervised learning and differentiable embedding methods significantly improves communication costs.

[0005] Current image compression algorithms and models typically employ fixed encoding bit rates, failing to dynamically adjust encoding strategies based on image correlations between different views. This results in low encoding efficiency and bandwidth utilization, making them unsuitable for the computationally and resource-constrained demands of distributed scenarios. Furthermore, deep learning-based image compression models often utilize millions of parameters and complex network structures, increasing computational complexity and cost, and thus cannot be directly applied to distributed scenarios.

[0006] Therefore, the applicant considers adopting a multi-view image compression model with variable coding bit rate, which can significantly improve coding efficiency, optimize bandwidth utilization and enhance the adaptability of the image compression model in real distributed scenarios. At the same time, it can also reduce computational complexity without affecting image compression quality. Summary of the Invention

[0007] Based on the problems existing in the prior art, the purpose of this invention is to provide a lightweight variable bit rate multi-view image compression method and model, which can realize variable bit rate encoding and crop the model by converting feature maps with different bit rates. It reduces computational complexity and storage overhead without significantly reducing image compression performance, and can be applied to a wider range of application scenarios, especially suitable for resource-constrained distributed devices, while providing a better user experience and higher system adaptability.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A lightweight variable bit rate multi-view image compression method includes the following steps:

[0010] 1) A feature map is obtained by downsampling and feature extraction of a single-view image, and then the feature map is scaled to obtain a potential representation of the target bit rate;

[0011] 2) Quantize and lossless entropy encode the latent representation to obtain the final compressed bitstream;

[0012] 3) Perform lossless entropy decoding and feature inverse scaling on the final compressed bitstream to recover the latent representation;

[0013] 4) The restored latent representations from different views are fused and upsampled to generate a reconstructed compressed image.

[0014] Preferably, in step 1), the feature scaling uses the following feature scaling function:

[0015] y′=VBF i (y i ) = A i y i +B i

[0016] Where y' is the feature map of the input image after feature scaling, A i Represents the scaling factor, y i Given the feature map of the input image, B i This represents the translation factor.

[0017] Preferably, step 2) specifically includes:

[0018] 2.1) Quantize the latent representation of a single-view image to reduce the amount of data;

[0019] 2.2) The arithmetic encoder performs lossless entropy encoding on the quantized latent representation based on the Gaussian entropy estimation model to obtain the final compressed bitstream.

[0020] Preferably, step 4) includes the following steps:

[0021] 4.1) Perform the first feature fusion on the latent representations after the recovery of different views, so that the latent representations of each view can learn relevant information from other views, and then perform upsampling;

[0022] 4.2) Perform a second feature fusion on the latent representations of all views after upsampling in step 4.1) to further learn relevant information from other views and integrate image features from the latent representations of each view;

[0023] 4.3) After the second feature fusion, upsampling is performed to generate the reconstructed compressed image.

[0024] A lightweight variable bitrate multi-view image compression model includes:

[0025] The main encoder performs downsampling and feature extraction on single-view images to obtain feature maps;

[0026] The feature scaling module is used to scale the feature map to obtain a potential representation of the target bit rate.

[0027] The quantization module is used to perform quantization operations on the latent representation;

[0028] The autoregressive entropy model is used to predict the probability of the next discrete symbol appearing based on a known data sequence. Its prediction results are passed to the arithmetic encoder and arithmetic decoder to achieve efficient compression and decompression.

[0029] Arithmetic encoders are used to perform lossless entropy encoding on the quantized latent representation, encoding it into a compact bit stream to achieve efficient data compression;

[0030] An arithmetic decoder is used to perform lossless entropy decoding on a bitstream obtained by an arithmetic encoder, gradually recovering the latent representation.

[0031] The feature inverse scaling module is used to inversely scale features to avoid additional distortion in the compressed image caused by introducing the feature scaling module;

[0032] The decoder performs feature fusion and upsampling on the latent representations of different views to generate a reconstructed compressed image.

[0033] Preferably, the main encoder includes four convolutional layers arranged sequentially, with a generalized split-normalization layer between adjacent convolutional layers; the number of output channels of the convolutional layer is 192, the kernel size is 5, and the stride is 2.

[0034] Preferably, a masking layer is provided after the generalized splitting is classified into one layer.

[0035] Preferably, the autoregressive entropy model is a Gaussian entropy estimation model.

[0036] The cropping method of the aforementioned lightweight variable bit rate multi-view image compression model includes the following steps: training the lightweight variable bit rate multi-view image compression model by minimizing the sparse loss function until the sparsity of the model matches the number of channels of the target small-scale model; and then fine-tuning the lightweight variable bit rate multi-view image compression model.

[0037] Preferably, the sparse loss function is minimized as follows:

[0038]

[0039] Where x is the mask layer parameter.

[0040] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the aforementioned lightweight variable bit rate multi-view image compression method.

[0041] Compared with the prior art, the beneficial effects of the present invention are:

[0042] (1) The present invention uses a lightweight variable bit rate multi-view image compression algorithm, which can efficiently compress image data, reduce computational complexity, and reduce storage space occupation while preserving image details and maintaining image quality, thus providing faster speed and lower bandwidth requirements for image transmission and storage.

[0043] (2) For real-world distributed application scenarios with limited computing resources: Compared with traditional image compression algorithms and deep learning-based image compression algorithms, this invention uses a sparse loss function to crop the image compression model and then performs fine-tuning training to reduce computational complexity while maintaining maximum compression and reconstruction performance, making the image compression model more suitable for real-world distributed application scenarios with limited computing resources.

[0044] (3) Preserving key details and image quality: The method of the present invention adopts an encoder and decoder architecture, uses lossless entropy encoding and lossless entropy decoding to reconstruct the image, minimizes information loss, and preserves important visual key details in the compressed image, thus maintaining image quality. Attached Figure Description

[0045] Figure 1 This is a flowchart of a lightweight variable bit rate multi-view image compression method according to a specific embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of a lightweight variable bit rate multi-view image compression model according to a specific embodiment of the present invention.

[0047] Figure 3 This is a schematic diagram of the main encoder and decoder of a lightweight variable bit rate multi-view image compression model after adding a masking layer, according to a specific embodiment of the present invention. Detailed Implementation

[0048] To more clearly express the objectives, technical solutions, and advantages of this invention, a detailed description will be provided below in conjunction with the accompanying drawings and embodiments. In describing the embodiments of this application, the following terms and nouns are defined as follows. Providing these definitions ensures a more accurate and clear description of the invention. This helps the reader better understand the technical solutions and advantages of this invention.

[0049] Definitions of some terms and nouns in this application:

[0050] Image compression is a core task in many popular fields such as digital media transmission and storage, drone and satellite imagery, video conferencing, and video streaming. Its goal is to reduce image size while maintaining visual quality. It is widely used in digital image processing, transmission, and storage because image compression can effectively reduce storage space, lower bandwidth requirements, and accelerate image transmission.

[0051] Encoding bitrate: Encoding bitrate is a metric for image compression effectiveness, representing the average number of bits required per pixel. During image compression, the original image undergoes encoding, quantization, and compression steps to reduce redundant information and unnecessary details. Encoding bitrate represents the amount of data occupied by each pixel during image compression. A lower encoding bitrate means higher compression efficiency and smaller file size, but may result in some loss of image quality. Conversely, a higher encoding bitrate retains more detail and accuracy, but produces a larger file size. Choosing an appropriate encoding bitrate requires a trade-off between image quality and storage space. Applications requiring high-fidelity display or precise analysis may require a higher encoding bitrate; while for ordinary viewing or network transmission scenarios, a lower encoding bitrate can be chosen to balance image quality with storage / transmission costs.

[0052] Model pruning is a technique used to reduce the size and computational complexity of deep neural network models by removing redundant connections or parameters and reducing the model's size. In deep neural networks, the training process often generates many unnecessary weights and connections, which may contribute little or nothing to the final performance. The goal of model pruning is to optimize the model by eliminating these redundant parts, reducing the model's size and computational cost while maintaining low loss accuracy. Model pruning has potential benefits in practical applications, such as reducing storage and transmission requirements, improving inference speed, and adapting to resource-constrained environments. However, choosing a suitable pruning strategy requires careful consideration of the balance between the degree of pruning and the performance impact, and thorough validation.

[0053] Peak Signal-to-Noise Ratio (PSNR) is a metric used to evaluate image or video quality. It quantifies the degree of distortion in an image or video by measuring the peak signal-to-noise ratio between the original signal and the signal after compression or distortion. PSNR values ​​are typically expressed in decibels (dB); a higher PSNR indicates better image or video quality because a higher PSNR corresponds to lower distortion. PSNR is commonly used to evaluate image and video compression algorithms, as well as for quality comparison and optimization in image processing and video coding. It helps measure the impact of distortion on human visual perception and assesses the effect of different algorithms or parameter settings on image or video quality.

[0054] Multi-Scale Structural Similarity Index (MS-SSIM) primarily addresses the issue that SSIM (Structural Similarity Index) does not comprehensively consider structural information in image quality assessment. It introduces multi-scale analysis, comparing the structural similarity of images at different scales, thus providing a more comprehensive and accurate evaluation of image quality. MS-SSIM calculates the similarity index by comparing the original image with its Gaussian blurred versions at different scales. MS-SSIM is better adapted to the rich structural information in natural scenes, enabling more accurate evaluation of image quality.

[0055] Floating-point operations (FLOPs) are commonly used to measure the computational complexity and resource consumption of a model. In deep learning, FLOPs represent the sum of all floating-point operations performed across all layers (such as convolutional layers, fully connected layers, etc.) of a neural network. Since different operations (such as addition and multiplication) consume different amounts of hardware resources, FLOPs do not perfectly reflect actual computational requirements, but they can serve as a reference value to compare the computational complexity of different models. Generally, more FLOPs indicate a more complex and computationally resource-intensive model. In practice, FLOPs are often used to evaluate the computational resources required by a model during the training and inference phases.

[0056] The specific embodiments of the present invention are described in detail below:

[0057] Some specific embodiments of the present invention perform image compression on two or more single-view images to obtain a single reconstructed compressed image. This compressed image contains the main features of all the single-view images.

[0058] like Figure 1 As shown, the present invention provides a lightweight variable bit rate multi-view image compression method, comprising the following steps:

[0059] 1) Input the single-view image into the main encoder and then through the feature scaling module to obtain the potential representation of the corresponding target bit rate;

[0060] 2) Quantize the latent representation and perform lossless entropy coding to reduce the amount of data and obtain the final compressed bitstream;

[0061] 3) Perform lossless entropy decoding on the compressed bitstream of the single-view image to recover its latent representation;

[0062] 4) Then input the latent representations of different views into the feature fusion module and upsample them to generate the reconstructed compressed image.

[0063] In step 1), the main encoder includes convolutional layers and generalized divisive normalization (GDN) layers; the main encoder downsamples and extracts features from the input single-view image;

[0064] The feature scaling module performs feature scaling on the feature map generated by the main encoder from the input image to obtain the corresponding potential representation of the target bit rate.

[0065] like Figure 2As shown, the main encoder consists of four convolutional layers with a kernel size of 5 and a stride of 2, and three generalized split-and-normalize layers (GDN). The number of output channels of the convolutional layers is 192. One convolutional layer is followed by one generalized split-and-normalize layer, which is repeated three times, and then another convolutional layer is added to form the main encoder.

[0066] Generalized Split Normalization (GDN) is a normalization technique that normalizes a feature map by dividing each channel by an estimate of its local pixel mean and variance. Specifically, for each channel in the feature map, GDN first calculates the local pixel mean and variance estimates for that channel, and then divides each pixel value of that channel by the corresponding mean and variance. This normalization operation allows the channels of the feature map to have a relatively consistent scale and dynamic range, which is beneficial for network training and optimization. Furthermore, GDN is adaptive, capable of adjusting itself based on the statistical characteristics of the input data, improving the model's robustness to different data distributions.

[0067] The feature scaling module performs feature scaling on the feature map generated by the main encoder of the input image. The feature scaling module uses a variable bit factor (VBF) for feature scaling. By scaling and shifting the corresponding features, the feature scaling module can achieve a level of detail blur in the input image similar to that of JPEG quantization. The specific formula is as follows:

[0068] y′=VBF i (y i ) = A i y i +B i

[0069] Where y' is the feature map of the input image after feature scaling, A i Represents the scaling factor, y i Given the feature map of the input image, B i Indicates the translation factor. Scaling factor A i Translation factor B i It can be adjusted according to actual needs.

[0070] Step 2) above specifically includes:

[0071] 2.1) Quantize the latent representation of a single-view image to reduce the amount of data;

[0072] Specifically, quantization converts continuous feature values ​​into discrete representations to reduce data representation space and storage requirements. Quantization is achieved by mapping the values ​​of a feature map to a set of predefined discrete levels, resulting in a feature map that is a data sequence composed of discrete symbols. Quantization can be divided into three steps:

[0073] (1) Determine the quantization level or resolution. The choice of quantization level determines the degree of discretization of the feature map. A higher quantization level can provide a finer representation, but will generate more data; a lower quantization level can reduce the amount of data, but will lose some detail information. When choosing the quantization level, storage and transmission limitations, as well as the requirements for the quality of the reconstructed image, need to be considered.

[0074] (2) Based on the selected quantization level, establish a quantization table or mapping function. The quantization table maps continuous feature values ​​to discrete quantization levels. The optimal mapping function needs to be designed and optimized according to the evaluation metrics used. Generally, a locally optimal mapping function can be found in practice. The evaluation metrics used to evaluate the model of this invention are Peak Signal-to-Noise Ratio (PSNR) and Multi-Scale Structural Similarity Index (MS-SSIM).

[0075] (3) Map the value of each pixel in the feature map to the corresponding level in the quantization table. This can be achieved by comparing the pixel value in the feature map with the quantization table and finding the closest quantization level. Replace each pixel value in the feature map with the corresponding quantization level to achieve discretization.

[0076] Quantization reduces the representation space and storage requirements of feature maps because discrete quantization levels can be represented with fewer bits. However, quantization also introduces information loss because continuous feature values ​​are mapped to discrete levels. Therefore, a trade-off between compression efficiency and reconstructed image quality must be struck when choosing the quantization level. The optimal quantization level depends on the specific application scenario and data characteristics. In distributed scenarios, the optimal quantization level for quantizing the latent representation of the input image is to achieve the highest possible compression efficiency while maintaining the quality of the reconstructed image.

[0077] 2.2) The arithmetic encoder performs lossless entropy encoding on the quantized latent representation based on the Gaussian entropy estimation model to obtain the final compressed bitstream.

[0078] Specifically, lossless entropy coding is a lossless data compression technique used to further reduce the size of encoded data. By utilizing the statistical properties and probabilistic models in the data, common patterns and symbols are represented as shorter coded byte sequences. A Gaussian entropy estimation model is used to describe the probability of occurrence of different symbols in the feature map, and then the symbols in the feature map are mapped to the entropy coding dictionary based on the Gaussian entropy estimation model. Each symbol in the coding dictionary corresponds to a short coding sequence; common symbols correspond to shorter codes, while uncommon symbols correspond to longer codes. By using the probability information in the probabilistic model, more frequently occurring symbols can use shorter codes, thus achieving higher compression efficiency. Next, the quantized feature map is traversed, replacing each symbol with its corresponding short code. The code generation process can be achieved by looking up the coding dictionary or using dynamic coding with a probabilistic model. Finally, the generated lossless entropy coded data is output as the final compressed coded sequence. Lossless entropy coding typically generates shorter coded data, further reducing the representation space and storage requirements of the feature map.

[0079] The final compressed bitstream is obtained through quantization and lossless entropy coding, which reduces the amount of image data while preserving image features.

[0080] Step 3) above performs lossless entropy decoding on the compressed bitstream of the single-view image to recover the latent representation, specifically including:

[0081] Lossless entropy decoding is the process of decoding lossless entropy encoded data back to the original data. It uses the same probabilistic model and decoding algorithm as lossless entropy encoding to convert the entropy-encoded data back into a quantized feature map. During decoding, a probabilistic model is used to reconstruct the state information from the encoding process, reading each encoded byte from the entropy-encoded data. After decoding, the resulting feature map is the final decoding result. This feature map should be identical to the quantized feature map before encoding, achieving lossless data decoding. Then, the decoded feature map is inversely scaled to avoid introducing additional distortion to the compressed image due to the feature scaling module. The inverse scaling is performed based on the inverse function of the scaling function.

[0082] Reconstructing images using lossless entropy encoding and decoding can minimize information loss, preserve important visual details in the compressed image, and maintain image quality.

[0083] Each single-view image is processed through steps 1) to 3) to obtain the potential representation of all single-view images after reconstruction.

[0084] Step 4 above specifically includes:

[0085] 4.1) Input the restored latent representations of different views into the first feature fusion module, so that the latent representations between views can learn relevant information from other views, and then perform upsampling;

[0086] Specifically, the latent representation of the view recovered by lossless entropy decoding is processed by the first feature fusion module. The module uses the feature information existing in multiple different single views to learn relevant information from other views through fusion. Then, it is upsampled to reconstruct the features of the latent representation in order to maintain image quality.

[0087] 4.2) After upsampling, the latent representations reconstructed from all single views are input into the second feature fusion module to further learn relevant information from other views and integrate image features from the latent representations of each view;

[0088] 4.3) Then perform upsampling to generate the reconstructed compressed image.

[0089] Specifically, the upsampled latent representation of the view is input into the feature fusion module, which utilizes different feature information from multiple different views to enable the views to learn relevant information, and then performs upsampling to reconstruct the compressed image.

[0090] Based on the above methods, such as Figure 2 As shown, a lightweight variable bit rate multi-view image compression model of the present invention includes:

[0091] The main encoder performs downsampling and feature extraction on single-view images to obtain feature maps;

[0092] The feature scaling module is used to scale the feature map to obtain a potential representation of the target bit rate.

[0093] A quantization module is used to quantize the latent representation;

[0094] The autoregressive entropy model is used to predict the probability of the next discrete symbol appearing based on a known data sequence. The prediction results are passed to the arithmetic encoder and arithmetic decoder to achieve efficient compression and decompression.

[0095] Arithmetic encoders are used to perform lossless entropy encoding on the quantized latent representation, encoding it into a compact bit stream to achieve efficient data compression;

[0096] An arithmetic decoder is used to perform lossless entropy decoding on a bitstream obtained by an arithmetic encoder, gradually recovering the latent representation.

[0097] The feature inverse scaling module is used to inversely scale features to avoid additional distortion in the compressed image caused by introducing the feature scaling module;

[0098] The decoder performs feature fusion and upsampling on the latent representations of different views to generate a reconstructed compressed image.

[0099] The main encoder consists of four convolutional layers with a kernel size of 5 and a stride of 2, and three generalized split-normalization layers (GDNs). The number of output channels of the convolutional layers is 192. One convolutional layer is followed by one GDN, and this process is repeated three times, followed by another convolutional layer to form the main encoder.

[0100] The above autoregressive entropy model is a Gaussian entropy estimation model.

[0101] The decoder consists of two feature fusion modules, four deconvolutional layers, and three inverse generalized split-and-revert layers (IGDN). An inverse generalized split-and-revert layer is set between every two deconvolutional layers. The first feature fusion module is located before the first deconvolutional layer, and the second feature fusion module is located between the second inverse generalized split-and-revert layer and the third deconvolutional layer.

[0102] The aforementioned lightweight variable bit rate multi-view image compression model needs to be pruned before being used in the aforementioned multi-view image compression method. This is to effectively transform the complex large-scale compression model into a simpler and smaller model while maintaining maximum compression and reconstruction performance.

[0103] Specifically, such as Figure 3 As shown, this invention introduces a mask layer into the aforementioned lightweight variable bitrate multi-view image compression model to identify and label the weights that contribute the least to model performance during model pre-training. These weights typically correspond to channels in the model suitable for cropping. Each mask layer consists of a set of trainable parameters, all initially set to 1. During the forward propagation of the model, the mask layer parameters are multiplied by the input features channel by channel to adjust the importance of different channels in the model. The larger the mask layer parameters, the greater the contribution of the corresponding channel to model performance; conversely, the smaller the parameters, the smaller the contribution. Mask layer parameters close to 0 correspond to channels marked as candidate channels requiring cropping. Meanwhile, in order to make full use of the normalization function provided by the GDN layer, the masking layer is added after the GDN and IGDN layers, instead of after the convolutional and deconvolutional layers; then the enhanced lightweight variable bit rate multi-view image compression model is trained by minimizing the sparse loss function, so that the masking layer parameters have more weights close to zero during training, until the sparsity of the model matches the number of channels of the target small-scale model, at which point training stops.

[0104] The sparse loss function used in this invention is concave within the weight range of 0 to 1, imposing a larger penalty on smaller weights, thus tending to bring the weight parameters closer to 0; it imposes a smaller penalty on weights close to 1 to preserve key information in the image compression model of this invention; when the weight is greater than 1, the sparse loss function becomes convex, resulting in some attenuation of larger weights, but due to the presence of linear and constant terms, the attenuation is limited, maintaining the stability of the model to a certain extent. The formula for the above sparse loss function is as follows:

[0105]

[0106] Where Lsparse is the sparse loss function; x is the mask layer parameter.

[0107] Based on the sparse mask layer parameters, this invention merges the mask layer and the original convolutional layer, thereby pruning channels marked as non-critical and unimportant and obtaining the corresponding parameters of a small-scale model. Then, in order to compensate for the performance degradation caused by pruning the model, this invention performs fine-tuning training on the model to ensure that the model maintains its performance in compressing images while compressing its size.

[0108] Table 1. Comparison of Variable Bit Rate Multi-View Compression Algorithm with Existing Cutting-Edge Technologies

[0109] Image compression algorithm PSNR / dB MS-SSIM Model parameters / M FLOPs / M NDIC 36.47 0.98 15.74 58982.40 DSIC 36.60 0.98 17.13 641469.10 VBMIC 38.44 0.99 2.03 9518.51

[0110] Typically, PSNR and MS-SSIM are used to evaluate the performance of image compression models. A higher PSNR value indicates a higher similarity between the reconstructed image and the original image, meaning a better quality reconstructed image. MS-SSIM, on the other hand, introduces multi-scale analysis to compare the structural similarity of images at different scales, thus providing a more comprehensive and accurate assessment of image quality.

[0111] Existing cutting-edge image compression research or technologies, such as Neural Distributed Image Compression (NDIC) and Deep Stereo Image Compression (DSIC), while achieving good compression performance for fixed bitrate images, cannot modify the compressed image bitrate according to requirements or distributed scenarios. Their model parameters and computational complexity are also relatively high, making them unsuitable for real-world distributed scenarios with limited computing resources. In contrast, the Variable Bitrate Multi-view Image Compression (VBMIC) algorithm designed in this invention achieves comparable image compression performance. Furthermore, it allows for variation of the compressed image bitrate according to specific tasks or requirements, and its model parameters and computational complexity are reduced after model cropping, making it more adaptable to real-world distributed scenarios with limited computing resources. A comparison of the Variable Bitrate Multi-view Image Compression algorithm with the Neural Distributed Image Compression (NDIC) and Deep Stereo Image Compression (DSIC) algorithms is shown in Table 1.

[0112] As shown in Table 1, the PSNR and MS-SSIM values ​​of the lightweight variable bit rate multi-view image compression model of this invention are higher than those of DSIC and NDIC, indicating that the compressed image quality obtained by the lightweight variable bit rate multi-view image compression model of this invention is higher. The model parameter values ​​and FLOPs values ​​of the lightweight variable bit rate multi-view image compression model of this invention are significantly lower than those of DSIC and NDIC, indicating that the lightweight variable bit rate multi-view image compression model of this invention has low computational complexity, small storage space occupation, low computational resource consumption, and wide applicability.

[0113] Some specific embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described lightweight variable bit rate multi-view image compression method.

[0114] Some specific embodiments of the present invention provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described lightweight variable bit rate multi-view image compression method.

Claims

1. A lightweight variable bit rate multi-view image compression method, characterized in that: Includes the following steps: 1) A feature map is obtained by downsampling and feature extraction of a single-view image, and then the feature map is scaled to obtain a potential representation of the corresponding target bit rate; 2) Quantize and lossless entropy encode the latent representation to obtain the final compressed bitstream; 3) Perform lossless entropy decoding and inverse scaling on the final compressed bitstream to recover the latent representation; 4) The restored latent representations from different views are fused and upsampled to generate a reconstructed compressed image; The downsampling and feature extraction are implemented through a main encoder, which includes a convolutional layer, a generalized split normalization layer, and a masking layer; the masking layer is located after the generalized split normalization layer. The masking layer is trained by minimizing a sparse loss function until the sparsity of the model matches the number of channels of the target small-scale model; the minimization of the sparse loss function is as follows: ； Where Lsparse is the sparse loss function; x is the masking layer parameter.

2. The lightweight variable bit rate multi-view image compression method according to claim 1, characterized in that: In step 1), the feature scaling uses the following feature scaling function: in, This is the feature map of the input image after feature scaling. Indicates the scaling factor. The feature map of the input image, This represents the translation factor.

3. The lightweight variable bit rate multi-view image compression method according to claim 1, characterized in that: Step 4) includes the following steps: 4.1) Perform the first feature fusion on the latent representations after the recovery of different views, so that the latent representations of each view can learn relevant information from other views, and then perform upsampling; 4.2) Perform a second feature fusion on the latent representations of all views after upsampling in step 4.1) to further learn relevant information from other views and integrate image features from the latent representations of each view; 4.3) After the second feature fusion, upsampling is performed to generate the reconstructed compressed image.

4. A lightweight variable bit rate multi-view image compression device, characterized in that: include: The main encoder performs downsampling and feature extraction on single-view images to obtain feature maps; The main encoder includes a convolutional layer, a generalized split normalization layer, and a masking layer; The masking layer is located after the generalized split normalization layer; the masking layer is trained by minimizing a sparse loss function until the sparsity of the model matches the number of channels of the target small-scale model; the minimization of the sparse loss function is as follows: ； Where Lsparse is the sparse loss function; x is the masking layer parameter; The feature scaling module is used to scale the feature map to obtain a potential representation of the target bit rate. The quantization module is used to perform quantization operations on the latent representation; The autoregressive entropy model is used to predict the probability of the next discrete symbol appearing based on a known data sequence. Its prediction results are passed to the arithmetic encoder and arithmetic decoder to achieve efficient compression and decompression. Arithmetic encoders are used to perform lossless entropy encoding on the quantized latent representation, encoding it into a compact bit stream to achieve efficient data compression; An arithmetic decoder is used to perform lossless entropy decoding on a bitstream obtained by an arithmetic encoder, gradually recovering the latent representation. The feature inverse scaling module is used to inversely scale features to avoid additional distortion in the compressed image caused by introducing the feature scaling module; The decoder performs feature fusion and upsampling on the latent representations of different views to generate a reconstructed compressed image.

5. The lightweight variable bit rate multi-view image compression device according to claim 4, characterized in that: The main encoder includes four convolutional layers arranged in sequence, with a generalized split-level layer between adjacent convolutional layers; the number of output channels of the convolutional layer is 192, the kernel size is 5, and the stride is 2.

6. The lightweight variable bit rate multi-view image compression device according to claim 4, characterized in that: The autoregressive entropy model is a Gaussian entropy estimation model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the lightweight variable bit rate multi-view image compression method as described in any one of claims 1 to 3.

Images