Variable Bit Rate Image Compression Method Based on Feature Symmetric Transformation

Abstract

The present application relates to the field of image processing technology, and discloses a variable bit rate image compression method, device, computer equipment and storage medium based on feature symmetric transformation. The method expands an SRDB module into a twin TRDB module as a feature extraction module, and uses TRDB to extract features from a perceptual quality map at a decoding end to obtain conditional features; different conditional features are used in the codec to perform different affine transformations on a source image to achieve variable bit rate image compression and reconstruction based on feature symmetric transformation; by using a perceptual quality map at a decoding end constructed by the same continuous control factor at the codec end, the TRDB module obtains the same input during the coding and decoding process, and the TRDB module performs the same feature extraction operation on the perceptual quality map at the coding end and the decoding end, thereby ensuring the symmetry of feature extraction during coding and decoding, and helping to improve the performance of the compression model and the quality of the reconstructed image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a variable bit rate image compression method, device, computer equipment and storage medium based on feature symmetry transformation. Background Art

[0002] In modern society, image data is widely used in various fields such as communication, storage, transmission and display. High-resolution and high-quality images often take up a lot of storage space and bandwidth resources, resulting in increased data transmission and storage costs. However, resources such as storage space and network bandwidth are still limited. Therefore, the research and application of image compression technology has become an urgent need.

[0003] In recent years, lossy image compression methods based on deep learning have been proven to be superior to traditional image compression methods. However, most of these methods can usually only achieve fixed bitrate compression. Fixed bitrate image compression models need to train multiple models to achieve different bitrate compression, which increases the storage burden. When key resources such as computing resources and network bandwidth are still limited, fixed compression rate methods often cannot meet the flexible needs of different scenarios. These methods may cause serious degradation of image quality at high compression rates, and cannot be implemented in limited network resources at low compression rates. Variable bitrate image compression based on feature symmetry transformation can solve this problem. Variable bitrate image compression based on feature symmetry transformation means that different bitrate compression can be achieved by training one model, which can maximize the use of limited resources. Single model variable bitrate compression allows the compression rate to be flexibly adjusted in different transmission scenarios to maximize the use of limited resources. When the bandwidth is sufficient, the model can choose a lower compression rate to maintain the high quality of the image. In the environment of limited network bandwidth, the model can intelligently choose a higher compression rate to ensure that data can flow efficiently in real-time transmission. Therefore, how to fully and flexibly utilize limited bandwidth to achieve high-performance image compression has become one of the technical problems that need to be solved urgently.

[0004] Although deep learning models have achieved impressive performance on many tasks, these models usually have huge number of parameters and computational complexity. This poses challenges to the storage, transmission speed and inference speed of the models, especially in resource-constrained devices and environments. Therefore, how to reduce the complexity of the model and improve the transmission and inference speed has become another technical problem that needs to be solved urgently. Summary of the invention

[0005] The present application provides a variable bit rate image compression method, device, computer equipment and storage medium based on feature symmetry transformation to improve the compression quality of the image.

[0006] In a first aspect, the present application provides a variable bit rate image compression method based on feature symmetry transformation, the method comprising:

[0007] Based on the continuous control factor, determining a decoder-side perceptible quality map corresponding to the source image;

[0008] Constructing a symmetric rate-distortion balance control SRDB module, and expanding the SRDB module into a twin symmetric rate-distortion balance control TRDB module;

[0009] At least one conditional feature of the perceptible quality map at the decoding end is extracted through the TRDB module, and an affine transformation is performed on the source image through each of the conditional features to achieve image compression of the source image.

[0010] Further, based on the continuous control factor, determining a decoder-end perceptible quality map corresponding to the source image includes:

[0011] Generate an initial tensor according to the source image;

[0012] Filling each of the continuous control factors into each pixel point of the initial tensor to generate an initial uniform quality map;

[0013] The decoding end perceptible quality map is generated according to each of the initial uniform quality maps and all of the continuous control factors.

[0014] Further, extracting at least one conditional feature of the perceptible quality map of the decoding end through the TRDB module includes:

[0015] Taking the decoder perceptual quality map as input, and using the same decoder perceptual quality map as input to the TRDB module at the encoder and the decoder;

[0016] A downsampling convolution operation is performed on the decoder-end perceptible quality map to extract features of the decoder-end perceptible quality map to generate the conditional features.

[0017] Furthermore, performing affine transformation on the source image through each of the conditional features to achieve image compression of the source image includes:

[0018] Determine affine parameters based on the conditional features, and perform affine transformation on the source image through an affine transformation formula and the conditional features;

[0019] Among them, the affine transformation formula is: , , is the conditional characteristic, is the preset mapping function, is the affine parameter, is the feature map of the source image.

[0020] Furthermore, before determining the affine parameters based on the conditional features and performing affine transformation on the source image using an affine transformation formula and the conditional features, the method includes:

[0021] The features of the decoder-end perceptible quality map are extracted to generate a feature map of the source image.

[0022] Furthermore, a symmetric rate-distortion balance control SRDB module is constructed, and the SRDB module is expanded into a twin symmetric rate-distortion balance control TRDB module, including:

[0023] Based on a preset parameter sharing method, the SRDB module is expanded into the TRDB module.

[0024] In a second aspect, the present application further provides a variable bit rate image compression device based on feature symmetry transformation, the device comprising:

[0025] A decoding end perceptible quality map determining module, used for determining a decoding end perceptible quality map corresponding to a source image based on a continuous control factor;

[0026] A TRDB module determination module is used to construct a symmetric rate-distortion balance control SRDB module, and expand the SRDB module into a twin symmetric rate-distortion balance control TRDB module;

[0027] A compression and reconstruction module is used to extract at least one conditional feature of the perceptible quality map of the decoding end through the TRDB module, and perform affine transformation on the source image through each of the conditional features to achieve image compression of the source image.

[0028] Furthermore, the decoding end may have a perceptible quality map determination module, including:

[0029] An initial tensor generating unit, used for generating an initial tensor according to the source image;

[0030] An initial uniform quality map generating unit, used for filling each of the continuous control factors into each pixel point of the initial tensor to generate an initial uniform quality map;

[0031] The decoding end perceptible quality map determining unit is used to generate the decoding end perceptible quality map according to each of the initial uniform quality maps and all of the continuous control factors.

[0032] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the variable bit rate image compression method based on feature symmetry transformation as described above when executing the computer program.

[0033] In a fourth aspect, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the variable bit rate image compression method based on feature symmetry transformation as described above.

[0034] The present application discloses a variable bit rate image compression method, apparatus, computer equipment and storage medium based on feature symmetric transformation. The variable bit rate image compression method based on feature symmetric transformation includes determining a decoding end perceptible quality map corresponding to a source image based on a continuous control factor; constructing a symmetric rate distortion balance control SRDB module, and expanding the SRDB module into a twin symmetric rate distortion balance control TRDB module; extracting at least one conditional feature of the decoding end perceptible quality map through the TRDB module, and performing an affine transformation on the source image through each of the conditional features to achieve image compression of the source image. Through the above method, the present application expands the SRDB module into a twin symmetric rate-distortion balance control TRDB module as a feature extraction module, and uses TRDB to extract features of the perceptible quality map at the decoding end to obtain conditional features; the SFT layer in the codec uses different conditional features to perform different affine transformations on the source image to achieve variable bit rate image compression and reconstruction based on feature symmetric transformation; by using a quality map constructed by the same continuous control factor at the codec end, that is, the perceptible quality map at the decoding end, it can be ensured that the feature extraction module obtains the same input during the encoding and decoding process, and the TRDB module performs the same feature extraction operation on the perceptible quality map at the encoding and decoding ends, ensuring the symmetry of feature extraction during encoding and decoding, which helps to improve the performance of the compression model and the quality of the reconstructed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 is a schematic flow chart of a variable bit rate image compression method based on feature symmetry transformation provided by an embodiment of the present application;

[0037] Figure 2 A schematic diagram of a network framework of a variable bit rate image compression method based on feature symmetry transformation provided in an embodiment of the present application;

[0038] Figure 3 A simplified network framework diagram of a variable bit rate image compression method based on feature symmetry transformation provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of the internal structure of an SFT layer of a variable bit rate image compression method based on feature symmetry transformation provided in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of a lightweight model of a variable bit rate image compression method based on feature symmetry transformation provided in an embodiment of the present application;

[0041] Figure 6 A schematic block diagram of a variable bit rate image compression device based on feature symmetric transformation provided in an embodiment of the present application;

[0042] Figure 7 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0045] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0046] It should also be understood that the term “and / or” used in the specification and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0047] The embodiments of the present application provide a variable bit rate image compression method, device, computer equipment and storage medium based on feature symmetry transformation. Among them, the variable bit rate image compression method based on feature symmetry transformation can be applied to a server. By using a continuous control factor, the perceptible quality map at the decoding end can more accurately reflect the importance of different areas in the image to human vision, thereby retaining more visual details during the image compression process. The SRDB module performs the same feature extraction operation on the perceptible quality map at the encoding end and the decoding end, ensuring the symmetry of feature extraction during encoding and decoding, which helps to improve the reconstruction quality of the compressed image and improves the compression quality of the image. Among them, the server can be an independent server or a server cluster.

[0048] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0049] See also Figure 1 , Figure 1 This is a schematic flow chart of a variable bit rate image compression method based on feature symmetry transformation provided by an embodiment of the present application. The variable bit rate image compression method based on feature symmetry transformation can be applied to a server, and is used to expand the SRDB module into a twin symmetric rate distortion balance control TRDB module, as a feature extraction module, and uses TRDB to extract features from the perceptible quality map at the decoding end to obtain conditional features; the SFT layer in the codec uses different conditional features to perform different affine transformations on the source image to achieve variable bit rate image compression and reconstruction based on feature symmetry transformation; by using a quality map constructed by the same continuous control factor at the codec end, that is, the perceptible quality map at the decoding end, it can be ensured that the feature extraction module obtains the same input during the encoding and decoding process, and the TRDB module performs the same feature extraction operation on the perceptible quality map at the encoding end and the decoding end, ensuring the symmetry of feature extraction during encoding and decoding, which helps to improve the performance of the compression model and the quality of the reconstructed image.

[0050] like Figure 1 As shown, the variable bit rate image compression method based on feature symmetric transformation specifically includes steps S10 to S30.

[0051] Step S10: determining a decoding end perceptible quality map corresponding to the source image based on the continuous control factor;

[0052] In one embodiment, the quality map is used only as conditional information for the encoding process, and an estimated map w of the quality map is used as conditional information for the decoding process. w is the output of the super-prior network obtained by a series of transformations of the upsampling network. This means that the conditional information used in the encoding process and the decoding process is different, resulting in the inability of the Spatially Adaptive Feature Transform (SFT) layer in the codec to perform a fully reversible affine transformation, resulting in a loss in the quality of the reconstructed image. To solve this problem, this embodiment uses the same conditional information in the encoding and decoding processes, and refers to this conditional information as the perceptible quality map m at the decoding end.

[0053] In addition, one of the reasons for image blur is that the contour in the image is not obvious, the grayscale change of the contour edge is not strong, and the layering is not strong. The grayscale change of the object contour edge in the clear picture is obvious, and the layering is strong. In order to better restore the contour and texture details in the image, the conditional information used in this embodiment is a gradient-guided non-uniform quality map.

[0054] Step S20: construct a symmetric rate-distortion balance control SRDB module, and expand the SRDB module into a twin symmetric rate-distortion balance control TRDB module;

[0055] Step S30: extract at least one conditional feature of the perceptible quality map at the decoding end through the TRDB module, and perform affine transformation on the source image through each of the conditional features to achieve image compression of the source image.

[0056] Specifically, Figure 2 As shown, Figure 2 A schematic diagram of the network framework of a variable bit rate image compression method based on feature symmetry transformation provided in an embodiment of the present application. The perceptible quality map m at the decoding end and the source image x are respectively input into the twin symmetry rate distortion balance control module and the encoder. The twin symmetry rate distortion balance control module extracts features from m, and the extracted multi-scale features are input into the spatially adaptive feature transform (SFT) layer in the encoder. The SFT layer performs an affine transformation on the features of the input image using the multi-scale features as conditional features, and obtains the potential representation y after encoding, which is then quantized by the quantizer. Since scalar quantization is not differentiable during back propagation, this embodiment uses additive uniform noise instead of scalar quantization during training. After quantization, a Gaussian entropy model is used for bit rate estimation. During decoding, the potential representation obtained after entropy encoding and entropy decoding and the perceptible quality map m at the decoding end are respectively input into the decoder and the twin symmetry rate distortion balance control module. The SFT layer in the decoder performs an affine transformation using the multi-scale features output by the twin symmetry rate distortion balance control module as conditional features, and finally obtains the reconstructed image. In addition, this embodiment also introduces a super-prior module to eliminate the spatial structural redundancy of the potential representation, and splices the potential representation output by the encoder and the perceptible quality map m at the decoding end and inputs them into the super-prior network. The output of the super-prior network guides the entropy encoding and entropy decoding of the potential representation, thereby achieving better bit rate estimation.

[0057] like Figure 3 As shown, Figure 3 A simplified network framework diagram of a variable bit rate image compression method based on feature symmetric transformation provided in an embodiment of the present application. In order to achieve variable bit rate compression, this embodiment adds some spatially adaptive feature transform (SFT) layers in the codec, such as Figure 2 As shown in Figure 1, there are five "Conv-GDN-SFT" blocks in each codec. Figure 2 In the decoder, the "Conv-GDN-SFT" blocks are the convolutional layers, normalization layers, and spatial feature transforms of the encoder and decoder. Specifically, the five "Conv-GDN-SFT" blocks in the encoder are convolutional layer 48-normalization layer-spatial feature transform, convolutional layer 196↓2-normalization layer-spatial feature transform, convolutional layer 192↓2-normalization layer-spatial feature transform, convolutional layer 192↓2-normalization layer-spatial feature transform, and convolutional layer 192↓2-normalization layer-spatial feature transform. The five "Conv-GDN-SFT" blocks in the decoder are spatial feature transform-normalization layer-convolutional layer 48↑2, spatial feature transform-normalization layer-convolutional layer 96↑2, spatial feature transform-normalization layer-convolutional layer 192↑2, spatial feature transform-normalization layer-convolutional layer 192↑2, and spatial feature transform-spatial feature transform. In addition, the decoder is different from the encoder in that it has a separate convolutional layer-3. The internal structure of the SFT layer is as follows: Figure 4 As shown, Figure 4 The schematic diagram of the internal structure of the SFT layer of a variable bit rate image compression method based on feature symmetric transformation provided in the embodiment of the present application is to perform an affine transformation on the feature map of the source image according to the conditional features. The formula of the affine transformation is as follows:

[0058] ;

[0059]

[0060] in, It is the conditional feature, that is, the feature obtained after feature extraction of conditional information. is the mapping function learned by the SFT layer, that is Figure 4 The convolution operation in . is the affine parameter learned by the SFT layer, For source image The feature map of is the multiplication of the corresponding elements of the matrix, For feature splicing, It is the feature map output by the SFT layer. Different conditional features can be obtained by constructing different conditional information, so that different affine transformations can be performed on the source image features during encoding to obtain different potential representations and achieve different bit rate compression.

[0061] The present embodiment discloses a variable bit rate image compression method, apparatus, computer equipment and storage medium based on feature symmetry transformation. The variable bit rate image compression method based on feature symmetry transformation includes determining a decoding end perceptible quality map corresponding to a source image based on a continuous control factor; constructing a symmetric rate distortion balance control SRDB module, and expanding the SRDB module into a twin symmetric rate distortion balance control TRDB module; extracting at least one conditional feature of the decoding end perceptible quality map through the TRDB module, and performing an affine transformation on the source image through each of the conditional features to achieve image compression of the source image. Through the above method, the present application expands the SRDB module into a twin symmetric rate-distortion balance control TRDB module as a feature extraction module, and uses TRDB to extract features of the perceptible quality map at the decoding end to obtain conditional features; the SFT layer in the codec uses different conditional features to perform different affine transformations on the source image to achieve variable bit rate image compression and reconstruction based on feature symmetric transformation; by using a quality map constructed by the same continuous control factor at the codec end, that is, the perceptible quality map at the decoding end, it can be ensured that the feature extraction module obtains the same input during the encoding and decoding process, and the TRDB module performs the same feature extraction operation on the perceptible quality map at the encoding and decoding ends, ensuring the symmetry of feature extraction during encoding and decoding, which helps to improve the performance of the compression model and the quality of the reconstructed image.

[0062] based on Figure 1 In the embodiment shown, in this embodiment, step S10 includes:

[0063] Generate an initial tensor according to the source image;

[0064] Filling each of the continuous control factors into each pixel point of the initial tensor to generate an initial uniform quality map;

[0065] The decoding end perceptible quality map is generated according to each of the initial uniform quality maps and all of the continuous control factors.

[0066] based on Figure 1 In the embodiment shown, in this embodiment, step S30 includes:

[0067] Taking the decoder perceptual quality map as input, and using the same decoder perceptual quality map as input to the TRDB module at the encoder and the decoder;

[0068] A downsampling convolution operation is performed on the decoder-end perceptible quality map to extract features of the decoder-end perceptible quality map to generate the conditional features.

[0069] Specifically, since the above variable bit rate image compression model based on feature symmetric transformation is relatively complex, this embodiment uses knowledge distillation and parameter sharing to reduce its complexity and realize a lightweight model. Figure 5 As shown, Figure 5 A schematic diagram of a lightweight model of a variable bit rate image compression method based on feature symmetric transformation provided in an embodiment of the present application. The lightweight model in this embodiment can be independently selected according to the performance of the device.

[0070] Specifically, the symmetric rate-distortion balance control block (Symmetric Rate-distortion Balance, SRDB) is first expanded into a twin symmetric rate-distortion balance control module (Twin Symmetric Rate-distortion Balance, TRDB) through parameter sharing technology. Parameter sharing can not only reduce the amount of model parameters, but also make the perceptible quality map of the decoder input to the feature map of the SFT layer through the twin symmetric rate-distortion balance control module completely consistent in content and structure, thereby improving the accuracy of image reconstruction. In addition, in this embodiment, each module of the variable bit rate image compression model based on feature symmetric transformation based on TRDB is reduced by 40% to construct a lightweight model. In order to achieve lightweight while ensuring model performance, this embodiment uses the model based on TRDB as a teacher model and uses knowledge distillation technology to train the constructed lightweight student model. Specifically, this embodiment uses the features of the penultimate layer of the encoder and decoder of the teacher model to supervise the training of the penultimate layer of the encoder and decoder of the student model. This embodiment uses Euclidean distance to represent the distillation loss:

[0071]

[0072] in, represents the features of the penultimate layer of the teacher model encoder, represents the features of the penultimate layer of the student model encoder, represents the features of the penultimate layer of the teacher model decoder, Represents the features of the penultimate layer of the student model decoder.

[0073] Based on any of the above embodiments, in this embodiment, a rate-distortion function based on different bit rates is also selected as a loss function.

[0074] Specifically, the rate-distortion function is ,in, It is a different The vector composed of , Refers to the gradient-guided non-uniform quality map. is a predefined monotonically increasing function, .

[0075] This embodiment uses the Gaussian entropy model provided by the hyper-prior model to perform bit rate estimation, selects the MSE loss as the distortion term, and the bit rate estimation formula is as follows:

[0076]

[0077] Among them, the first item is the bit rate of the quantized super-prior encoder output z^, and the second item is the bit rate of the quantized potential representation y^. As for the specific bit rate estimation method, this embodiment uses the bit rate estimation method used in the image compression method with scale super-prior proposed in the super-prior model.

[0078] For the teacher model, i.e., the variable bitrate image compression model based on feature symmetric transformation based on the TRDB module, the training loss function is as follows:

[0079]

[0080] For the student model, the training loss function is as follows:

[0081]

[0082] Among them, the first three items are the same as the teacher model, and the last two items represent the distillation loss measured by Euclidean distance.

[0083] represents the features of the penultimate layer of the teacher model encoder, represents the features of the penultimate layer of the student model encoder, represents the features of the penultimate layer of the teacher model decoder, Represents the features of the penultimate layer of the student model decoder.

[0084] In a specific embodiment, performing affine transformation on the source image through each of the conditional features to achieve image compression of the source image includes:

[0085] Determine affine parameters based on the conditional features, and perform affine transformation on the source image through an affine transformation formula and the conditional features;

[0086] Among them, the affine transformation formula is: , , is the conditional characteristic, is the preset mapping function, is the affine parameter, is the feature map of the source image.

[0087] Specifically, It is the conditional feature, that is, the feature obtained after feature extraction of conditional information. is the mapping function learned by the SFT layer, that is Figure 4 The convolution operation in . is the affine parameter learned by the SFT layer, For source image The feature map of is the multiplication of the corresponding elements of the matrix, For feature splicing, It is the feature map output by the SFT layer. Different conditional features can be obtained by constructing different conditional information, so that different affine transformations can be performed on the source image features during encoding to obtain different potential representations and achieve different bit rate compression.

[0088] Considering that the affine transformation performed by SFT during encoding and decoding should be reversible, that is, the conditional information used in the encoding and decoding processes should be the same, the conditional features obtained by the SFT layer during the encoding and decoding processes are the same, and thus the learned affine parameters are the same. Therefore, during decoding, the conditional information is the same gradient-guided non-uniform quality map transmitted from the encoding end, which is referred to as the perceptible quality map at the decoding end in this embodiment. In order to To perform the same feature extraction operation, this embodiment constructs a symmetric rate-distortion balance control block (SRDB) as a feature extraction module. Figure 4As shown in the figure, the SRDB module consists of two parts, one at the encoder and one at the decoder, and the structures of the two parts are exactly the same, both of which perform a series of downsampling operations on the perceptual quality map m at the decoder. The SRDB at the encoder and decoder is composed of a series of convolutional layers and activation layers, which extract multi-scale features from the input perceptual quality map m at the decoder. The SRDB at the encoder and decoder is divided into five modules, corresponding to the five "Conv-GDN-SFT" blocks in the codec. The output of each module is input to the SFT layer of each "Conv-GDN-SFT" block in the codec, and the dimension of the output of each module must be consistent with the dimension of the source image feature input to the corresponding SFT layer. The output of each module in SRDB is input to the corresponding SFT layer in the codec as a conditional feature, guiding each SFT layer to perform an affine transformation on the current input source image feature. The feature extraction network with the same structure at the encoder and decoder enables the SFT layer to use almost the same conditional features during the encoding and decoding process, thereby realizing a reversible affine transformation.

[0089] Since the SRDB modules at the codec end are only the same in structure, the network parameters cannot be guaranteed to be the same, that is, the conditional features input to the SFT layer of the codec cannot be guaranteed to be completely consistent. In order to ensure that the features of the perceptible quality map at the decoder end extracted during the encoding and decoding process are completely consistent in content and structure, this embodiment uses parameter sharing to expand the SRDB module into a twin symmetric code twin symmetric rate-distortion balance control module (Twin Symmetric Rate-distortion Balance, TRDB), that is, the TRDB module at the codec end has the same internal structure (same as SRDB) and the same internal parameters. When TRDB is used to extract features of the perceptible quality map at the decoder end, it can be ensured that the output features of each block in each TRDB during the encoding process and the decoding process are the same in content and structure, that is, the conditional features of the corresponding SFT layer of each block in the TRDB input to the codec are the same in content and structure. In this way, the corresponding SFT layer can achieve a completely inverse affine transformation during the encoding and decoding process, thereby improving the quality of the reconstructed image.

[0090] In a specific embodiment, before determining the affine parameters based on the conditional features and performing affine transformation on the source image using the affine transformation formula and the conditional features, the method includes:

[0091] The features of the decoder-end perceptible quality map are extracted to generate a feature map of the source image.

[0092] In a specific embodiment, based on Figure 1 In the embodiment shown, in this embodiment, step S30 further includes:

[0093] Based on a preset parameter sharing method, the SRDB module is expanded into the TRDB module.

[0094] See also Figure 6 , Figure 6 The embodiment of the present application provides a schematic block diagram of a variable bit rate image compression device based on feature symmetry transformation, and the variable bit rate image compression device based on feature symmetry transformation is used to execute the aforementioned variable bit rate image compression method based on feature symmetry transformation. The variable bit rate image compression device based on feature symmetry transformation can be configured on a server.

[0095] like Figure 6 As shown, the variable bit rate image compression device 400 based on feature symmetric transformation includes:

[0096] A decoding end perceptible quality map determining module 410, configured to determine a decoding end perceptible quality map corresponding to a source image based on a continuous control factor;

[0097] A TRDB module determination module 420 is used to construct a symmetric rate-distortion balance control SRDB module and expand the SRDB module into a twin symmetric rate-distortion balance control TRDB module;

[0098] The compression and reconstruction module 430 is used to extract at least one conditional feature of the perceptible quality map at the decoding end through the TRDB module, and perform affine transformation on the source image through each of the conditional features to achieve image compression of the source image.

[0099] Furthermore, the decoding end perceptible quality map determination module 410 includes:

[0100] An initial tensor generating unit, used for generating an initial tensor according to the source image;

[0101] An initial uniform quality map generating unit, used for filling each of the continuous control factors into each pixel point of the initial tensor to generate an initial uniform quality map;

[0102] The decoding end perceptible quality map determining unit is used to generate the decoding end perceptible quality map according to each of the initial uniform quality maps and all of the continuous control factors.

[0103] Furthermore, the compression and reconstruction module 430 includes:

[0104] A TRDB module input determination unit, configured to take the decoder perceptual quality map as input, and use the same decoder perceptual quality map as input of the TRDB module at the encoding end and the decoding end;

[0105] The conditional feature generating unit is used to perform a downsampling convolution operation on the perceptible quality map of the decoding end, extract features of the perceptible quality map of the decoding end, and generate the conditional feature.

[0106] Furthermore, the compression and reconstruction module 430 further includes:

[0107] An affine transformation unit, configured to determine affine parameters based on the conditional features, and perform affine transformation on the source image through an affine transformation formula and the conditional features;

[0108] Among them, the affine transformation formula is: , , is the conditional characteristic, is the preset mapping function, is the affine parameter, is the feature map of the source image.

[0109] Furthermore, the compression and reconstruction module 430 includes:

[0110] The source image feature map generating unit is used to extract the features of the perceptible quality map of the decoding end and generate the feature map of the source image.

[0111] Further, the TRDB module determination module 420 includes:

[0112] The TRDB module generating unit is configured to expand the SRDB module into the TRDB module based on a preset parameter sharing method.

[0113] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0114] The above-mentioned device can be implemented in the form of a computer program. Figure 7 Runs on the computer device shown.

[0115] See also Figure 7 , Figure 7 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0116] See also Figure 7 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0117] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any variable bit rate image compression method based on feature symmetric transformation.

[0118] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0119] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any variable bit rate image compression method based on feature symmetric transformation.

[0120] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0121] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0122] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0123] Based on the continuous control factor, determining a decoder-side perceptible quality map corresponding to the source image;

[0124] Constructing a symmetric rate-distortion balance control SRDB module, and expanding the SRDB module into a twin symmetric rate-distortion balance control TRDB module;

[0125] At least one conditional feature of the perceptible quality map at the decoding end is extracted through the TRDB module, and an affine transformation is performed on the source image through each of the conditional features to achieve image compression of the source image.

[0126] In one embodiment, based on the continuous control factor, a decoder-side perceptible quality map corresponding to the source image is determined to achieve:

[0127] Generate an initial tensor according to the source image;

[0128] Filling each of the continuous control factors into each pixel point of the initial tensor to generate an initial uniform quality map;

[0129] The decoding end perceptible quality map is generated according to each of the initial uniform quality maps and all of the continuous control factors.

[0130] In one embodiment, at least one conditional feature of the perceptible quality map at the decoding end is extracted by the TRDB module to achieve:

[0131] Taking the decoder perceptual quality map as input, and using the same decoder perceptual quality map as input to the TRDB module at the encoder and the decoder;

[0132] A downsampling convolution operation is performed on the decoder-end perceptible quality map to extract features of the decoder-end perceptible quality map to generate the conditional features.

[0133] In one embodiment, the source image is affine transformed by each of the conditional features to achieve image compression of the source image, so as to achieve:

[0134] Determine affine parameters based on the conditional features, and perform affine transformation on the source image using an affine transformation formula and the conditional features;

[0135] Among them, the affine transformation formula is: , , is the conditional characteristic, is the preset mapping function, is the affine parameter, is the feature map of the source image.

[0136] In one embodiment, the affine parameters are determined based on the conditional features, and before the source image is affine transformed using an affine transformation formula and the conditional features, the following is implemented:

[0137] The features of the decoder-end perceptible quality map are extracted to generate a feature map of the source image.

[0138] In one embodiment, a symmetric rate-distortion balance control SRDB module is constructed, and the SRDB module is expanded into a twin symmetric rate-distortion balance control TRDB module to achieve:

[0139] Based on a preset parameter sharing method, the SRDB module is expanded into the TRDB module.

[0140] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any variable bit rate image compression method based on feature symmetry transformation provided in an embodiment of the present application.

[0141] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0142] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A variable bit rate image compression method based on feature symmetry transformation, characterized in that: include: Based on the continuous control factor, determining a decoder-side perceptible quality map corresponding to the source image; Constructing a symmetric rate-distortion balance control SRDB module, and expanding the SRDB module into a twin symmetric rate-distortion balance control TRDB module; At least one conditional feature of the perceptible quality map at the decoding end is extracted through the TRDB module, and an affine transformation is performed on the source image through each of the conditional features to achieve image compression of the source image.

2. The variable bit rate image compression method based on feature symmetric transformation according to claim 1, characterized in that: The step of determining a decoding end perceptible quality map corresponding to the source image based on the continuous control factor includes: Generate an initial tensor according to the source image; Filling each of the continuous control factors into each pixel point of the initial tensor to generate an initial uniform quality map; The decoding end perceptible quality map is generated according to each of the initial uniform quality maps and all of the continuous control factors.

3. The variable bit rate image compression method based on feature symmetric transformation according to claim 1, characterized in that: The extracting at least one conditional feature of the perceptible quality map of the decoding end through the TRDB module includes: Taking the decoder perceptual quality map as input, and using the same decoder perceptual quality map as input to the TRDB module at the encoder and the decoder; A downsampling convolution operation is performed on the decoder-end perceptible quality map to extract features of the decoder-end perceptible quality map to generate the conditional features.

4. The variable bit rate image compression method based on feature symmetric transformation according to claim 3 is characterized in that: The performing affine transformation on the source image through each of the conditional features to achieve image compression of the source image includes: Determine affine parameters based on the conditional features, and perform affine transformation on the source image through an affine transformation formula and the conditional features; Among them, the affine transformation formula is , , is the conditional characteristic, is the preset mapping function, is the affine parameter, is the feature map of the source image, is the spatial adaptive feature transformation layer, is the feature map of the source image after spatial adaptive transformation, is element-wise multiplication.

5. The variable bit rate image compression method based on feature symmetric transformation according to claim 4 is characterized in that: The determining of the affine parameters based on the conditional features, before performing affine transformation on the source image by using an affine transformation formula and the conditional features, comprises: The features of the decoder-end perceptible quality map are extracted to generate a feature map of the source image.

6. The variable bit rate image compression method based on feature symmetric transformation according to claim 1, characterized in that: The method of constructing a symmetric rate-distortion balance control SRDB module and expanding the SRDB module into a twin symmetric rate-distortion balance control TRDB module includes: Based on a preset parameter sharing method, the SRDB module is expanded into the TRDB module.

7. A variable bit rate image compression device based on feature symmetry transformation, characterized in that: include: A decoding end perceptible quality map determining module, used for determining a decoding end perceptible quality map corresponding to a source image based on a continuous control factor; A TRDB module determination module is used to construct a symmetric rate-distortion balance control SRDB module, and expand the SRDB module into a twin symmetric rate-distortion balance control TRDB module; A compression and reconstruction module is used to extract at least one conditional feature of the perceptible quality map of the decoding end through the TRDB module, and perform affine transformation on the source image through each of the conditional features to achieve image compression of the source image.

8. The variable bit rate image compression device based on feature symmetric transformation according to claim 7, characterized in that: The decoding end perceptible quality map determination module includes: An initial tensor generating unit, used for generating an initial tensor according to the source image; An initial uniform quality map generating unit, used for filling each of the continuous control factors into each pixel point of the initial tensor to generate an initial uniform quality map; The decoding end perceptible quality map determining unit is used to generate the decoding end perceptible quality map according to each of the initial uniform quality maps and all of the continuous control factors.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the variable bit rate image compression method based on feature symmetric transformation as described in any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the variable bit rate image compression method based on feature symmetric transformation according to any one of claims 1 to 6.

Images