High-compression-ratio image compression method based on optimal transmission mapping

Abstract

The invention discloses a high-compression-ratio image compression method based on optimal transmission mapping, which is suitable for application scenes such as network transmission, image storage and image transmission, and can remarkably improve the image transmission efficiency and save network bandwidth and storage resources. Compression can be carried out based on generative adversarial learning or a diffusion model, and the generative adversarial learning is utilized by the generative adversarial learning to train an image compression model based on optimal transmission mapping. The model comprises a feature extraction layer, a mapping layer, a quantization coding layer and a decoding reconstruction layer. The mapping layer realizes mapping and optimization of image features through optimal transmission mapping based on convex mapping; an image compression model of the latter comprises a feature extraction layer, a mapping layer, a quantization coding layer and a decoding reconstruction layer, the feature extraction layer is responsible for coding the preprocessed image into low-dimensional feature representation, quantized features are restored into image features through a denoising network of a diffusion model, and the decoding reconstruction layer is responsible for decoding the image. And decoding the image into a high-quality compressed image close to the original image.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision, and in particular relates to a high compression rate image compression method based on optimal transmission mapping. Background Art

[0002] Image compression is a key technology used to reduce the size of image files to save storage space and reduce transmission costs. Common image compression methods include lossy compression and lossless compression. Lossy compression achieves a higher compression rate by sacrificing a certain degree of image quality, while lossless compression can maintain the integrity of the image, but the compression rate is relatively low.

[0003] The development of high-compression image compression methods aims to achieve higher compression rates, that is, to reduce the size of image files while maintaining image quality. These methods improve the compression rate by optimizing coding algorithms, data compression techniques, or image transformations, using redundant information in images or performing appropriate transformations on images. The research and application of high-compression image compression methods are of great significance in the fields of digital image processing, computer vision, and image transmission.

[0004] Rate-distortion optimization is a key issue in image compression. Under a given compression rate, rate-distortion optimization aims to minimize the distortion of the compressed image by optimizing the coding scheme or algorithm. Distortion can be understood as the loss of image quality, so the goal of rate-distortion optimization is to minimize the degree of image distortion while maintaining the compression rate requirement. This involves the study of image quality assessment and optimization algorithms to balance the trade-off between compression rate and image quality.

[0005] In the current information age, the generation and transmission of images are becoming more and more common and frequent. From digital photography to medical imaging, from video transmission to social media sharing, people's demand for image compression is becoming more and more urgent. Traditional compression methods have achieved certain results, but with the rise of high-definition images and large-scale image data, the demand for higher compression rates and better image quality is also growing.

[0006] Based on the above requirements for higher compression rate and better image quality, the existing technology still has the following technical problems:

[0007] [1] Distortion problem: Traditional image compression methods such as JPEG will introduce significant distortion at high compression rates, especially in terms of preserving details and edges. This distortion is unacceptable for some application scenarios (such as medical imaging and satellite imaging). Therefore, the technical problem to be solved by the present invention is how to achieve a high compression rate while reducing the degree of image distortion and improving image quality.

[0008] [2] Trade-off between compression rate and image quality: Existing image compression methods usually have a trade-off between compression rate and image quality. A higher compression rate is often accompanied by lower image quality, while better image quality may sacrifice a certain compression rate. The technical problem to be solved by the present invention is how to maintain good image quality while achieving a high compression rate and a high-quality image compression effect.

[0009] [3] Processing speed and computational complexity: Some existing image compression methods may require complex calculations and a large amount of computing resources, resulting in a long compression time, which is not conducive to real-time applications or large-scale image processing. Therefore, the technical problem to be solved by the present invention is how to improve the processing speed of image compression and reduce the computational complexity to meet the needs of practical applications.

[0010] [4] Applicability in application fields: Existing image compression methods have different applicability in different application fields. Some methods may perform well in specific fields or on specific types of images, but perform poorly in other fields or image types. The technical problem to be solved by the present invention is how to design a universal image compression method with high compression ratio that is applicable to multiple application fields and various types of images. Summary of the invention

[0011] The purpose of the embodiments of the present application is to provide a high-compression rate image compression method based on optimal transmission mapping to solve the above-mentioned problems existing in the related art.

[0012] According to a first aspect of an embodiment of the present application, a high compression rate image compression method based on optimal transmission mapping is provided, comprising:

[0013] Acquire an original image to be compressed and preprocess the original image;

[0014] Based on the preprocessed original image, a generative adversarial learning or diffusion model is used to train an image compression model based on an optimal transmission mapping, wherein the image compression model based on the optimal transmission mapping comprises a feature extraction layer, a mapping layer, a quantization coding layer, and a decoding and reconstruction layer, and the mapping layer uses an optimal transmission mapping based on a convex mapping to realize feature mapping;

[0015] Get the image to be compressed and compress it using the trained image compression model.

[0016] Furthermore, the preprocessing includes denoising, image enhancement and color space conversion.

[0017] Furthermore, in the image compression model based on optimal transmission mapping:

[0018] The feature extraction layer is used to extract features from the preprocessed image;

[0019] The mapping layer is used to map the extracted image features to feature space distribution based on the optimal transmission mapping;

[0020] The quantization coding layer is used to quantize and encode the mapped features;

[0021] The decoding and reconstruction layer is used to decode and reconstruct the encoded features to obtain a compressed image.

[0022] Furthermore, in the mapping layer, a transfer matrix is ​​constructed using convex mapping, and the transfer matrix is ​​optimized, wherein the optimization problem is:

[0023]

[0024] Among them, λ is the regularization parameter, c(x,y) represents the transmission cost from point x to point y, x and y are points in the source feature space and target feature space respectively, T(x,y) represents the transmission amount from x to y, N and M represent the number of samples in the source distribution and target distribution respectively.

[0025] Furthermore, in the decoding and reconstruction layer, decoding and reconstruction are achieved through a denoising network of a diffusion model or by performing inverse operations of a mapping layer and a quantization coding layer.

[0026] Furthermore, the training process of the image compression model based on the optimal transmission mapping includes:

[0027] S21: construct an initial transmission matrix;

[0028] S22: The feature extraction layer extracts features from the preprocessed original image, the mapping layer optimizes the transmission matrix using the extracted features, and maps the features using the optimized optimal transmission matrix, the quantization and coding layer quantizes and encodes the mapped features using the optimized optimal transmission matrix to obtain a compressed image feature representation, and the decoding and reconstruction layer decodes and reconstructs the compressed image feature representation to obtain a compressed image;

[0029] S23: Based on the compressed image and the corresponding preprocessed original image, calculate the training loss and update the parameters of the image compression model through a back propagation algorithm;

[0030] S24: Repeat steps S22-S23 until the model converges.

[0031] Further, in step S23, if the decoding reconstruction layer adopts the inverse operation of the mapping layer and the quantization coding layer, it is trained by generative adversarial learning, the generator, that is, the image compression model and the discriminator are adversarially trained, the adversarial loss is used to calculate the training loss, and the model parameters are updated by the back propagation algorithm;

[0032] If the decoding and reconstruction layer adopts a denoising network of a diffusion model, noise is added to the compressed image feature representation through the forward process of the diffusion model, denoising is performed through the reverse process of the diffusion model, the training loss is calculated based on the mean square error of the denoising target, and the model parameters are updated using the back propagation algorithm.

[0033] According to a second aspect of an embodiment of the present application, there is provided a high compression rate image compression device based on optimal transmission mapping, comprising:

[0034] A preprocessing module, used for acquiring an original image to be compressed and preprocessing the original image;

[0035] A training module is used to train an image compression model based on an optimal transmission mapping by using a generative adversarial learning or diffusion model based on the preprocessed original image, wherein the image compression model based on the optimal transmission mapping includes a feature extraction layer, a mapping layer, a quantization coding layer, and a decoding and reconstruction layer, and the mapping layer uses an optimal transmission mapping based on a convex mapping to realize feature mapping;

[0036] The compression module is used to obtain the image to be compressed and compress it using the trained image compression model.

[0037] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including:

[0038] one or more processors;

[0039] A memory for storing one or more programs;

[0040] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the first aspect.

[0041] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0042] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0043] [1] High compression rate: The present invention adopts a technical solution based on optimal transmission mapping, and realizes feature mapping by optimizing the transmission matrix, so that the image can achieve a higher compression rate while maintaining a high quality. Compared with traditional methods, the present invention can significantly reduce the amount of data after compression and improve the compression efficiency of the image.

[0044] [2] Maintaining image quality: Through preprocessing, feature extraction and reconstruction, the present invention can effectively maintain the visual quality and detail information of the image at a high compression rate. The application of optimal transfer mapping enables the reconstructed image to have better visual effects and image fidelity. Compared with traditional compression methods, the present invention can reduce problems such as image distortion, artifacts and block effects, and provide clearer and more realistic compressed images.

[0045] [3] Improve image transmission efficiency: The present invention uses the optimal transmission mapping principle to perform feature mapping and quantize the mapped features, thereby reducing the redundancy and repetitiveness of image data during transmission and storage. Compared with traditional methods, the present invention can significantly improve image transmission efficiency, save network bandwidth and storage resources, and is suitable for application scenarios such as network transmission, image storage and image transmission.

[0046] [4] Good adaptability and flexibility: The technical solution of the present invention can adapt to images of different types and characteristics and has certain adaptability and flexibility. By optimizing the transmission matrix and feature extraction process, the present invention can optimize and adjust according to the characteristics of different images to achieve better compression effect and image quality.

[0047] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] Figure 1 The present invention is a flowchart showing a high compression rate image compression method based on optimal transmission mapping according to an exemplary embodiment.

[0050] Figure 2 The invention is a block diagram showing a high compression ratio image compression device based on optimal transmission mapping according to an exemplary embodiment.

[0051] Figure 3 is a schematic diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0052] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0053] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0054] Figure 1 is a flowchart of a high compression rate image compression method based on optimal transmission mapping according to an exemplary embodiment. Figure 1 As shown, the method is applied in a terminal and may include the following steps:

[0055] S1: obtaining an original image to be compressed and preprocessing the original image;

[0056] Specifically, the preprocessing includes steps such as denoising, image enhancement and color space conversion. In one embodiment, the denoising process can use a convolutional autoencoder to learn the noise model of the image, and use the learned model to remove the noise in the image. Image enhancement can use an adaptive histogram equalization algorithm to improve the contrast and visual effect of the image by adjusting the distribution of image pixels. Color space conversion converts the image from RGB space to YCbCr space to separate brightness and chrominance information.

[0057] S2: Based on the preprocessed original image, the image compression model based on the optimal transmission mapping is trained by using a generative adversarial learning or diffusion model, wherein the image compression model based on the optimal transmission mapping includes a feature extraction layer, a mapping layer, a quantization coding layer, and a decoding and reconstruction layer, and the mapping layer uses an optimal transmission mapping based on a convex mapping to realize feature mapping;

[0058] Specifically, the image compression model based on optimal transmission mapping includes:

[0059] (1) Feature extraction layer: extract features from the preprocessed image;

[0060] In one embodiment, a pre-trained CNN model (ResNet) can be used to extract features of an image. For the GAN model and the diffusion model, the CNN model we use has different focuses, which will be described in detail later.

[0061] (2) Mapping layer: Based on the optimal transmission mapping, the extracted image features are mapped to the feature space distribution;

[0062] Specifically, the optimal transfer mapping is used to solve the optimal mapping problem between two probability distributions. It can be imagined as the process of accurately moving a bunch of points from one distribution to another distribution, so that the total cost of the movement is minimized. In image processing, the optimal transfer mapping can achieve the optimal deformation or alignment of the image by optimizing the distribution of pixels, thereby improving image quality and visualization.

[0063] The optimal transfer mapping minimizes the transfer cost (usually measured by a "distance metric", such as the Wasserstein distance) between the source feature and the target feature by optimizing the transfer matrix. The transfer matrix represents the probability transfer from the source distribution to the target distribution. Specifically, each element of the transfer matrix represents the transmission probability from the source feature to the target feature. In the present application, an adaptive transfer matrix is ​​designed to be optimized according to the importance and relevance of the features. This means that the construction process of the transfer matrix takes into account the relevance between the features and the extent to which they contribute to the final image quality. For example, some features may be more important to the visual effect of the image, and therefore will be given a higher weight in the transfer matrix. Specifically, the convex mapping in the optimal transfer theory is used to achieve the optimal transfer mapping between features. Convex mapping is a special mapping that maintains a convex structure. It can maintain the distance relationship in the feature space, thereby better retaining the structural information of the features.

[0064] The construction and optimization process of the transfer matrix usually involves numerical optimization methods, such as iterative methods or gradient descent methods. During the optimization process, the distance between the source and target features, as well as the correlation and importance between the features, are considered, and the elements of the transfer matrix are adjusted to minimize the transfer cost.

[0065] In this application, the construction and optimization process of the transmission matrix is ​​as follows:

[0066] Assume that the source feature distribution is P and the target feature distribution is Q. These two distributions can be represented by probability density functions p(x) and q(y) in the feature space, where x and y are points in the source feature space and the target feature space, respectively.

[0067] The optimal transmission problem can be described as finding a transmission matrix T that minimizes the transmission cost from the source distribution P to the target distribution Q. Usually, this cost can be expressed by the Wasserstein distance or other metrics:

[0068]

[0069] Among them, c(x,y) represents the transmission cost from point x to point y (for example, the square of the Euclidean distance), X and Y represent the source feature space and the target feature space respectively, T(x,y) represents the transmission amount from x to y, π(x,y) is a joint distribution function or joint probability distribution, which represents the probability transfer from the source distribution x to the target distribution y, π(x,y) is defined as the amount of "quality" or "probability" transmitted between the source point x and the target point y. It must satisfy the constraints of the marginal distribution:

[0070] For each source point x, the integral of π(x,y) over all y should equal the probability of the source distribution being at x;

[0071] For each target point y, the integral of π(x,y) over all x should equal the probability of the target distribution being at y.

[0072] The key to constructing the transfer matrix using convex mapping is to ensure the convexity of the mapping process. Specifically, the convex mapping φ:X→Y satisfies the following conditions:

[0073] φ(λx1+(1-λ)x2)≤λφ(x1)+(1-λ)φ(x2), for all x1,x2∈X and λ∈[0,1]

[0074] In the optimization process of the transmission matrix T, the following optimization problem needs to be solved:

[0075]

[0076] Where N and M represent the number of samples of the source distribution and the target distribution respectively;

[0077] The constraints are:

[0078]

[0079] Where p(x i ) and q(y j ) represent the weights (i.e., probability densities) of the source and target features at their respective positions.

[0080] Since the optimal transmission problem is usually a high-dimensional optimization problem, directly solving the transmission matrix may be very complicated. It can be solved by using an iterative method such as the Sinkhorn-Knopp algorithm. The Sinkhorn-Knopp algorithm converts the optimization problem into a series of solvable small problems by introducing an entropy regularization term:

[0081]

[0082] Among them, λ is the regularization parameter.

[0083] After the optimization process is completed, the output transfer matrix T* is the result of the optimal transfer mapping. This matrix contains the probability transfer information from the source feature to the target feature, which can be used for subsequent feature reconstruction and image compression.

[0084] In practice, the calculation of the transmission matrix is ​​usually completed through numerical optimization techniques. You can use OT libraries in Python (such as POT library) to solve the optimal transmission problem, or manually implement a customized optimal transmission algorithm based on a deep learning framework (such as PyTorch or TensorFlow).

[0085] (3) Quantization and coding layer: quantize and encode the mapped features;

[0086] Specifically, quantization converts continuous feature values ​​into discrete quantization indices to reduce the complexity of data representation. Encoding encodes the quantized features to achieve data compression. In specific implementations, vector quantization (VQ) can be used for quantization to obtain discrete features, and then adaptive arithmetic coding, Huffman coding or arithmetic coding can be used to encode according to the probability distribution of discrete features to improve compression efficiency.

[0087] (4) Decoding and reconstruction layer: decode and reconstruct the encoded features to obtain a compressed image;

[0088] Specifically, decoding and reconstruction can be performed through the denoising network of the diffusion model, or the inverse operation can be performed according to the mapping layer and the quantization coding layer to achieve decoding and reconstruction. The specific process of the latter is:

[0089] The decoding process, if a method such as adaptive arithmetic coding is used, the decoding process may involve a corresponding decoding algorithm, such as adaptive arithmetic decoding;

[0090] The reconstruction process restores the quantized features to the original features through the inverse optimal transmission mapping and the inverse quantization process. Finally, the image is reconstructed and restored based on the restored features and preprocessing information to obtain a high-quality compressed image. The inverse optimal transmission mapping is to remap the decoded features back to the original feature space through inverse mapping. In implementation, this may involve inversely transforming the decoded features according to the optimal transmission mapping algorithm used during compression to restore features that are close to the original image. The inverse quantization process is to restore the inverse mapped features to continuous original feature values. In specific implementations, the inverse quantization process can be completed by mapping the quantization index back to the range of the original feature value. For example, if uniform quantization is used, the inverse quantization process is to map the discrete quantization index back to the original continuous feature value range.

[0091] Finally, image reconstruction and restoration are performed by combining the restored features with the original image information preprocessed in step S1, which involves mapping the restored features back to the original image space and applying the inverse operation of the preprocessing step to generate a high-quality compressed image. In a specific implementation, various image reconstruction algorithms and techniques, such as interpolation algorithms, filtering algorithms, etc., can be used to achieve accurate reconstruction and restoration of the image.

[0092] For the above models, the Diffusion Model or Generative Adversarial Learning (GAN) can be used for training to ensure that the generated compressed images have high quality and high compression rate. Specifically, in generative adversarial learning, the generator (that is, the entire image compression model) is trained against the discriminator. The generator tries to generate high-quality compressed images, and the discriminator tries to distinguish between compressed images and original images. The goal of the generator is to deceive the discriminator into thinking that the compressed image is the original image. During the diffusion model training process, the mean square error (MSE) loss or perceptual loss can be used to evaluate the difference between the compressed image and the original image.

[0093] Specifically, the model training process may include:

[0094] S21: construct an initial transmission matrix;

[0095] Specifically, before starting training, an initial transfer matrix is ​​constructed using the initial parameters. This initial transfer matrix can be based on preliminary statistical information of the data or randomly initialized. The initial transfer matrix T_0 is used to guide how the model performs feature mapping in the early stage.

[0096] S22: The feature extraction layer extracts features from the preprocessed original image, the mapping layer optimizes the transmission matrix using the extracted features, and maps the features using the optimized optimal transmission matrix, the quantization and coding layer quantizes and encodes the mapped features using the optimized optimal transmission matrix to obtain a compressed image feature representation, and the decoding and reconstruction layer decodes and reconstructs the compressed image feature representation to obtain a compressed image;

[0097] Specifically, the training process starts with the feature extraction part of the model. The feature extraction network (e.g., a pre-trained CNN model such as ResNet) extracts features from the input image and generates a latent space representation. The feature extraction layer in GAN is mainly used for generation and classification. The generator gradually converts low-dimensional noise into a high-resolution image, while the discriminator gradually extracts features from the image to judge the authenticity of the image. In this process, the feature extraction layer is designed to gradually construct or deconstruct the image, and gradually generate or distinguish image details from coarse to fine. Feature extraction layer in Diffusion: focuses on capturing and preserving the global semantic information of the image, and supports the addition and removal of noise during the diffusion process. The feature extraction layer is designed to encode the image into a feature representation suitable for processing so that effective denoising operations can be performed in a high-dimensional noise space. At this point, the parameters of the feature extraction network can be updated. After feature extraction, the update of the current transfer matrix is ​​calculated using the current feature representation and optimal transfer theory. Here, the optimization of the transfer matrix can be achieved by a numerical method (e.g., the Sinkhorn-Knopp algorithm). When updating the transfer matrix, the distance metric between the currently extracted feature distribution and the target distribution (e.g. Wasserstein distance) is considered. Parameters related to the transfer matrix are also updated in this process. According to the latest transfer matrix, the quantization and encoding layer discretizes and encodes the features. The output of this step is the compressed image feature representation. The decoding and reconstruction layer decodes and reconstructs the compressed features to reconstruct the compressed image.

[0098] S23: Based on the compressed image and the corresponding preprocessed original image, calculate the training loss and update the parameters of the image compression model through a back propagation algorithm;

[0099] Specifically, if the decoding and reconstruction layer uses the inverse operation of the mapping layer and the quantization coding layer, it can be trained by generative adversarial learning. The generator (i.e., the entire image compression model) and the discriminator are trained adversarially. The generator tries to generate high-quality compressed images, and the discriminator tries to distinguish between compressed images and original images. The goal of the generator is to deceive the discriminator into thinking that the compressed image is the original image. The adversarial loss is used to calculate the training loss, and the back-propagation algorithm is used to update the model parameters.

[0100] If the decoding and reconstruction layer adopts the denoising network of the diffusion model, the training method of the diffusion model is used, which is divided into two main processes:

[0101] 1. Forward Process (Diffusion Process): gradually add noise to the image features to generate a series of gradually noisy representations. In this application, the encoded features (compressed image representations) are gradually added with noise to diffuse them into a simple Gaussian noise distribution. By gradually adding noise, a series of intermediate representations from the original features to Gaussian noise are generated;

[0102] The forward process is a Markov chain process that starts from the original feature x0 (i.e., the compressed image representation), gradually adds noise, and generates a series of intermediate representations x1, x2, ..., x T , and finally get the noise x T .

[0103] Given the original feature x0, the noise feature x at step t t Generated by:

[0104]

[0105] Among them, β t is a predefined constant (called the noise scheduler) that increases with the time step t and usually takes a value between (0,1).

[0106] According to the properties of Markov chain, the above process can be expressed as a one-step Gaussian distribution:

[0107]

[0108] in,

[0109] You can generate x from x0 in one step t :

[0110]

[0111] in is the noise sampled from a standard normal distribution.

[0112] 2. Reverse Process (Denoising Process): gradually denoise from the noisy representation and restore the original image features. In this application, it is the decoding and reconstruction process of the decoding and reconstruction layer (ie, the denoising network of the diffusion model).

[0113] The reverse process is a learned denoising process that gradually transforms x T Restored to x0. The core task of the diffusion model is to learn the conditional probability p θ (x t-1 |x t), which is achieved through neural networks such as UNet.

[0114] For the t-th step reverse process, it is approximately in the following form:

[0115]

[0116] where μ θ (x t ,t) and Σ θ (x t ,t) are the mean and covariance matrices of the neural network parameterization.

[0117] In the simplest case, we can assume that Σ θ (x t ,t) is a constant, so the training goal of the reverse process is to learn the denoising network ∈ θ (x t ,t), so that its estimated noise∈:

[0118]

[0119] Here ∈ θ (x t ,t) is the output of the network, representing the estimation of the added noise ∈.

[0120] The loss function used in the training process is usually the mean square error (MSE) based on the denoising target, and the formula is as follows:

[0121]

[0122] By minimizing this loss function, the denoising network ∈ θ It can predict noise more accurately, thereby gradually denoising it in the reverse process and ultimately restoring high-quality image features.

[0123] S24: Repeat steps S22-S23 until the model converges, that is, the loss function no longer decreases significantly or reaches a predetermined number of training times.

[0124] In summary, this application proposes two image compression schemes:

[0125] Solution 1: Image compression method based on generative adversarial learning

[0126] First, the original image to be compressed is obtained and preprocessed, including denoising, image enhancement, and color space conversion. Then, based on the preprocessed image, the image compression model based on the optimal transfer mapping is trained using generative adversarial learning. The model consists of a feature extraction layer, a mapping layer, a quantization coding layer, and a decoding and reconstruction layer. The mapping layer maps and optimizes image features through the optimal transfer mapping based on convex mapping. The trained model can effectively compress the input image and reduce data redundancy through quantization coding. The decoding and reconstruction layer in the compression process maps the compressed features back to the original image space to reconstruct a high-quality image.

[0127] Solution 2: Image compression method based on diffusion model

[0128] This scheme also first preprocesses the image to be compressed. Then, the diffusion model is used to train the image compression model based on the optimal transmission mapping. The feature extraction layer in the diffusion model is responsible for encoding the preprocessed image into a low-dimensional feature representation. Then, the mapping layer optimizes and maps these features in the feature space through the optimal transmission mapping based on convex mapping. During the compression process, the quantization coding layer quantizes the mapped features to reduce data redundancy. Finally, in the decoding and reconstruction stage, the quantized features are restored to image features through the inverse denoising process of the diffusion model (i.e., the step-by-step denoising network), and decoded into a high-quality compressed image close to the original image.

[0129] In this application, the optimization of the transfer matrix and the training of other parts of the model are performed alternately. Through this dynamic optimization method, the transfer matrix can be continuously adjusted to better match the changes in the feature space, while optimizing the feature extraction and compression process; the loss function includes not only the reconstruction error, but also the regularization term of the transmission cost, which helps to learn a better feature mapping; in each iteration, the transfer matrix is ​​recalculated or updated according to the current feature representation and target distribution to ensure that the learning direction of the model is continuously optimized.

[0130] S3: Obtain the image to be compressed, and compress it using the trained image compression model;

[0131] Specifically, compression is performed using the feature extraction layer, mapping layer, and quantization coding layer in the trained image compression model, and the image to be compressed should have the same or similar category as the original image to be compressed in S1.

[0132] Corresponding to the aforementioned embodiment of the high-compression ratio image compression method based on optimal transmission mapping, the present application also provides an embodiment of the high-compression ratio image compression device based on optimal transmission mapping.

[0133] Figure 2is a block diagram of a high compression rate image compression device based on optimal transmission mapping according to an exemplary embodiment. Figure 2 , the device may include:

[0134] A preprocessing module 21, used for acquiring an original image to be compressed and preprocessing the original image;

[0135] A training module 22 is used to train an image compression model based on an optimal transmission mapping by using a generative adversarial learning or diffusion model based on the preprocessed original image, wherein the image compression model based on the optimal transmission mapping includes a feature extraction layer, a mapping layer, a quantization coding layer, and a decoding and reconstruction layer, and the mapping layer uses an optimal transmission mapping based on a convex mapping to realize feature mapping;

[0136] The compression module 23 is used to obtain the image to be compressed and compress it using the trained image compression model.

[0137] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0138] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0139] Accordingly, the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the high-compression rate image compression method based on optimal transmission mapping as described above.

[0140] Accordingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the high compression rate image compression method based on optimal transmission mapping as described above. Figure 3 As shown, it is a hardware structure diagram of any device with data processing capability in which a deep learning data set access system provided by an embodiment of the present invention is located, except Figure 3In addition to the processor, memory and network interface shown, any device with data processing capability in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capability, which will not be described in detail.

[0141] Accordingly, the present application also provides a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by a processor, the high-compression image compression method based on optimal transmission mapping as described above is implemented. The computer-readable storage medium can be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Further, the computer-readable storage medium can also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output.

[0142] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0143] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A high compression rate image compression method based on optimal transmission mapping, characterized in that: include: Acquire an original image to be compressed and preprocess the original image; Based on the preprocessed original image, a generative adversarial learning or diffusion model is used to train an image compression model based on an optimal transmission mapping, wherein the image compression model based on the optimal transmission mapping comprises a feature extraction layer, a mapping layer, a quantization coding layer, and a decoding and reconstruction layer, and the mapping layer uses an optimal transmission mapping based on a convex mapping to realize feature mapping; Get the image to be compressed and compress it using the trained image compression model.

2. The method according to claim 1, characterized in that The preprocessing includes denoising, image enhancement and color space conversion.

3. The method according to claim 1, characterized in that In the image compression model based on optimal transmission mapping: The feature extraction layer is used to extract features from the preprocessed image; The mapping layer is used to map the extracted image features to feature space distribution based on the optimal transmission mapping; The quantization coding layer is used to quantize and encode the mapped features; The decoding and reconstruction layer is used to decode and reconstruct the encoded features to obtain a compressed image.

4. The method according to claim 3, characterized in that In the mapping layer, a transfer matrix is ​​constructed using convex mapping, and the transfer matrix is ​​optimized, wherein the optimization problem is: Among them, λ is the regularization parameter, c(x,y) represents the transmission cost from point x to point y, x and y are points in the source feature space and target feature space respectively, T(x,y) represents the transmission amount from x to y, N and M represent the number of samples in the source distribution and target distribution respectively.

5. The method according to claim 3, characterized in that: In the decoding and reconstruction layer, decoding and reconstruction are achieved through a denoising network of a diffusion model or by performing inverse operations of a mapping layer and a quantization coding layer.

6. The method according to claim 1, characterized in that The training process of the image compression model based on the optimal transmission mapping includes: S21: construct an initial transmission matrix; S22: The feature extraction layer extracts features from the preprocessed original image, the mapping layer optimizes the transmission matrix using the extracted features, and maps the features using the optimized optimal transmission matrix, the quantization and coding layer quantizes and encodes the mapped features using the optimized optimal transmission matrix to obtain a compressed image feature representation, and the decoding and reconstruction layer decodes and reconstructs the compressed image feature representation to obtain a compressed image; S23: Based on the compressed image and the corresponding preprocessed original image, calculate the training loss and update the parameters of the image compression model through a back propagation algorithm; S24: Repeat steps S22-S23 until the model converges.

7. The method according to claim 6, characterized in that In step S23, if the decoding and reconstruction layer adopts the inverse operation of the mapping layer and the quantization coding layer, it is trained by generative adversarial learning, the generator, that is, the image compression model and the discriminator are trained adversarially, the adversarial loss is used to calculate the training loss, and the model parameters are updated by the back propagation algorithm; If the decoding and reconstruction layer adopts a denoising network of a diffusion model, noise is added to the compressed image feature representation through the forward process of the diffusion model, denoising is performed through the reverse process of the diffusion model, the training loss is calculated based on the mean square error of the denoising target, and the model parameters are updated using the back propagation algorithm.

8. A high compression ratio image compression device based on optimal transmission mapping, characterized in that: include: A preprocessing module, used for acquiring an original image to be compressed and preprocessing the original image; A training module is used to train an image compression model based on an optimal transmission mapping by using a generative adversarial learning or diffusion model based on the preprocessed original image, wherein the image compression model based on the optimal transmission mapping includes a feature extraction layer, a mapping layer, a quantization coding layer, and a decoding and reconstruction layer, and the mapping layer uses an optimal transmission mapping based on a convex mapping to realize feature mapping; The compression module is used to obtain the image to be compressed and compress it using the trained image compression model.

9. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images