Image compression method, device, electronic equipment and storage medium

Abstract

The present application relates to the field of image processing technology, and in particular to an image compression method, device, electronic device and storage medium. The present application first cuts the acquired original image to obtain multiple regional images, and respectively determines the number of channels of the effective channels required for each regional image in the image compression process, and then, by inputting the number of channels corresponding to the original image and each regional image of the original image into the trained spatial domain adaptive compression autoencoder, the target potential representation and target reconstructed image of the original image after transformation by the neural network are obtained. The present application uses a spatial domain adaptive compression autoencoder to compress each regional image of the original image according to their respective adapted feature channel numbers, which can significantly improve the computational efficiency of the image compression process while ensuring that the target reconstructed image after decompression meets a certain quality, and can simultaneously take into account the joint optimization of distortion and coding overhead.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image compression method, device, electronic device and storage medium. Background Art

[0002] In recent years, with the advent of the digital information age and the development of multimedia computer technology, the amount of data people face has increased dramatically, and the research on data compression technology has received more and more attention. Image compression is to transform, encode and compress image data under the premise of meeting a certain fidelity and image quality, remove redundant data to reduce the amount of data required to identify digital images, and facilitate image storage and transmission.

[0003] At present, image compression methods mainly include traditional image compression methods and deep learning image compression methods. Among them, traditional image compression methods generally use transform coding technology, or a hybrid coding framework that combines predictive coding technology and transform coding technology to achieve image compression. Deep learning image compression methods perform nonlinear transformations on full-resolution input images to complete the entire image compression process.

[0004] However, the above-mentioned image compression methods generally only consider the optimization of the distortion rate which is common to compression problems, or only consider the optimization of the coding overhead during the encoding process, but cannot take into account the joint optimization of the distortion rate and the coding overhead at the same time, resulting in a waste of computing resources and time, or a loss of compression effect. Summary of the invention

[0005] In view of this, the embodiments of the present application at least provide an image compression method, device, electronic device and storage medium, which can significantly improve the computational efficiency in the image compression process while ensuring that the target reconstructed image after decompression meets a certain quality, and can simultaneously take into account the joint optimization of distortion and coding overhead.

[0006] This application mainly includes the following aspects:

[0007] In a first aspect, an embodiment of the present application provides an image compression method, which includes: obtaining an original image to be compressed; cutting the original image to obtain multiple regional images, and determining the number of channels of effective channels required for each regional image in the image compression process; inputting the number of channels corresponding to the original image and each regional image of the original image into a trained spatial domain adaptive compression autoencoder to obtain a target potential representation and a target reconstructed image after the original image is transformed by a neural network.

[0008] In a possible implementation, for any of the regional images of the original image; the number of channels corresponding to the regional image is determined according to the following steps: the regional image is input into a first backbone network in a trained first neural network model to determine image feature information corresponding to the regional image; the image feature information corresponding to the regional image is input into a first routing decision network in the first neural network model to determine the number of channels corresponding to the regional image; wherein the first neural network model is obtained by training in dependence on the second neural network model, and the network scale of the first neural network model is smaller than the network scale of the second neural network model.

[0009] In a second aspect, an embodiment of the present application also provides an image compression device, which includes: an acquisition module for acquiring an original image to be compressed; a determination module for cutting the original image to obtain multiple regional images, and respectively determining the number of channels of effective channels required for each regional image in the image compression process; a processing module for inputting the number of channels corresponding to the original image and each regional image of the original image into a trained spatial domain adaptive compression autoencoder to obtain a target potential representation and a target reconstructed image after the original image is transformed by a neural network.

[0010] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the image compression method described in the first aspect or any possible implementation manner of the first aspect.

[0011] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image compression method described in the first aspect or any possible implementation manner of the first aspect are executed.

[0012] The image compression method, device, electronic device and storage medium provided in the embodiments of the present application use a spatial domain adaptive compression autoencoder to compress each area image of the original image according to the number of feature channels adapted to each area image. Compared with the image compression method in the prior art which generally only considers the optimization of the distortion rate which is common to compression problems, or only considers the optimization of the coding overhead during the encoding process, but cannot simultaneously take into account the joint optimization of the distortion and the coding overhead, resulting in a waste of computing resources and time, or a loss of compression effect, the present application can minimize the computing time spent on transform coding during the compression process while ensuring the compression performance, and can simultaneously take into account the joint optimization of the distortion and the coding overhead.

[0013] Furthermore, the image compression method provided in the embodiment of the present application can also determine the number of channels adapted for each regional image by inputting each regional image in the original image into a first neural network model with a lightweight network scale, and the first neural network model is trained by relying on a second neural network model with a larger network scale. In this way, not only the accuracy of the number of channels of each regional image output can be guaranteed, but also the computational efficiency of calculating the number of channels of each regional image can be improved, thereby ensuring the computational efficiency of the entire image compression process.

[0014] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A flowchart of an image compression method provided by an embodiment of the present application is shown;

[0017] Figure 2 A flowchart of another image compression method provided by an embodiment of the present application is shown;

[0018] Figure 3 One of the functional module diagrams of an image compression device provided in an embodiment of the present application is shown;

[0019] Figure 4 A second functional module diagram of an image compression device provided in an embodiment of the present application is shown;

[0020] Figure 5A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art, under the guidance of the content of the present application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0022] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0023] In order to enable those skilled in the art to use the contents of this application, the following implementation is given in combination with the specific application scenario "image compression". For those skilled in the art, the general principles defined here can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application.

[0024] The following methods, devices, electronic devices or computer-readable storage media of the embodiments of the present application can be applied to any scenario requiring image compression. The embodiments of the present application are not limited to specific application scenarios. Any scheme using the image compression method and device provided by the embodiments of the present application is within the protection scope of the present application.

[0025] It is worth noting that before the present application was proposed, the traditional image compression methods and deep learning image compression methods in the existing schemes generally only considered the optimization of the distortion rate which is common to compression problems, or only considered the optimization of the coding overhead during the encoding process, but could not take into account the joint optimization of the distortion rate and the coding overhead at the same time, resulting in a waste of computing resources and time, or a loss of compression effect.

[0026] In response to the above problems, the embodiment of the present application first cuts the acquired original image into multiple regional images, and respectively determines the number of effective channels required for each regional image in the image compression process, and then uses the spatial domain adaptive compression autoencoder to compress each regional image of the original image according to their respective adapted feature channel numbers. This can significantly improve the computational efficiency of the image compression process while ensuring that the target reconstructed image obtained after decompression meets a certain quality, and can simultaneously take into account the joint optimization of distortion and coding overhead.

[0027] To facilitate the understanding of the present application, the technical solution provided by the present application is described in detail below in conjunction with specific embodiments.

[0028] Figure 1 This is a flow chart of an image compression method provided by an embodiment of the present application. Figure 1 As shown, the image compression method provided in the embodiment of the present application includes the following steps:

[0029] S101: Acquire an original image to be compressed.

[0030] In a specific implementation, an original image to be compressed is obtained.

[0031] S102: cutting the original image to obtain a plurality of regional images, and determining the number of effective channels required for each regional image during the image compression process.

[0032] In a specific implementation, after obtaining the original image to be compressed, the original image is first cut according to a preset cutting rule to obtain multiple regional images corresponding to the original image, and then the number of channels of the effective channels required for each regional image in the compression process is determined respectively. Here, the number of channels corresponding to each regional image is determined by saving the calculation time of the transform coding link in the image compression process as much as possible, under the premise of ensuring that the reconstructed image obtained after decompression meets a certain quality (quality such as T image distortion).

[0033] Here, the number of effective channels required for multiple regional images of the original image during the image compression process determined in the present application can be displayed and stored in the form of a transform capacity map (TCM), which describes the spatial distribution of the specified number of output channels corresponding to each regional image.

[0034] Here, a specific example of a cutting process is given, where the original image is cut according to a preset block size to obtain a plurality of regional images corresponding to the original image, and the regions that are not sufficient to constitute the preset size after the original image is cut are filled, thereby obtaining a plurality of regional images that are consistent with the preset block size. The preset block size can be set according to the actual requirements of the image compression processing.

[0035] The preset cutting rule is pre-set before the original image is compressed. The preset cutting rule can be set according to the actual requirements of the image compression process.

[0036] It should be noted that there are many ways to determine the number of effective channels required for each regional image obtained by cutting the original image during the image compression process, including but not limited to determination through a neural network model, manual specification, determination through a machine learning method, random generation, etc.

[0037] S103: Input the number of channels corresponding to the original image and each regional image of the original image into the trained spatial domain adaptive compression autoencoder to obtain the target potential representation and target reconstructed image after the original image is transformed by the neural network.

[0038] In a specific implementation, after determining the number of effective channels required for each regional image after the original image is cut during the image compression process, the spatial domain adaptive compression autoencoder can be guided according to the number of channels corresponding to each regional image to perform image compression processing on the original image. The "guidance" here can be understood as an indication of the number of activated filters corresponding to each convolutional neural transformation unit in the spatial domain adaptive compression autoencoder, so that the original image can be processed by the number of activated filters corresponding to each convolutional neural transformation unit, thereby obtaining the target potential representation and target reconstructed image after the original image is transformed by the neural network.

[0039] It should be noted that the traditional autoencoder (AE) generally uses a fully connected layer, which has no effect on one-dimensional signals, but for two-dimensional images or video signals, the fully connected layer will lose spatial information. The Compressive Auto-Encoders (CAE) is a convolution-based compressed autoencoder that uses convolutional layers (Conv) and pooling layers (GDN) to replace the fully connected layers of the traditional autoencoder, which can extract the spatial information hidden in the image. In this way, the compressed autoencoder can well retain the spatial information of the two-dimensional signal through the convolution operation.

[0040] Here, the compressed autoencoder is generated by using a convolutional neural network (CNN) to construct the encoder and decoder, which is a special case of the traditional autoencoder. The encoder and decoder of CAE are both CNNs. The convolutional network of the encoder learns to encode the input into a set of signals, and then the decoder CNN tries to reconstruct the input of the autoencoder. Among them, CNN works as a general feature extractor and learns how to best capture the input features. As the convolutional layer is added, the spatial size information passed to the next layer is reduced, but in the encoder, the size and depth of the reconstructed image should be the same as the input image, which means that the decoder should adjust the image size and convolution in some way to reconstruct the original image. In addition, convolution calculation can be considered as a method to effectively extract image features. During convolution calculation, a square convolution kernel is generally used to slide on the feature map corresponding to the input target potential representation at a specified step size, traversing each pixel in the input feature map. For each step size, the convolution kernel will overlap with the input feature map. The corresponding elements of the overlapping area are multiplied, summed, and then the bias term is added to obtain a pixel point of the output feature. Among them, the depth (number of channels) of the input feature map determines the depth of the convolution kernel of the current layer; the number of convolution kernels in the current layer determines the depth of the output feature map of the current layer.

[0041] However, due to the complexity of image content (which can be expressed as the richness of texture, the number of high-frequency components, and the amount of information) between different images and within the same image, the current compression autoencoder has a static neural network reasoning process, which results in different images and different regions in the same image always being subjected to indifferent calculation and processing processes, while ignoring the different requirements for the processing power of the convolutional neural transformation unit (i.e., model computing power). In other words, the current compression autoencoder cannot allocate appropriate computing power for different image contents under certain computing and processing resources, ultimately resulting in a waste of computing resources and time, or a loss of compression effect.

[0042] In response to the above-mentioned problems of existing compression autoencoders, this application proposes a spatially adaptive compression autoencoder (Spatially-Adaptive Convolutional Auto-Encoders, SA-CAE), which is an improved encoder of the compression autoencoder. The spatially adaptive compression autoencoder can realize image compression of the original image under the condition of variable spatial domain calculation, that is, different computing power can be allocated to each area image of the original image for image compression, and spatially variable image transformation can be realized under the guidance of the input transformation specification map. In this way, the calculation time of transform coding can be saved as much as possible while ensuring the compression performance, so as to achieve the joint optimization of image rate distortion and coding complexity.

[0043] The specific improvement process is as follows: This application first redefines the convolutional neural transformation unit in the existing compression autoencoder, so that the spatial domain adaptive compression autoencoder in this application can apply a set of scalable filters to part of the input tensor, that is, it can support a neural network with spatial adaptive scalable reasoning. The adaptive convolutional neural transformation unit provided by this application includes a spatial domain adaptive convolution layer and a spatial domain adaptive pooling layer, namely SA-Conv and SA-TConv, and its calculation is controlled by the transformation specification map proposed in this application. The transformation specification map describes the spatial distribution of the specified number of output channels corresponding to each regional image. Accordingly, SA-GDN plays its nonlinear normalization role according to the number of effective channels at different spatial positions in the input feature map.

[0044] Here, for the convolution calculation, assuming stride = 1 (step length) and padding = 0 (filling), the output feature map is It can be based on the input feature map of fixed size calculate:

[0045]

[0046] in, The number of feature channels at any position on the output feature map is specified; As the weight tensor of the SA-Conv operator; u, v are the coordinates of the 2D convolution kernel; x, y are the coordinates on the input feature map and the output feature map; m is the channel number of the output feature map; n is the channel number of the input feature map; C is the maximum number of channels of the input feature map; C' is the maximum number of channels of the output feature map, or the maximum number of convolution kernels (filters) of the convolution layer; K is the side length of the convolution kernel; W is the width of the input feature map; H is the height of the input feature map; W' is the width of the output feature map; H' is the height of the output feature map.

[0047] Here, the value range of m varies with the spatial coordinates (x, y) of the feature map, and the specification on the channel dimension can be considered to be given by TCM. For the inverse transform operation, it is processed by SA-TConv, where SA-TConv has an equivalent definition.

[0048] In addition, the TCM in the formula is aligned pixel by pixel with the output feature map, so the original image can be divided into blocks or not. In order to reduce the bit overhead, the present application segments the original image, specifically, using block-level TCM for implementation. If the granularity of TCM is fine enough (for example, reaching the pixel level), the overhead of memory access (mainly reflected in the addressing overhead and intermediate data cache overhead caused by multiple scattered indexes of the input feature map during the calculation process), coupled with the limited choice of hardware acceleration implementation of the convolution algorithm on the device side, may limit the number of floating-point operations per second (Floating-Point Operations Persecond, FLOPS) performed to reduce the conversion into actual reasoning acceleration. On the contrary, the present application follows a block-based computing power allocation strategy, and the calculation of two-dimensional convolution evolves from the proposed partition-based mechanism. First, the input feature map is cropped into smaller blocks to ensure the minimum occurrence of overlapping pixels and the best efficiency. Afterwards, the segmented feature map is processed separately to generate the convolution result. Finally, the intermediate results are merged to reconstruct the full-size output feature map.

[0049] Further, the specific implementation process of using the determined transformation specification map corresponding to the original image and the spatial domain adaptive compression autoencoder to obtain the target potential representation and the target reconstructed image after the original image is transformed by the neural network is described below. Here, the spatial domain adaptive compression autoencoder includes an encoder, an entropy model and a decoder, that is, in step S103, the number of channels corresponding to the original image and each regional image of the original image are respectively input into the trained spatial domain adaptive compression autoencoder to obtain the target potential representation and the target reconstructed image after the original image is transformed by the neural network, including:

[0050] Step 1031: Based on the number of effective channels required for any regional image in the original image during the image compression process, determine the number of effective channels required for each convolutional neural transformation unit of the encoder; the number of channels corresponding to the regional image is used to represent the number of channels corresponding to the target potential representation finally output by the encoder.

[0051] In a specific implementation, the number of channels corresponding to each region image in the transformation specification map represents the number of channels corresponding to the feature map finally output by the encoder in the spatial domain adaptive compression autoencoder. Here, the feature map output by the encoder is the target potential representation, and the target potential representation is also the latent space representation. The number of channels corresponding to each region image indicated by the latent space representation can determine the number of channels required by each convolutional neural transformation unit of the encoder in the spatial domain adaptive compression autoencoder.

[0052] Step 1032: Perform a neural network transformation on the region image according to the number of effective channels required by each convolutional neural transformation unit in the encoder to obtain a target potential representation of the region image.

[0053] Here, the target latent representation can be understood as the spatial representation after encoding the original image.

[0054] Step 1033: Input the target potential representation into the encoding unit in the entropy model for data encoding to obtain a target code stream for network transmission.

[0055] Step 1034: Input the target code stream into the decoding unit in the entropy model for data decoding to restore the target potential representation.

[0056] Step 1035: input the recovered target latent representation into the decoder for neural network inverse transformation to obtain a decompressed target reconstructed image.

[0057] In a specific implementation, the target potential representation indicates the number of channels corresponding to each region, so decoding can be performed according to the indicated number of channels to obtain a decompressed target reconstructed image.

[0058] In an embodiment of the present application, the acquired original image is first cut to obtain multiple regional images, and the number of channels of the effective channels required for each regional image in the image compression process is determined respectively, and then, by inputting the number of channels corresponding to the original image and each regional image of the original image into the trained spatial domain adaptive compression autoencoder, the target potential representation and target reconstructed image of the original image after transformation by the neural network are obtained. The present application uses a spatial domain adaptive compression autoencoder to compress each regional image of the original image according to the number of feature channels adapted to each region, which can significantly improve the computational efficiency of the image compression process while ensuring that the target reconstructed image after decompression meets a certain quality, and can simultaneously take into account the joint optimization of distortion and coding overhead.

[0059] Figure 2 This is a flow chart of an image compression method provided by an embodiment of the present application. Figure 2 As shown, the image compression method provided in the embodiment of the present application includes the following steps:

[0060] S201: Acquire an original image to be compressed.

[0061] S202: Cutting the original image to obtain a plurality of regional images.

[0062] S203: For any of the regional images of the original image, input the regional image into the first backbone network in the trained first neural network model to determine image feature information corresponding to the regional image; input the image feature information corresponding to the regional image into the first routing decision network in the first neural network model to determine the number of channels corresponding to the regional image.

[0063] The first neural network model is obtained by training based on the second neural network model, and the network scale of the first neural network model is smaller than that of the second neural network model.

[0064] It should be noted that in order to achieve the goal of jointly optimizing the distortion of the image during the compression process and the coding computational overhead, that is, rate-distortion optimization, it is necessary to ensure that the decompressed image saves as much computational time as possible in the transformation coding process while meeting a certain quality, that is, to allocate appropriate computing power to the image in this area, and the amount of computing power allocated is determined by the number of channels of the activated effective channels corresponding to each convolutional neural transformation unit. Here, a neural network model (NN) can be used to determine the number of channels of effective channels required for each regional image.

[0065] In the specific implementation, on the one hand, considering that the overall computational overhead spent in the image compression process should be minimized, this requires that the computational overhead of determining the number of effective channels required for each regional image should not be too large, otherwise it may offset the computational overhead reduced by using the computing power adapted to each regional image. Therefore, this application chooses to use a first neural network model (student model) with a lightweight network scale to determine the number of channels corresponding to the regional image, so that the computational overhead spent on determining the number of channels corresponding to each regional image can be reduced. The computational overhead of the first neural network model is almost negligible compared to the computational overhead generated by the spatial domain adaptive compression autoencoder encoding. On the other hand, the lightweight neural network model is limited by the network scale and has weak representation capabilities, and the accuracy of the determined number of channels cannot be fully guaranteed. In this regard, this application first trains a second neural network model (teacher model) with a larger network scale, and then trains the first neural network model through a knowledge distillation method, so that the trained first neural network model can approximate the behavior of the second neural network model, so that the output of the first neural network model can be as accurate as the output of the second neural network model as possible, thereby improving the accuracy of the number of channels output by the first neural network model.

[0066] Here, the first neural network model and the second neural network model can use a lightweight network (MobileNet), the network scale of the first neural network model is smaller than the network scale of the second neural network model, and the theoretical calculation amount of the second neural network model is about three times the theoretical calculation amount of the first neural network model. In one example, the first neural network model can use mobilenet-v3 large, and the second neural network model can use mobilenet-v3 small.

[0067] Here, in order to control the network scale of the neural network model, the present application has developed a lightweight first neural network model with a streamlined backbone network (first backbone network) and a single routing decision prediction head (first routing decision network) to support the online decision function of spatial adaptive transformation routing under stricter resource constraints. For the first neural network model, by limiting the network depth, that is, the number of layers of stacked reverse residual blocks, it has a theoretical computational amount of less than 100M FLOPS, and can complete reasoning in a very short time on most computing devices.

[0068] Furthermore, since the lightweight neural network model has a small model specification and limited expression ability, the general optimization process is not conducive to its convergence to an ideal performance level. Therefore, the present application proposes to use the knowledge distillation method for optimization and training to generate a first neural network model. Specifically, the first neural network model is trained according to the following steps:

[0069] Step 2031: Acquire multiple sample images, and determine the number of real channels corresponding to each sample region image obtained by cutting each sample image.

[0070] Step 2032: For any of the sample area images, the sample area image is input into the trained second neural network model to determine the number of first prediction channels of the sample area image, and the sample area image is input into the small initial neural network model to determine the number of second prediction channels of the sample area image.

[0071] Step 2033: Determine a first joint entropy loss corresponding to the sample area image based on a first entropy loss between the actual number of channels of the sample area image and the second predicted number of channels, and a second entropy loss between the first predicted number of channels of the sample area image and the second predicted number of channels.

[0072] Step 2034: Based on the first joint entropy loss corresponding to each sample area image of the multiple sample images, the model parameters of the small initial neural network model are adjusted until the first preset training cutoff condition is met, and the training of the small initial neural network model is stopped to generate the first neural network model.

[0073] In a specific implementation, the present application uses multiple sample images, the actual channel number (first label) of each sample area image corresponding to each sample image, and the second neural network model to determine the first predicted channel number (second label) of each sample area image, to jointly train a small initial neural network model and generate a first neural network model.

[0074] Here, the loss function corresponding to the first joint entropy loss is also the distillation loss function The definition is as follows:

[0075]

[0076] Where KL(·) represents the Kullback-Leibler divergence, represents the action loss function, the parameter σ is the weight coefficient of the distillation target, z pred is the number of the second prediction channels corresponding to the sample area image, z teacher is the number of the first prediction channel corresponding to the sample area image, z true is the actual number of channels corresponding to the sample area image, is the first entropy loss between the real number of channels of the sample area image and the second predicted number of channels, KL(z pred ,z teacher ) is the second entropy loss between the first predicted channel number and the second predicted channel number of the sample area image.

[0077] It should be noted that the first preset training cutoff condition can be set according to actual needs, or the loss value of the first joint entropy loss can meet the first preset threshold, or the number of training rounds can meet the preset number of times, etc.

[0078] Here, according to the above content, it can be known that in the training process of the first neural network model, the first prediction channel number of each sample area image output by the second neural network model is needed, that is, the first neural network model is trained to use the trained second neural network model. The following is an explanation of the training process of the second neural network model, that is, the second neural network model is trained according to the following steps:

[0079] Step a1: Acquire multiple sample images, and determine the number of real channels and the real degradation distortion cost value corresponding to each sample area image obtained by cutting each sample image.

[0080] Step a2: for any of the sample area images, input the sample area image into a large initial neural network model to determine the number of third prediction channels and the predicted degradation distortion cost value of the sample area image.

[0081] Step a3: Determine the second joint entropy loss of the sample area image based on the third entropy loss between the actual number of channels of the sample area image and the third predicted number of channels, and the fourth entropy loss between the predicted degradation distortion cost value and the actual degradation distortion cost value of the sample area image.

[0082] Step a4: Based on the second joint entropy loss corresponding to each sample area image of the multiple sample images, the model parameters of the large initial neural network model are adjusted until a second preset training cutoff condition is met, and the training of the large initial neural network model is stopped to generate the second neural network model.

[0083] It should be noted that the second neural network model includes a second backbone network, a second routing decision network and a cost regression network. The architecture of the second neural network model can be summarized as being based on a shared backbone network. There are two predictor branches above. Among them, the backbone network The architecture is determined to be a CNN feature extractor (second backbone network) composed of stacked reverse residual blocks. Two prediction heads (second routing decision network and cost regression network) are placed at the back of the backbone network to handle different learning objectives. The second routing decision network is used to learn the degradation distortion cost value objective, and the cost regression network is used to learn the channel number loss objective.

[0084] Here, through the second routing decision network The number of third prediction channels z of each sample area image can be determined pred ; Through the cost regression network The predicted degradation distortion cost ΔCost of each sample area image can be determined pred ; The mapping relationship between the two is:

[0085]

[0086]

[0087] Among them, the second routing decision network and cost regression network A similar network structure is used, which consists of two adjacent fully connected layers with a central activation function.

[0088] Here, for optimization and The loss functions of internal parameters are action loss functions And the Surrogate Loss Function The optimal decision goal of the former and the learning goal of replacing the actual running SA-CAE encoder to obtain the compression performance test results of the latter are defined as follows:

[0089]

[0090]

[0091] in, is a third entropy loss between the actual number of channels of the sample area image and the third predicted number of channels; The fourth entropy loss between the predicted degradation distortion cost value and the actual degradation distortion cost value of the sample area image; ΔCost pred ΔCost is the predicted degradation distortion cost of the sample area image; true It is the actual degradation distortion cost value of the image in the sample area.

[0092] Combining the two loss functions of the above formula, we get the loss function corresponding to the second joint entropy loss of the second neural network model:

[0093]

[0094] Among them, the parameter γ is the balance coefficient of the two sets of interrelated learning objectives, which controls the influence of the corresponding sub-loss function on the overall optimization process.

[0095] It should be noted that the second preset training cutoff condition can be set according to actual needs, or the loss value of the second joint entropy loss can meet the second preset threshold, or the number of training rounds can meet the preset number of times, etc.

[0096] It should also be noted that, in the process of training the first neural network model and the second neural network model, the actual number of channels z of each sample area image needs to be determined in advance. true , the actual number of channels z for each sample region image true It is determined by the trained spatial adaptive compression autoencoder (SA-CAE). Specifically, the actual number of channels z of the sample area image is determined according to the following steps: true :

[0097] Step b1: input the sample area image and a plurality of preset channel numbers into the trained spatial domain adaptive compression autoencoder to obtain sample reconstructed images corresponding to each preset channel number.

[0098] The sample reconstructed image refers to an image obtained after image compression processing is performed on the sample area image.

[0099] Here, the number of routable preset channels is limited to a number of fixed level levels, thus simplifying labeling. true (real number of channels) generation process and reduce the burden on related learning models. A binary search algorithm can be used to further accelerate the label generation process. In addition, if the action space of the dynamic reasoning routing supports a more fine-grained list of options for the number of routable preset channels w, the iterative steps of the binary search can be limited to a finite number of cycles, thereby providing an approximate solution to the best routing target as much as possible. The specifications of the routable subnetworks of a fixed level can be specified, and the time complexity of the enumeration method can also be controlled within a reasonable range. The collection of cost functions can be completed through sequential reasoning of the full quantum model.

[0100] Step b2: for any of the preset channel numbers, calculating the image distortion entropy loss between the sample reconstructed image corresponding to the preset channel number and the sample area image.

[0101] Here, the loss function corresponding to the rate-distortion joint objective of the traditional compressed autoencoder (CAE) is:

[0102]

[0103] Among them, the image quality distortion term Based on The loss function of the norm is given by the code rate term The estimation is performed by variational inference method. When the balance coefficient λ is fixed, the optimization degree of CAE can be obtained by the loss function The numerical value is evaluated.

[0104] in, The image distortion entropy loss between the sample reconstructed image and the sample area image corresponding to the preset number of channels.

[0105] Step b3: Calculate the loss difference between the image distortion entropy loss of the maximum number of channels and the image distortion entropy loss of the preset number of channels, and determine the loss difference as the degradation distortion cost value corresponding to the sample reconstructed image obtained by the sample area image under the preset number of channels; the maximum number of channels is the maximum number of channels that the convolutional neural transformation unit in the spatial domain adaptive compression autoencoder has.

[0106] It should also be noted that in order to avoid the spatial adaptive compression autoencoder (SA-CAE) provided in this application, the capacity degradation of the convolutional neural transformation unit in the SA-CAE (due to the way not using all channels to work) will cause different degrees of impact and consequences. In this regard, on the one hand, the applicant takes into account that on the rate-distortion scale, the displacement of a point cannot be scored only by distance measurement, because the direction of the displacement vector may also have an important impact on its influence. On the other hand, the applicant takes into account that although the degraded coding rate and distortion indicators can be considered to be near the level corresponding to the highest capacity, a single slope indicator cannot fully reflect the degree of loss of compression performance, because the rate-distortion performance drift caused by degradation is far and near. Inspired by the loss function corresponding to the rate-distortion joint objective of the traditional compression autoencoder (CAE), the degradation distortion cost ΔCost is defined as:

[0107]

[0108] in, is the image distortion entropy loss between the sample reconstruction image and the sample area image corresponding to the maximum number of channels; μ is the balance coefficient in the background of the transformation routing, which can be calculated based on μ 0 =1 is the center of the adjustment range; distortion measurement The original distortion metric After normalization as follows:

[0109]

[0110] Among them, λ′ is the time to calculate the routing cost, and The balance coefficient for calculating the binding can take a different value from λ. It should be noted that Both D′ and R are evaluated in a fine-grained manner.

[0111] Step b4: From the plurality of preset channel quantities, the preset channel quantities corresponding to which the degradation distortion cost value is less than or equal to the preset loss threshold are determined as candidate channel quantities.

[0112] In the specific implementation, the rate-distortion data at the image sample level or block level is used for the corresponding cost calculation. By adding a preset loss threshold, namely the tolerance threshold ε, to the original target, the maximum tolerance of the coding loss after the transform capacity degradation is Defined as:

[0113]

[0114] Among them, the image distortion entropy loss corresponding to the number of candidate channels should be less than

[0115] Step b5: Selecting the minimum channel number from multiple candidate channel numbers and determining it as the actual channel number of the sample area image.

[0116] Here, by setting the balance coefficient μ, to the original cost line The extrapolated distance d can be used to determine the threshold for the increase of the encoding cost for subsequent routing operations. Among all the candidate routing target model specifications that meet the specified range of negligible encoding penalties, the sub-model capacity with the minimum computational overhead is selected as the expected action (ExpectedAction) of the dynamic transformation routing. The expected action label z true (the true number of channels) can be determined as the Just-Unpenalized Model Capacity (JUMC) given by:

[0117]

[0118] Where w is the capacity of the transform network expressed by the number of convolution kernels (2D convolution filters) in the main encoder branch, that is, the number of preset channels, Represents the entropy loss of image distortion evaluated under the corresponding sub-model specification.

[0119] S204: Input the number of channels corresponding to the original image and each regional image of the original image into the trained spatial domain adaptive compression autoencoder to obtain the target potential representation and target reconstructed image after the original image is transformed by the neural network.

[0120] Here, the training sequence of the first neural network model, the second neural network model, and the spatial domain adaptive compression autoencoder is generally described. Step c1 is to first train the spatial domain adaptive compression autoencoder, and then use the trained spatial domain adaptive compression autoencoder to compress each sample area image according to different preset channel numbers, obtain the real degradation distortion cost value corresponding to each preset channel number, and determine the real channel number corresponding to each sample area image. Step c2 is to train the second neural network model based on the real degradation distortion cost value (label) corresponding to each preset channel number obtained by the spatial domain adaptive compression autoencoder and the real channel number corresponding to each sample area image. Step c3 is to use the trained second neural network model to output the first predicted channel number (label) of each sample area image and the real channel number of each sample area image calculated based on the trained spatial domain adaptive compression autoencoder to train the first neural network model.

[0121] Furthermore, a complete description of an image compression method provided by the present application is provided. The spatial domain adaptive compression autoencoder is used in combination with the first neural network model to achieve spatial domain adaptive image compression. The specific process is as follows: First, the input original image is pre-analyzed in blocks by the first neural network model to obtain a transform specification diagram for controlling the spatial domain adaptive compression autoencoder. Then, under the control of the corresponding transform specification diagram, the spatial domain adaptive compression autoencoder dynamically adjusts the transform specifications (in terms of the number of feature channels) used for images in different regions, thereby saving the calculation time of transform coding as much as possible while ensuring the compression performance, and taking into account the joint optimization of distortion and coding overhead. Here, the present application also compares the coding performance and time complexity of the spatial domain consistent image compression method and the spatial domain adaptive image compression method (the image compression method provided by the present application), wherein Table 1 is the comparison result of the verification data set, and Table 2 is the comparison result of the evaluation data set.

[0122] Table 1 Comparison of encoding performance and time complexity (validation dataset)

[0123]

[0124] Table 2 Comparison of encoding performance and time complexity (evaluation dataset)

[0125]

[0126] Experiments have shown that the image compression method provided by this application has significant improvements over the deep learning image compression method using traditional compression autoencoders and the image compression method using spatially consistent scalable CNN-CAE. Compared with the original static CNN-CAE model, this application can bring a 30% to 40% increase in model execution speed with minimal loss in compression efficiency (the average bit rate increase is controlled within 1%). Compared with the spatially consistent scalable CNN-CAE model, this application can provide more than 10% additional operation acceleration at the same or better compression efficiency.

[0127] The image compression method disclosed in this application can adaptively adjust the number of feature channels by block division, significantly improve the computational efficiency of the relevant neural network model without losing compression efficiency, and accelerate the image encoding and decoding process. This application can improve the characteristics of long computation time and poor real-time performance of image compression methods based on deep learning, which is conducive to the large-scale application and multi-platform deployment of such methods.

[0128] Among them, the description of S201, S202, and S204 can refer to the description of S101 to S103, and can achieve the same technical effect, which is not repeated here.

[0129] Based on the same application concept, an image compression device corresponding to the image compression method provided in the above embodiment is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the image compression method in the above embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0130] like Figure 3 , Figure 4 As shown, Figure 3 This is one of the functional module diagrams of an image compression device 300 provided in an embodiment of the present application. Figure 4 FIG2 is a second functional module diagram of an image compression device 300 provided in an embodiment of the present application. Figure 3 and Figure 4 As shown, the image compression device 300 includes:

[0131] The acquisition module 310 is used to acquire the original image to be compressed; the determination module 320 is used to cut the original image into multiple regional images, and respectively determine the number of channels of the effective channels required for each regional image during the image compression process; the processing module 330 is used to input the number of channels corresponding to the original image and each regional image of the original image into the trained spatial domain adaptive compression autoencoder to obtain the target potential representation and target reconstructed image after the original image is transformed by the neural network.

[0132] In a possible implementation, Figure 4 As shown, the determination module 320 includes a first determination unit 321; for any of the regional images of the original image, the first determination unit 321 is used to determine the number of channels corresponding to the regional image according to the following steps: inputting the regional image into the first backbone network in the trained first neural network model to determine the image feature information corresponding to the regional image; inputting the image feature information corresponding to the regional image into the first routing decision network in the first neural network model to determine the number of channels corresponding to the regional image; wherein, the first neural network model is obtained by training in dependence on the second neural network model, and the network scale of the first neural network model is smaller than the network scale of the second neural network model.

[0133] In a possible implementation, Figure 4As shown, the determination module 320 includes a first training unit 322; the first training unit 322 is used to train the first neural network model according to the following steps: obtain multiple sample images, and determine the real channel numbers corresponding to each sample area image obtained by cutting each sample image; for any of the sample area images, input the sample area image into the trained second neural network model to determine the first predicted channel number of the sample area image, and input the sample area image into the small initial neural network model to determine the second predicted channel number of the sample area image; determine the first joint entropy loss corresponding to the sample area image according to the first entropy loss between the real channel number and the second predicted channel number of the sample area image, and the second entropy loss between the first predicted channel number and the second predicted channel number of the sample area image; based on the first joint entropy loss corresponding to each sample area image of the multiple sample images, adjust the model parameters of the small initial neural network model until the first preset training cutoff condition is met, stop training the small initial neural network model, and generate the first neural network model.

[0134] In a possible implementation, Figure 4 As shown, the determination module 320 includes a second training unit 323; the second training unit 323 is used to train the second neural network model according to the following steps: obtain multiple sample images, and determine the real channel number and real degradation distortion cost value corresponding to each sample area image obtained by cutting each sample image; for any of the sample area images, input the sample area image into the large initial neural network model to determine the third predicted channel number and predicted degradation distortion cost value of the sample area image; determine the second joint entropy loss of the sample area image according to the third entropy loss between the real channel number and the third predicted channel number of the sample area image, and the fourth entropy loss between the predicted degradation distortion cost value and the real degradation distortion cost value of the sample area image; based on the second joint entropy loss corresponding to each sample area image of the multiple sample images, adjust the model parameters of the large initial neural network model until the second preset training cutoff condition is met, stop training the large initial neural network model, and generate the second neural network model.

[0135] In a possible implementation, Figure 4As shown, the determination module 320 includes a second determination unit 324; the second determination unit 324 is used to determine the real channel number of the sample area image according to the following steps: input the sample area image and multiple preset channel numbers into the trained spatial domain adaptive compression autoencoder to obtain sample reconstructed images corresponding to each preset channel number; for any of the preset channel numbers, calculate the image distortion entropy loss between the sample reconstructed image corresponding to the preset channel number and the sample area image; calculate the loss difference between the image distortion entropy loss of the maximum channel number and the image distortion entropy loss of the preset channel number, and determine the loss difference as the degradation distortion cost value corresponding to the sample reconstructed image obtained by the sample area image under the preset channel number; the maximum channel number is the maximum number of channels that the convolutional neural transformation unit in the spatial domain adaptive compression autoencoder has; from the multiple preset channel numbers, determine the preset channel number whose corresponding degradation distortion cost value is less than or equal to the preset loss threshold as the candidate channel number; select the minimum channel number from the multiple candidate channel numbers and determine it as the real channel number of the sample area image.

[0136] In a possible implementation, the spatial domain adaptive compression autoencoder includes an encoder, an entropy model and a decoder; Figure 4 As shown, the processing module 330 is specifically used to: determine the number of channels of effective channels required by each convolutional neural transformation unit of the encoder based on the number of channels of effective channels required by any of the regional images in the original image during the image compression process; the number of channels corresponding to the regional image is used to characterize the number of channels corresponding to the target potential representation output by the encoder; according to the number of channels of effective channels required by each convolutional neural transformation unit in the encoder, perform a neural network transformation on the regional image to obtain the target potential representation of the regional image; input the target potential representation into the encoding unit in the entropy model for data encoding to obtain a target code stream for network transmission; input the target code stream into the decoding unit in the entropy model for data decoding to restore the target potential representation; input the restored target potential representation into the decoder for a neural network inverse transformation to obtain a decompressed target reconstructed image.

[0137] In an embodiment of the present application, the original image acquired by the acquisition module 310 is first cut to obtain multiple regional images, and the number of channels of the effective channels required for each regional image in the image compression process is determined according to the determination module 320, and then the number of channels corresponding to the original image and each regional image of the original image is input into the trained spatial domain adaptive compression autoencoder through the processing module 330 to obtain the target potential representation and target reconstructed image after the original image is transformed by the neural network. The present application uses the spatial domain adaptive compression autoencoder to compress each regional image of the original image according to the number of feature channels adapted to each region, which can significantly improve the computational efficiency of the image compression process while ensuring that the target reconstructed image after decompression meets a certain quality, and can simultaneously take into account the joint optimization of distortion and coding overhead.

[0138] Based on the same application concept, see Figure 5 As shown, it is a structural schematic diagram of an electronic device 500 provided in an embodiment of the present application, including: a processor 510, a memory 520 and a bus 530, the memory 520 stores machine-readable instructions executable by the processor 510, when the electronic device 500 is running, the processor 510 and the memory 520 communicate with each other through the bus 530, and the machine-readable instructions are executed by the processor 510 when running to perform the steps of the image compression method as described in any of the above embodiments.

[0139] Specifically, when the machine-readable instructions are executed by the processor 510, the following processing can be performed: obtaining an original image to be compressed; cutting the original image to obtain multiple regional images, and determining the number of channels of effective channels required for each regional image during the image compression process; inputting the number of channels corresponding to the original image and each regional image of the original image into a trained spatial domain adaptive compression autoencoder to obtain a target potential representation and a target reconstructed image after the original image is transformed by a neural network.

[0140] In an embodiment of the present application, the acquired original image is first cut to obtain multiple regional images, and the number of channels of the effective channels required for each regional image in the image compression process is determined respectively, and then, by inputting the number of channels corresponding to the original image and each regional image of the original image into the trained spatial domain adaptive compression autoencoder, the target potential representation and target reconstructed image of the original image after transformation by the neural network are obtained. The present application uses a spatial domain adaptive compression autoencoder to compress each regional image of the original image according to the number of feature channels adapted to each region, which can significantly improve the computational efficiency of the image compression process while ensuring that the target reconstructed image after decompression meets a certain quality, and can simultaneously take into account the joint optimization of distortion and coding overhead.

[0141] Based on the same application concept, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image compression method provided in the above embodiment are executed.

[0142] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, the above-mentioned image compression method can be executed. The spatial domain adaptive compression autoencoder can compress the images of each region of the original image according to their respective adapted feature channel numbers. Under the premise of ensuring that the target reconstructed image after decompression meets a certain quality, the computational efficiency of the image compression process can be significantly improved, and the joint optimization of distortion and coding overhead can be taken into account at the same time.

[0143] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0144] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0145] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0146] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0147] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image compression method, characterized in that: The image compression method comprises: Obtaining the original image to be compressed; Cutting the original image to obtain a plurality of regional images, and determining the number of effective channels required for each regional image in the image compression process; Inputting the number of channels corresponding to the original image and each region image of the original image into the trained spatial domain adaptive compression autoencoder to obtain a target potential representation and a target reconstructed image after the original image is transformed by a neural network; For any of the region images of the original image, determine the number of channels corresponding to the region image according to the following steps: Inputting the region image into a first backbone network in a trained first neural network model to determine image feature information corresponding to the region image; Inputting image feature information corresponding to the regional image into a first routing decision network in the first neural network model to determine the number of channels corresponding to the regional image; The first neural network model is obtained by training based on the second neural network model, and the network scale of the first neural network model is smaller than the network scale of the second neural network model; The first neural network model is trained according to the following steps: Acquire multiple sample images, and determine the number of real channels corresponding to each sample area image obtained by cutting each sample image; For any of the sample area images, input the sample area image into the trained second neural network model to determine the number of first prediction channels of the sample area image, and input the sample area image into a small initial neural network model to determine the number of second prediction channels of the sample area image; Determine a first joint entropy loss corresponding to the sample region image according to a first entropy loss between a real number of channels of the sample region image and a second predicted number of channels, and a second entropy loss between the first predicted number of channels of the sample region image and the second predicted number of channels; Based on the first joint entropy loss corresponding to each sample region image of the multiple sample images, adjusting the model parameters of the small initial neural network model until a first preset training cutoff condition is met, stopping the training of the small initial neural network model, and generating the first neural network model; The second neural network model is trained according to the following steps: Acquire multiple sample images, and determine the number of real channels and real degradation distortion cost values ​​corresponding to each sample area image obtained by cutting each sample image; For any of the sample area images, input the sample area image into a large initial neural network model to determine the number of third prediction channels and a predicted degradation distortion cost value of the sample area image; Determine a second joint entropy loss of the sample area image according to a third entropy loss between a real number of channels of the sample area image and a third predicted number of channels, and a fourth entropy loss between a predicted degradation distortion cost value and a real degradation distortion cost value of the sample area image; Based on the second joint entropy loss corresponding to each sample region image of the multiple sample images, adjusting the model parameters of the large initial neural network model until a second preset training cutoff condition is met, stopping the training of the large initial neural network model, and generating the second neural network model; The actual number of channels of the sample area image is determined according to the following steps: Inputting the sample area image and a plurality of preset channel numbers into the trained spatial domain adaptive compression autoencoder to obtain sample reconstructed images corresponding to each preset channel number; For any of the preset number of channels, calculating the image distortion entropy loss between the sample reconstructed image corresponding to the preset number of channels and the sample region image; Calculate the loss difference between the image distortion entropy loss of the maximum number of channels and the image distortion entropy loss of the preset number of channels, and determine the loss difference as the degradation distortion cost value corresponding to the sample reconstructed image obtained by the sample area image under the preset number of channels; the maximum number of channels is the maximum number of channels that the convolutional neural transformation unit in the spatial domain adaptive compression autoencoder has; From the plurality of preset channel quantities, determining the preset channel quantities whose corresponding degradation distortion cost values ​​are less than or equal to the preset loss threshold as the candidate channel quantities; A minimum channel number is selected from a plurality of candidate channel numbers and determined as the actual channel number of the sample region image.

2. The image compression method according to claim 1, characterized in that: The spatial domain adaptive compression autoencoder includes an encoder, an entropy model and a decoder; the number of channels corresponding to the original image and each region image of the original image are input into the trained spatial domain adaptive compression autoencoder to obtain the target potential representation and the target reconstructed image after the original image is transformed by the neural network, including: Based on the number of effective channels required by any of the regional images in the original image during the image compression process, determining the number of effective channels required by each convolutional neural transformation unit of the encoder; the number of channels corresponding to the regional image is used to characterize the number of channels corresponding to the target potential representation output by the encoder; According to the number of effective channels required by each convolutional neural transformation unit in the encoder, the region image is subjected to a neural network transformation to obtain a target potential representation of the region image; Inputting the target potential representation into the encoding unit in the entropy model for data encoding to obtain a target bit stream for network transmission; Inputting the target code stream into a decoding unit in the entropy model to perform data decoding, and recovering the target potential representation; The recovered target potential representation is input into the decoder for neural network inverse transformation to obtain a decompressed target reconstructed image.

3. The image compression method according to claim 2, characterized in that: Each convolutional neural transform unit consists of a spatially adaptive convolution layer and a spatially adaptive pooling layer.

4. An image compression device, characterized in that: The image compression device comprises: An acquisition module, used for acquiring the original image to be compressed; A determination module, used for cutting the original image to obtain a plurality of regional images, and respectively determining the number of effective channels required for each regional image during the image compression process; A processing module, used for inputting the number of channels corresponding to the original image and each regional image of the original image into a trained spatial domain adaptive compression autoencoder to obtain a target potential representation and a target reconstructed image after the original image is transformed by a neural network; The determination module is specifically used for: Inputting the region image into a first backbone network in a trained first neural network model to determine image feature information corresponding to the region image; Inputting image feature information corresponding to the regional image into a first routing decision network in the first neural network model to determine the number of channels corresponding to the regional image; The first neural network model is obtained by training based on the second neural network model, and the network scale of the first neural network model is smaller than the network scale of the second neural network model; The first neural network model is trained according to the following steps: Acquire multiple sample images, and determine the number of real channels corresponding to each sample area image obtained by cutting each sample image; For any of the sample area images, input the sample area image into the trained second neural network model to determine the number of first prediction channels of the sample area image, and input the sample area image into a small initial neural network model to determine the number of second prediction channels of the sample area image; Determine a first joint entropy loss corresponding to the sample region image according to a first entropy loss between a real number of channels of the sample region image and a second predicted number of channels, and a second entropy loss between the first predicted number of channels of the sample region image and the second predicted number of channels; Based on the first joint entropy loss corresponding to each sample region image of the multiple sample images, adjusting the model parameters of the small initial neural network model until a first preset training cutoff condition is met, stopping the training of the small initial neural network model, and generating the first neural network model; The second neural network model is trained according to the following steps: Acquire multiple sample images, and determine the number of real channels and real degradation distortion cost values ​​corresponding to each sample area image obtained by cutting each sample image; For any of the sample area images, input the sample area image into a large initial neural network model to determine the number of third prediction channels and a predicted degradation distortion cost value of the sample area image; Determine a second joint entropy loss of the sample area image according to a third entropy loss between a real number of channels of the sample area image and a third predicted number of channels, and a fourth entropy loss between a predicted degradation distortion cost value and a real degradation distortion cost value of the sample area image; Based on the second joint entropy loss corresponding to each sample region image of the multiple sample images, adjusting the model parameters of the large initial neural network model until a second preset training cutoff condition is met, stopping the training of the large initial neural network model, and generating the second neural network model; The actual number of channels of the sample area image is determined according to the following steps: Inputting the sample area image and a plurality of preset channel numbers into the trained spatial domain adaptive compression autoencoder to obtain sample reconstructed images corresponding to each preset channel number; For any of the preset number of channels, calculating the image distortion entropy loss between the sample reconstructed image corresponding to the preset number of channels and the sample region image; Calculate the loss difference between the image distortion entropy loss of the maximum number of channels and the image distortion entropy loss of the preset number of channels, and determine the loss difference as the degradation distortion cost value corresponding to the sample reconstructed image obtained by the sample area image under the preset number of channels; the maximum number of channels is the maximum number of channels that the convolutional neural transformation unit in the spatial domain adaptive compression autoencoder has; From the plurality of preset channel quantities, determining the preset channel quantities whose corresponding degradation distortion cost values ​​are less than or equal to the preset loss threshold as the candidate channel quantities; A minimum channel number is selected from a plurality of candidate channel numbers and determined as the actual channel number of the sample region image.

5. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the image compression method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image compression method according to any one of claims 1 to 3 are executed.

Images