Image compression encoding method, device and equipment without entropy encoding and storage medium

Abstract

The application discloses an image compression coding method without entropy coding, an image compression coding device without entropy coding, an electronic device and a storage medium. The image compression coding method without entropy coding comprises the following steps: performing sampling convolution on an original image to obtain image data after sampling convolution, performing quantization processing on the image data after sampling convolution to obtain image quantization data, and performing adaptive run-length coding on the image quantization data to obtain image compression data. The image compression coding method without entropy coding provided by the embodiment of the application can obtain image compression data through adaptive run-length coding, improve the compression ratio, improve the image data compression effect, and the compression amount obtained through the encoder can meet the needs of actual application, and the following condition existing in the related art is improved: the compression amount obtained through the encoder in the related art is not high enough in terms of compression ratio, which leads to the difficulty in meeting the needs of actual application in terms of compression amount, and the compression ratio needs to be further improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an entropy-free image compression coding method, an entropy-free image compression coding device, an electronic device, and a storage medium. Background Technology

[0002] Image compression methods in related technologies mainly consist of two parts: transform and entropy coding. The computation is complex, and the process typically involves first using transform to obtain a compressed value with low entropy, and then using entropy coding to encode this value into a uniformly distributed binary bitstream. The compression ratio obtained by the encoder in these technologies is insufficient, making it difficult to meet the needs of practical applications. Further improvements in compression ratio are needed.

[0003] The above statements are for the purpose of providing background information in relation to this application only, and do not necessarily constitute prior art. Summary of the Invention

[0004] The purpose of this application is to provide an entropy-free image compression coding method, an entropy-free image compression coding apparatus, an electronic device, and a storage medium to improve the following situation in related technologies: the compression amount obtained by the encoder in related technologies is not high enough in terms of compression ratio, making it difficult to meet the needs of practical applications in terms of compression amount, and the compression ratio needs to be further improved. To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0005] According to one aspect of the embodiments of this application, an entropy-free image compression coding method is provided, applied to an encoder, the entropy-free image compression coding method comprising:

[0006] The original image is sampled and convolved to obtain the sampled and convolved image data;

[0007] The sampled convolutional image data is quantized to obtain quantized image data;

[0008] Adaptive run-length encoding is performed on the quantized image data to obtain compressed image data.

[0009] In some embodiments of this application, adaptive run-length encoding is performed on the image quantization data to obtain image compressed data, including:

[0010] The image quantization data is expanded and re-represented to obtain the expanded and re-represented data segment;

[0011] Based on the image resolution, determine the data segments in the expanded and re-represented data segments that meet the preset requirements and the data segments that do not meet the preset requirements;

[0012] Run-length encoding is performed on the data segments that meet the preset requirements, and the data segments that do not meet the preset requirements are left unchanged. The image resolution is added to the run-length encoded data to obtain the image compressed data.

[0013] In some embodiments of this application, the image quantization data is subjected to unwrap and re-representation processing to obtain unwrap and re-represented data segments, including:

[0014] The image quantization data is unfolded into a one-dimensional vector;

[0015] Traverse the one-dimensional vector and record the number of consecutive repetitions of consecutive repeating characters in the one-dimensional vector;

[0016] For characters that are repeated multiple times in the quantized image data, the number of times the characters are repeated is represented by numbers to obtain the expanded and re-represented data segment.

[0017] In some embodiments of this application, the encoder includes three downsampling convolutional layers and a binary mapping layer connected in series; the step of performing sampling convolution on the original image to obtain sampled convolutional image data includes:

[0018] The encoder sequentially passes the original image through the three downsampling convolutional layers to perform downsampling convolution processing, thereby obtaining image data after downsampling convolution processing.

[0019] The image data after downsampling convolution is binary mapped using a binary mapping layer to obtain the image data after sampling convolution.

[0020] In some embodiments of this application, the sampled convolutional image data is quantized to obtain quantized image data, including:

[0021] The image data after sampling and convolution is quantized and mapped to a uniformly distributed binary bitstream to obtain image quantized data.

[0022] In some embodiments of this application, the entropy-free image compression coding method further includes: sending the image compression data to a decoder so that the decoder receives the image compression data from the encoder, performs run-length decoding on the image compression data, and obtains the decoded reconstructed image.

[0023] According to another aspect of the embodiments of this application, an entropy-free image compression coding method is provided, comprising:

[0024] The decoder receives image compression data from the encoder, performs run-length decoding on the image compression data, and obtains the decoded reconstructed image;

[0025] The post-processing network model processes the reconstructed image to obtain a post-processed optimized image;

[0026] The post-processing network model includes a first convolutional layer, a downsampling convolutional layer, a GRDB module, an upsampling convolutional layer, a CBAM module, and a second convolutional layer connected in sequence.

[0027] In some embodiments of this application, the GRDB module includes a first RDB module, a second RDB module, a third RDB module, a splicing module, a convolution module, and an addition module connected sequentially from front to back. The output terminals of the first RDB module, the second RDB module, and the third RDB module are respectively connected to the input terminal of the splicing module; the input terminal of the first RDB module is connected to the input terminal of the addition module.

[0028] In some embodiments of this application, any one of the first RDB module, the second RDB module, and the third RDB module includes a first convolution activation module, a second convolution activation module, a third convolution activation module, a splicing module, a convolution module, and an addition module connected sequentially from front to back; the input terminal of the first convolution activation module is connected to the input terminal of the second convolution activation module, the input terminal of the third convolution activation module, and the input terminal of the addition module; the input terminal of the second convolution activation module is connected to the input terminal of the third convolution activation module and the input terminal of the splicing module; the input terminal of the third convolution activation module is connected to the input terminal of the splicing module.

[0029] According to another aspect of the embodiments of this application, an entropy-free image compression coding apparatus is provided, including an encoder, a decoder and a post-processing network model;

[0030] The encoder is used to perform sampling convolution on the original image to obtain sampled convolutional image data; to perform quantization processing on the sampled convolutional image data to obtain image quantized data; and to perform adaptive run-length encoding on the image quantized data to obtain image compressed data.

[0031] The decoder is used to receive image compression data from the encoder, perform run-length decoding on the image compression data, and obtain the decoded reconstructed image.

[0032] The post-processing network model is used to process the reconstructed image to obtain a post-processed optimized image; wherein, the post-processing network model includes a first convolutional layer, a downsampling convolutional layer, a GRDB module, an upsampling convolutional layer, a CBAM module, and a second convolutional layer connected in sequence.

[0033] According to another aspect of the embodiments of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the entropy-free image compression coding method described in any embodiment of this application.

[0034] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, the computer program being executed by a processor to implement the entropy-free image compression coding method described in any embodiment of this application.

[0035] One aspect of the technical solution provided in this application embodiment may include the following beneficial effects:

[0036] The entropy-free image compression coding method provided in this application performs sampling convolution on the original image to obtain sampled convolutional image data, quantizes the sampled convolutional image data to obtain quantized image data, and performs adaptive run-length encoding on the quantized image data to obtain compressed image data. By obtaining compressed image data through adaptive run-length encoding, the compression ratio is improved, and the image data compression effect is enhanced. The compression amount obtained by the encoder can meet the needs of practical applications, thus improving the following situation in related technologies: the compression amount obtained by the encoder in related technologies has an insufficient compression ratio, making it difficult to meet the needs of practical applications in terms of compression amount, and the compression ratio needs to be further improved.

[0037] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1A flowchart of an entropy-free image compression coding method according to an embodiment of this application is shown.

[0040] Figure 2 A flowchart of step S10 in one embodiment of this application is shown.

[0041] Figure 3 A flowchart of step S30 in one embodiment of this application is shown.

[0042] Figure 4 A schematic diagram of run-length encoding processing for compression is shown in one embodiment of this application.

[0043] Figure 5 A schematic diagram of adaptive run-length encoding in one embodiment of this application is shown.

[0044] Figure 6 A schematic diagram of the image compression process after incorporating adaptive run-length encoding is shown in one embodiment of this application.

[0045] Figure 7 A schematic diagram of a GRDN model in one embodiment of this application is shown.

[0046] Figure 8 A schematic diagram of the GRDB module in one embodiment of this application is shown.

[0047] Figure 9 A schematic diagram of an RDB module in one embodiment of this application is shown.

[0048] Figure 10 A schematic diagram of a CBAM module in one embodiment of this application is shown.

[0049] Figure 11 A schematic diagram of channel attention in a CBAM module according to one embodiment of this application is shown.

[0050] Figure 12 A schematic diagram of spatial attention in a CBAM module according to one embodiment of this application is shown.

[0051] Figure 13 A block diagram of an entropy-free image compression coding apparatus according to an embodiment of this application is shown.

[0052] Figure 14 A block diagram of an electronic device structure according to an embodiment of this application is shown.

[0053] Figure 15 A schematic diagram of a computer-readable storage medium according to an embodiment of this application is shown. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0055] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have a meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0056] In related image compression methods, the encoder mainly consists of two parts: transform and entropy coding. The compression ratio obtained by the encoder in these methods is not high enough, making it difficult to meet the needs of practical applications in terms of compression volume; further improvement in the compression ratio is needed. Furthermore, in these methods, entropy coding is a relatively time-consuming part of the encoder, resulting in a longer image compression time and making the efficiency of these image compression methods insufficient for practical applications.

[0057] In response to the problems existing in the relevant technologies, refer to Figure 1 As shown, one embodiment of this application provides an entropy-free image compression coding method applied to an encoder. This entropy-free image compression coding method may include steps S10 to S30:

[0058] S10. Perform sampling convolution on the original image to obtain the sampled convolutional image data.

[0059] In some examples, the encoder may include three downsampling convolutional layers and a binary mapping layer connected in series. (See reference) Figure 2 As shown, performing sampling convolution on the original image to obtain sampled convolutional image data may include steps S101 to S102:

[0060] S101, The encoder sequentially passes the original image through three downsampling convolutional layers to perform downsampling convolution processing, thereby obtaining the downsampling convolutional image data.

[0061] S102. Use a binary mapping layer to perform binary mapping on the image data after downsampling convolution to obtain the image data after sampling convolution.

[0062] S20. Quantize the sampled convolutional image data to obtain quantized image data.

[0063] In some embodiments, quantizing the sampled convolutional image data to obtain image quantized data includes: mapping the sampled convolutional image data to a uniformly distributed binary bitstream through quantization to obtain image quantized data.

[0064] Specifically, since quantization is non-differentiable, a differentiable curve is used instead of a step function during training. For each value in the compression, a corresponding specific quantization parameter is learned during training, which can further improve the compression performance of the algorithm.

[0065] In the test, binary quantization was used to obtain the binary code stream, as shown in Equation (1).

[0066]

[0067] During training, the gradient in equation (1) is discontinuous, which fails to meet the requirement that the gradient is transferable during backpropagation. Therefore, it needs to be replaced with a differentiable fitting function. The previous approach was to use the Sigmoid function for fitting, as shown in equation (2), where η is a pre-set hyperparameter.

[0068]

[0069] Different values ​​of η will have different effects. When η is larger, the quantization formula (1.2) used for training is closer to the quantization formula (1) used for testing, the output error of training and testing is smaller, which is conducive to accurate quantization.

[0070] However, a larger η also results in a larger saturation region in equation (2). The gradient is only effective very close to the binary boundary point of 0.5; gradient vanishing is likely to occur in other regions. Therefore, a larger value for η is not always better; a compromise is needed to find the most suitable value. This application conducted experiments with different values ​​of η and ultimately found that the reconstructed image quality was highest when η was 64. η less than or greater than 64 both reduced the reconstructed image quality, which is why η was set to 64 in some examples. However, considering the compression... Setting the same η for all values ​​in y may not be optimal; each value in y... i Each should correspond to a unique η i This allows for the maximization of its compression potential; therefore, setting... Given an initial value of 64 for all values, η is used as part of the model parameters and optimized during end-to-end training.

[0071] In a specific example, the encoder uses convolutional transformation to process the original image to obtain compression; the encoder then maps the compression to a uniformly distributed binary bitstream through quantization, resulting in compressed image data. The encoder transmits the compressed image data to the decoder.

[0072] The encoder takes the original image to be compressed as input and outputs the compressed data. Based on the application context of this embodiment, the encoder's design principle is extreme simplicity, retaining only the most essential parts. The downsampling convolutions are an indispensable component; in a specific example, the encoder preferably consists of only three cascaded downsampling convolutional layers (0) with a kernel length of 3. The first two downsampling convolutional layers (0) are followed by a ReLU activation function (5). The downsampling convolutional layers reduce the resolution of the feature map; each downsampling convolution halves the resolution of the feature map. Let the input original image x∈R... H×W×1 The output compression amount Here, cr is the base compression ratio, which is the ratio of x to y in the data dimensions, but it is not the final compression ratio. In order to optimize the compression amount y to a uniform distribution during training, a binary mapping is applied after the last downsampling convolution.

[0073] During the testing phase, the binary mapping is a step function with 0.5 as the dividing line, as shown in equation (1).

[0074] After the data undergoes this binary mapping, it consists of 1 bit of 01 data. Therefore, when storing or transmitting, 8 data points can occupy one byte, so the final compression ratio is cr×8.

[0075] During the training phase, since the gradient of equation (1) is discontinuous, it cannot meet the requirement that the gradient can be propagated during backpropagation. Therefore, the binary mapping is replaced by equation (2) during the training phase, where the value of η is set to 64. The purpose is to simulate the step binary mapping and maintain continuity at the breakpoint so that the gradient can be propagated back during training.

[0076] The encoder uses three downsampling convolutional kernels of length 3. The first two convolutions have 64 kernels each, and the last convolution has 64 kernels per cr. Taking a compression ratio of 32 (cr = 4) as an example, the encoder's parameter size is 46.7KB. (Image x∈R) 128×128×1 For example, the encoder's floating-point operation volume is 169.869M FLOPs, and the number of parameters and the amount of computation are relatively small.

[0077] Quantization parameter optimization only introduces additional model parameters during the training phase and has no impact on the testing phase. Therefore, this method achieves a significant improvement in reconstruction quality without increasing the number of parameters or computational cost, and without changing any processing flow.

[0078] S30. Perform adaptive run-length encoding on the image quantization data to obtain compressed image data.

[0079] Image compression data obtained through adaptive run-length encoding improves the compression ratio and enhances the image data compression effect. The compression amount obtained by the encoder can meet the needs of practical applications, thus improving the following situation in related technologies: the compression amount obtained by the encoder in related technologies is not high enough, making it difficult to meet the needs of practical applications in terms of compression amount, and the compression ratio needs to be further improved.

[0080] refer to Figure 3 As shown, in some embodiments, adaptive run-length encoding is performed on the image quantization data to obtain image compressed data, which may include steps S301 to S303:

[0081] S301. Perform expansion and re-representation processing on the image quantization data to obtain the expanded and re-represented data segment.

[0082] In some examples, the image quantization data is expanded and re-represented to obtain the expanded and re-represented data segment. This may include: expanding the image quantization data into a one-dimensional vector; traversing the one-dimensional vector and recording the number of consecutive repetitions of consecutive repeating characters in the one-dimensional vector; and using numbers to represent the number of consecutive repetitions of characters that are repeated multiple times in the quantized image data to obtain the expanded and re-represented data segment.

[0083] S302. Determine the data segments that meet the preset requirements and the data segments that do not meet the preset requirements in the expanded and re-represented data segments according to the image resolution.

[0084] S303. Perform run-length encoding on data segments that meet the preset requirements, and leave data segments that do not meet the preset requirements unchanged. Add the image resolution to the run-length encoded data to obtain image compressed data.

[0085] Specifically, the entropy of the binary compressed value y obtained by the encoder is close to 1, meaning its distribution is very close to a uniform distribution, with the number of 0 and 1 values ​​approaching equalness. There is almost no possibility of further compression from a character statistics perspective. However, its spatial continuity has not been considered. In reality, the compressed value contains many spatially correlated parts. For these parts, run-length encoding can further improve the compression ratio without loss. For the parts with low spatial correlation, the original transmission method is used, adaptively selecting the parts to be run-length encoded. Run-length encoding only requires traversing the data once, resulting in very low computational cost, which has significant advantages for image processing.

[0086] Compression continuity analysis:

[0087] Adaptive run-length encoding is performed on the quantized image data to obtain the image compression amount, including: for characters that are repeated and appear consecutively multiple times in the quantized image data, the number of times they are repeated and appear consecutively is represented by numbers.

[0088] Run-length encoding (LLE) describes repeated, consecutively occurring characters using the formula "number of consecutive occurrences, a specific character". For example, the string "AAAAABBBBCCC" can be represented as "5A4B3C" using LLE. Information that originally required 12 characters can be represented with only 6 characters, thus achieving data compression. However, LLE is not effective in all cases. For instance, the string "ABCD" can be represented as "1A1B1C1D" using LLE. Information that originally required 4 characters now requires 8 characters. Therefore, the effectiveness of LLE is directly related to the continuity of the data; it is only cost-effective when the data has good continuity.

[0089] For example, we can choose a basic model with L2 optimization and 32x compression to explore the continuity of its compression amount y. For the original graph x∈R 128×128×1 The amount of compression y∈R obtained by the encoder 16×16×16 Expand it into a one-dimensional vector y∈R 4096 Its size is 512 bytes. The compressed quantity y is processed using run-length encoding: the number of consecutive occurrences of the same character replaces the string of characters, such as... Figure 4 As shown, since we only consider character continuity here, it is not necessary to distinguish which dot represents the character 0 or 1.

[0090] There are regions where the compression amount y has very good continuity, and these regions can be further compressed using run-length encoding.

[0091] Adaptive run-length encoding design:

[0092] Since the compressed quantity y has some regions with good continuity and some regions with poor continuity, we cannot directly apply run-length encoding to all regions of the compressed quantity y without distinction. We need to find the regions with good continuity and use run-length encoding there, while for the regions with poor continuity, we still use the original bit-by-bit transmission method. This requires designing an adaptive run-length encoding scheme.

[0093] A set of characters is defined as a marker for whether run-length encoding is used. This marker cannot be exactly the same as any segment of data in the compressed amount y, otherwise it will cause misinterpretation by the decoder. However, the marker cannot be too long, otherwise it will reduce the performance of adaptive run-length encoding. If 8-bit characters are used as markers, it is relatively easy for coincidences to occur, while the probability of coincidences with 16-bit characters is extremely low, so 16-bit characters are very suitable as markers. Here, "88170" is chosen as the marker, and its binary representation is "101100010101010". Testing on five datasets revealed that none of the compressed amounts y of any image contained a character segment identical to "88170". For the compressed amount y of a 128×128 image, the longest consecutive character length in the five datasets is less than 288, so choosing 8 data bits to represent its length is appropriate. Since it is uncertain whether the longest consecutive field length of the compressed amounts of all future test images will be longer, all cases need to be considered. For the compression ratio y of a 256×256 image, the longest consecutive character in the five datasets does not exceed 2000 characters, so choosing 12 data bits to represent its length is appropriate. The adaptive run-length encoding method is as follows... Figure 5 As shown, when encountering count consecutive characters, a marker is first inserted before the character, then the character is inserted, and finally the character is replaced by 8 or 12 bits of the binary representation of count.

[0094] After adaptive run-length encoding is applied to consecutive characters, a total of 32 or 36 bits are used to represent the character segment. Therefore, in terms of encoding effectiveness, the consecutive repetition count of the character segment must be greater than 32 or 36 to use run-length encoding. If the consecutive repetition count is less than 32 or 36, the original bit-by-bit transmission method is still used.

[0095] For a 128×128 image, if the number of consecutive repetitions of a character is greater than 255, 8 characters cannot represent this data. Therefore, the first 255 characters are treated as a group for run-length encoding. Then, it is determined whether the number of consecutive repetitions is greater than 32 before deciding whether to use run-length encoding. This process is repeated, and the number of bits after run-length encoding is shown in equation (3).

[0096]

[0097] For a 256×256 image, extensive experiments have verified that the number of consecutive repetitions of a character cannot exceed 4095(2^12-1). Therefore, when the number of consecutive repetitions is less than 36, run-length encoding is not performed, and the image is transmitted directly. When the number of consecutive repetitions is greater than 36, adaptive run-length encoding is performed. The number of bits after run-length encoding is shown in equation (4).

[0098]

[0099] In summary, the encoding and decoding steps of adaptive run-length encoding in the embodiments of this application may include:

[0100] (1) Run-length encoding: Expand the compressed amount y into a one-dimensional vector, traverse it, record the number of consecutive repetitions, and select according to the image resolution, based on formula (3) or (4), to perform run-length encoding on the data segments that meet the requirements, and leave the data segments that do not meet the requirements unchanged, and also add the image resolution to the transmitted data (the amount of information is very small and can be ignored).

[0101] (2) Run-length decoding: Traverse the received data. If the marker "101100010101010" is encountered, the first 8 bits of the subsequent data are the character. Select the last 8 or 12 bits as the number of times the character is repeated consecutively based on the image resolution, and decode it accordingly. Other areas remain unchanged.

[0102] The adaptive run-length encoding used in this embodiment requires only one traversal with a time complexity of O(n), where n is the dimension of the compression amount y. It involves no data computation, has a negligible impact on the encoder's complexity, and can be seamlessly inserted into a pre-trained model, making it simple to use and effectively improving decoding performance. However, the compression ratio improvement achieved by this method is closely related to the data characteristics of the compression amount y, so the compression ratio cannot be precisely controlled. Integrating adaptive run-length encoding into the entire image compression framework follows the process described below. Figure 6 As shown.

[0103] In some embodiments, the entropy-free image compression coding method may further include:

[0104] S40. Send the image compression data to the decoder so that the decoder receives the image compression data from the encoder, performs run-length decoding on the image compression data, and obtains the decoded reconstructed image.

[0105] The entropy-free image compression coding method of this application embodiment obtains compressed image data through adaptive run-length encoding, thereby improving the compression ratio and image data compression effect. The compression amount obtained by the encoder can meet the needs of practical applications, improving the following situation in related technologies: the compression amount obtained by the encoder in related technologies is not high enough in terms of compression ratio, making it difficult to meet the needs of practical applications in terms of compression amount, and the compression ratio needs to be further improved. In addition, the entropy-free image compression coding method of this application embodiment also realizes entropy-free image compression, shortening the image compression time and improving the image compression efficiency. This improves the situation in related technologies where entropy encoding leads to long image compression time, making it difficult for related image compression methods to meet the needs of practical applications in terms of efficiency.

[0106] Another embodiment of this application provides an entropy-free image compression coding method, which may include steps 501 to 502:

[0107] Step 501: The decoder receives image compression data from the encoder, performs run-length decoding on the image compression data, and obtains the decoded reconstructed image.

[0108] Specifically, in one example, run-length decoding may include: traversing the received data, and if the marker "101100010101010" is encountered, the first 8 bits of the subsequent data are the character. Depending on the image resolution, the number of times the character is repeated consecutively in the last 8 or 12 bits is selected, and the data is decoded accordingly, while other areas remain unchanged.

[0109] Step 502: The post-processing network model processes the reconstructed image to obtain a post-processed optimized image; wherein, the post-processing network model includes a first convolutional layer, a downsampling convolutional layer, a GRDB module, an upsampling convolutional layer, a CBAM module, and a second convolutional layer connected in sequence.

[0110] The reconstructed images obtained through the decoder often suffer from compression artifacts and relatively smooth texture details, requiring further improvement. The post-processing network model, following the decoder, is both reasonable and beneficial. However, post-processing is performed on the original resolution image, resulting in significant memory consumption. Therefore, an optimal balance must be struck between acceptable memory costs and the performance gains it provides, rather than indefinitely increasing the complexity of the post-processing network model.

[0111] Post-processing network models can be applied to end-to-end image compression. However, if the post-processing network model is optimized end-to-end along with the codec, it's equivalent to integrating the post-processing network model into the decoder, which has a weaker effect on improving reconstruction results. Therefore, in this embodiment, the post-processing network model is trained separately, making this task equivalent to the downstream task after decoding: reducing image compression distortion.

[0112] In this embodiment, a GRDN (Grouped Residual Dense Network) model is used to construct the post-processing network model. The GRDN model can be used for image denoising, and its network structure is as follows: Figure 7 As shown, it consists of a first convolutional layer, a downsampling convolutional layer, three GRDB (Grouped Residual Dense Block) modules, an upsampling convolutional layer, a CBAM module, a second convolutional layer, and an addition module. The use of downsampling and upsampling convolutional layers helps to eliminate the influence of noise. The upsampling convolutional layer can be a transposed convolutional layer.

[0113] For example, the GRDB module may include a first RDB module, a second RDB module, a third RDB module, a concatenation module, a convolution module, and an addition module connected sequentially from front to back. The output terminals of the first RDB module, the second RDB module, and the third RDB module are respectively connected to the input terminal of the concatenation module; the input terminal of the first RDB module is connected to the input terminal of the addition module.

[0114] For example, any one of the first, second, and third RDB modules includes a first convolutional activation module, a second convolutional activation module, a third convolutional activation module, a splicing module, a convolution module, and an addition module connected sequentially from front to back; the input of the first convolutional activation module is connected to the input of the second convolutional activation module, the input of the third convolutional activation module, and the input of the addition module; the input of the second convolutional activation module is connected to the input of the third convolutional activation module and the input of the splicing module; the input of the third convolutional activation module is connected to the input of the splicing module.

[0115] For example, the network structure of the GRDB module is as follows: Figure 8 As shown, multi-layered information is extracted by three RDB (Residual Dense Block) modules and then fused. The network structure of the RDB module is as follows: Figure 9 As shown, it consists of dense inter-layer connections between multiple convolutional layers, borrowing the design concept of DenseNet, but replacing tensor concatenation with element-wise addition to reduce computation and memory usage. This dense connection can more effectively enhance the transfer of features.

[0116] For example, the CBAM (Convolutional Block Attention Module) is a simple yet effective attention module applied to feedforward convolutional neural networks, and its network structure is as follows: Figure 10 As shown, given a feature map, the CBAM module calculates attention weights sequentially along two independent dimensions: channels and spatial pixels. Then, it performs a weighted multiplication of the attention weight map with the input feature map for adaptive feature optimization.

[0117] For example, the channel attention mechanism in the CBAM module, such as Figure 11 As shown, similar to SENet, both compress the feature map in the spatial dimension to obtain a one-dimensional vector. The difference is that this method not only considers global average pooling, but also introduces global max pooling as a supplement. The former provides feedback to every pixel on the feature map during backpropagation, while the latter only provides feedback to the areas with the largest response in the feature map, thus more effectively extracting the importance of the channel dimension.

[0118] Spatial attention in the CBAM module, such as Figure 12 As shown, the feature map is compressed using the same average pooling and max pooling modules. The difference is that the compression is performed at the channel level, resulting in two single-channel attention weight matrices that are then concatenated and fused.

[0119] The objective function of the post-processing network model is the sum of the original image x and the post-processed image x. Distortion measurement between Using L2 loss, as shown in Equation (5), where H represents the post-processing network model, E n Represents encoder, D e Represents the decoder.

[0120]

[0121] To further improve the compression performance of entropy-free image compression algorithms, the method in this application firstly designs a post-processing quality enhancement network and trains it separately instead of end-to-end, thereby achieving a 0.1dB improvement in PSNR. Then, the quantization parameters during training are optimized, learning unique quantization parameters for each compression level. Without changing any steps in the testing phase, this achieves a 0.3–0.5dB improvement in PSNR. Finally, an adaptive run-length encoding is designed for the compression levels. Based on the compression level characteristics of images at different resolutions, different run-length encoding boundaries are used, achieving a 0.6–1dB improvement in PSNR, thus enhancing compression performance.

[0122] refer to Figure 13 As shown, another aspect of this application provides an entropy-free image compression coding apparatus, including an encoder, a decoder, and a post-processing network model;

[0123] The encoder is used to perform sampling convolution on the original image to obtain sampled convolutional image data; to perform quantization processing on the sampled convolutional image data to obtain quantized image data; and to perform adaptive run-length encoding on the quantized image data to obtain compressed image data.

[0124] The decoder is used to receive image compression data from the encoder, perform run-length decoding on the image compression data, and obtain the decoded reconstructed image.

[0125] The post-processing network model is used to process the reconstructed image to obtain a post-processed optimized image; the post-processing network model includes a first convolutional layer, a downsampling convolutional layer, a GRDB module, an upsampling convolutional layer, a CBAM module, and a second convolutional layer connected in sequence.

[0126] In some embodiments, the encoder can also be used to: perform unwrap re-representation processing on the image quantization data to obtain unwrap re-represented data segments; determine the data segments that meet the preset requirements and the data segments that do not meet the preset requirements in the unwrap re-represented data segments according to the image resolution; perform run-length encoding on the data segments that meet the preset requirements, and leave the data segments that do not meet the preset requirements unchanged, and add the image resolution to the run-length encoded data to obtain image compressed data.

[0127] In some embodiments, the encoder can also be used to: expand the image quantization data into a one-dimensional vector; traverse the one-dimensional vector and record the number of consecutive repetitions of consecutively repeating characters in the one-dimensional vector; for characters that are repeated multiple times in the quantized image data, use numbers to represent the number of consecutive repetitions to obtain the expanded and re-represented data segment.

[0128] In some embodiments, the encoder includes three downsampling convolutional layers and a binary mapping layer connected in series; the encoder can also be used to: pass the original image through the three downsampling convolutional layers in sequence to perform downsampling convolution processing to obtain downsampling convolutional image data; and use the binary mapping layer to perform binary mapping on the downsampling convolutional image data to obtain sampled convolutional image data.

[0129] In some embodiments, the encoder can also be used to: map the sampled convolutional image data to a uniformly distributed binary bitstream to obtain image quantized data.

[0130] In some embodiments, the encoder can also be used to: send image compression data to the decoder so that the decoder receives the image compression data from the encoder, performs run-length decoding on the image compression data, and obtains the decoded reconstructed image.

[0131] In some embodiments, the GRDB module includes a first RDB module, a second RDB module, a third RDB module, a splicing module, a convolution module, and an addition module connected sequentially from front to back. The output terminals of the first RDB module, the second RDB module, and the third RDB module are respectively connected to the input terminal of the splicing module; the input terminal of the first RDB module is connected to the input terminal of the addition module.

[0132] In some embodiments, any one of the first, second, and third RDB modules includes a first convolutional activation module, a second convolutional activation module, a third convolutional activation module, a splicing module, a convolution module, and an addition module connected sequentially from front to back; the input terminal of the first convolutional activation module is connected to the input terminals of the second, third, and addition modules respectively; the input terminal of the second convolutional activation module is connected to the input terminal of the third convolutional activation module and the input terminal of the splicing module respectively; and the input terminal of the third convolutional activation module is connected to the input terminal of the splicing module.

[0133] The entropy-free image compression encoding device of this application obtains image compression data through adaptive run-length encoding, thereby improving the compression ratio and image data compression effect. The compression amount obtained by the encoder can meet the needs of practical applications, improving the following situation in related technologies: the compression amount obtained by the encoder in related technologies is not high enough in terms of compression ratio, making it difficult to meet the needs of practical applications in terms of compression amount, and the compression ratio needs to be further improved. In addition, the entropy-free image compression encoding device of this application also realizes entropy-free image compression, shortening the image compression time and improving the image compression efficiency, improving the situation in related technologies where the use of entropy encoding leads to long image compression time, which is difficult to meet the needs of practical applications.

[0134] Another embodiment of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method of any of the above embodiments.

[0135] refer to Figure 14 As shown, the electronic device 10 may include: a processor 100, a memory 101, a bus 102 and a communication interface 103. The processor 100, the communication interface 103 and the memory 101 are connected through the bus 102. The memory 101 stores a computer program that can run on the processor 100. When the processor 100 runs the computer program, it executes the entropy-free image compression encoding method provided in any of the foregoing embodiments of this application.

[0136] The memory 101 may include high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 103 (wired or wireless), which can use the Internet, wide area network, local area network, metropolitan area network, etc.

[0137] Bus 102 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. Memory 101 is used to store programs. After receiving an execution instruction, processor 100 executes the program. The methods disclosed in any of the foregoing embodiments of this application can be applied to processor 100, or implemented by processor 100.

[0138] Processor 100 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 100 or by instructions in software form. The processor 100 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), an Off-the-shelf Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 101. The processor 100 reads the information in memory 101 and, in conjunction with its hardware, completes the steps of the above method.

[0139] The electronic devices and methods provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods they employ, operate, or implement.

[0140] Another embodiment of this application provides a computer-readable storage medium storing a computer program thereon, which is executed by a processor to implement the entropy-free image compression coding method of any of the above embodiments. Reference Figure 15 As shown, the computer-readable storage medium is an optical disc 20, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the methods provided in any of the aforementioned embodiments.

[0141] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0142] The computer-readable storage medium provided in the above embodiments of this application and the method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0143] It should be noted that:

[0144] The term "module" is not intended to be limited to a specific physical form. Depending on the application, a module can be implemented as hardware, firmware, software, and / or a combination thereof. Furthermore, different modules may share common components or even be implemented using the same components. Clear boundaries may or may not exist between different modules.

[0145] The algorithms and displays provided herein are not inherently related to any particular computer, virtual device, or other equipment. Various general-purpose devices can also be used with the examples based on this. The required structure for constructing such devices is obvious from the above description. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this application.

[0146] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0147] The above embodiments merely illustrate the implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method of image compression encoding without entropy coding, characterized by, Applied to the encoder, the entropy-free image compression coding method includes: The original image is sampled and convolved to obtain the sampled and convolved image data; The sampled convolutional image data is quantized to obtain quantized image data; Adaptive run-length encoding is performed on the quantized image data to obtain compressed image data; The step of performing adaptive run-length encoding on the image quantization data to obtain image compressed data includes: The image quantization data is expanded and re-represented to obtain the expanded and re-represented data segment; Based on the image resolution, determine the data segments in the expanded and re-represented data segments that meet the preset requirements and the data segments that do not meet the preset requirements; Run-length encoding is performed on the data segments that meet the preset requirements, and the data segments that do not meet the preset requirements are left unchanged. The image resolution is added to the run-length encoded data to obtain the image compressed data.

2. The image compression encoding method without entropy coding according to claim 1, characterized in that, The process of expanding and re-representing the image quantization data to obtain the expanded and re-represented data segment includes: The image quantization data is unfolded into a one-dimensional vector; Traverse the one-dimensional vector and record the number of consecutive repetitions of consecutive repeating characters in the one-dimensional vector; For characters that are repeated multiple times in the quantized image data, the number of times the characters are repeated is represented by numbers to obtain the expanded and re-represented data segment.

3. The method of claim 1, wherein The encoder includes three downsampling convolutional layers and a binary mapping layer connected in series; the process of performing sampling convolution on the original image to obtain the sampled convolutional image data includes: The encoder sequentially passes the original image through the three downsampling convolutional layers to perform downsampling convolution processing, thereby obtaining image data after downsampling convolution processing. The image data after downsampling convolution is binary mapped using a binary mapping layer to obtain the image data after sampling convolution.

4. The image compression encoding method without entropy coding according to claim 1, characterized in that, The step of quantizing the sampled convolutional image data to obtain quantized image data includes: The image data after sampling and convolution is quantized and mapped to a uniformly distributed binary bitstream to obtain image quantized data.

5. The method of claim 1, wherein, The entropy-free image compression coding method further includes: sending the image compression data to the decoder so that the decoder receives the image compression data from the encoder, performs run-length decoding on the image compression data, and obtains the decoded reconstructed image.

6. An image compression encoding method without entropy coding, characterized by, include: The decoder receives image compression data from the encoder, performs run-length decoding on the image compression data, and obtains the decoded reconstructed image; The post-processing network model processes the reconstructed image to obtain a post-processed optimized image; The post-processing network model includes a first convolutional layer, a downsampling convolutional layer, a grouped residual dense block (GRDB) module, an upsampling convolutional layer, a convolutional block attention module (CBAM), and a second convolutional layer connected in sequence. The image compressed data is obtained through the following steps: sampling convolution is performed on the original image to obtain sampled convolutional image data; the sampled convolutional image data is quantized to obtain image quantized data; and adaptive run-length encoding is performed on the image quantized data to obtain image compressed data. The step of performing adaptive run-length encoding on the image quantization data to obtain image compressed data includes: The image quantization data is expanded and re-represented to obtain the expanded and re-represented data segment; Based on the image resolution, determine the data segments in the expanded and re-represented data segments that meet the preset requirements and the data segments that do not meet the preset requirements; Run-length encoding is performed on the data segments that meet the preset requirements, and the data segments that do not meet the preset requirements are left unchanged. The image resolution is added to the run-length encoded data to obtain the image compressed data.

7. The entropy coding-free image compression encoding method according to claim 6, characterized in that, The GRDB module includes a first residual dense block RDB module, a second residual dense block RDB module, a third residual dense block RDB module, a splicing module, a convolution module, and an addition module connected sequentially from front to back. The output terminals of the first RDB module, the second RDB module, and the third RDB module are respectively connected to the input terminal of the splicing module; the input terminal of the first RDB module is connected to the input terminal of the addition module.

8. The entropy coding-free image compression encoding method according to claim 7, characterized in that, Each of the first, second, and third RDB modules includes, from front to back, a first convolutional activation module, a second convolutional activation module, a third convolutional activation module, a splicing module, a convolutional module, and an addition module; the input of the first convolutional activation module is connected to the input of the second convolutional activation module, the input of the third convolutional activation module, and the input of the addition module; the input of the second convolutional activation module is connected to the input of the third convolutional activation module and the input of the splicing module; the input of the third convolutional activation module is connected to the input of the splicing module.

9. An image compression encoding apparatus without entropy coding, characterized by, Includes encoder, decoder, and post-processing network models; The encoder is used to perform sampling convolution on the original image to obtain sampled convolutional image data; to perform quantization processing on the sampled convolutional image data to obtain image quantized data; and to perform adaptive run-length encoding on the image quantized data to obtain image compressed data. The decoder is used to receive image compression data from the encoder, perform run-length decoding on the image compression data, and obtain the decoded reconstructed image. The post-processing network model is used to process the reconstructed image to obtain a post-processed optimized image; wherein, the post-processing network model includes a first convolutional layer, a downsampling convolutional layer, a grouped residual dense block (GRDB) module, an upsampling convolutional layer, a convolutional block attention module (CBAM), and a second convolutional layer connected in sequence. The encoder is further configured to: perform unwrap re-representation processing on the image quantization data to obtain unwrap re-represented data segments; determine the data segments that meet preset requirements and the data segments that do not meet preset requirements in the unwrap re-represented data segments according to the image resolution; perform run-length encoding on the data segments that meet preset requirements, and leave the data segments that do not meet preset requirements unchanged, and add the image resolution to the run-length encoded data to obtain the image compressed data.

10. An electronic device, comprising: A computer program product comprising a memory, a processor, and a computer program stored on the memory and loadable on the processor, the processor executing the computer program to implement the entropy coding-free image compression encoding method of any one of claims 1-8.

11. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the entropy coding-free image compression encoding method of any one of claims 1-8.

Images