Image transmission method and device under extremely low bit rate, equipment and storage medium

Abstract

The invention relates to the technical field of image transmission processing, in particular to an image transmission method and device under an extremely low bit rate, equipment and a storage medium, and the method comprises the steps: carrying out the quantization coding of an obtained to-be-compressed image, and obtaining an initial index set; performing compression coding on the to-be-compressed image to obtain continuous features; obtaining a target reconstruction index based on the initial index set and the continuous features; and obtaining a reconstructed image based on the target reconstruction index and the continuous features. According to the method and the device, the problems that image details of decoded images are seriously lost and the image reconstruction quality is obviously reduced under the condition of extremely low bit rate transmission caused by bandwidth limitation are solved.

Description

Technical Field

[0001] The present application relates to the technical field of image transmission and processing, and in particular to an image transmission method, device, equipment and storage medium at an extremely low bit rate. Background Art

[0002] With the popularity of high-resolution image formats such as 4K and 8K, many real-time applications, such as video conferencing, telemedicine, and drone video surveillance, need to transmit high-quality videos under limited bandwidth. Limited bandwidth often requires real-time transmission of videos at extremely low bit rates.

[0003] At present, the method of real-time transmission of video at an extremely low bit rate is: collecting traditional video coding standards such as H.264 and HEVC, compressing and encoding the video frame images in the video to obtain encoded data, or using a deep learning model to capture the complex correlation of video frame images in the video, and then compressing and encoding the video frame images according to the complex correlation to obtain encoded data; then, the encoded data is transmitted to the corresponding receiving device for decoding, thereby realizing image transmission at an extremely low bit rate.

[0004] However, judging from the actual transmission results, under the extremely low bit rate transmission condition caused by bandwidth limitation, the decoded image still suffers from serious loss of image details and significant degradation of image reconstruction quality. Summary of the invention

[0005] In order to improve the problem of severe loss of image details and significant degradation of image reconstruction quality in decoded images under extremely low bit rate transmission caused by limited bandwidth, the present application provides an image transmission method, apparatus, device and storage medium at an extremely low bit rate.

[0006] In a first aspect, the present application provides an image transmission method at an extremely low bit rate, comprising:

[0007] Performing quantization encoding on the acquired image to be compressed to obtain an initial index set;

[0008] Performing compression encoding on the image to be compressed to obtain continuous features;

[0009] Obtaining a target reconstruction index based on the initial index set and the continuous features;

[0010] A reconstructed image is obtained based on the target reconstruction index and the continuous features.

[0011] In a second aspect, the present application provides an image transmission device at an extremely low bit rate, comprising:

[0012] A quantization encoding module, used for performing quantization encoding on the acquired image to be compressed to obtain an initial index set;

[0013] A compression coding module, used for performing compression coding on the image to be compressed to obtain continuous features;

[0014] An index recovery module, used to obtain a target reconstructed index based on the initial index set and the continuous features;

[0015] The image reconstruction module is used to obtain a reconstructed image based on the target reconstruction index and the continuous features.

[0016] In a third aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method when executing the computer program.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0018] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0019] The above-mentioned image transmission method, device, equipment and storage medium at extremely low bit rate can convert high-dimensional image data into low-dimensional discrete indexes by quantizing and encoding the acquired image to be compressed, reduce the amount of data by vector quantization, and lay a foundation for subsequent compression and transmission; compress and encode the image to be compressed to obtain continuous features, which can retain the detailed texture and structural information of the image. These continuous features provide necessary supplementary information for high-fidelity reconstruction of the image, especially at extremely low bit rate, continuous features can help restore details and texture information that may be lost in discrete streams; based on the initial index The target reconstruction index is obtained by combining the set and the continuous features. The information in the continuous feature stream can be used to predict and reconstruct the untransmitted index part. This step is the key to improving compression and transmission efficiency because it allows the system to maintain the integrity and accuracy of image reconstruction while transmitting very small amounts of data. The reconstructed image is obtained based on the target reconstruction index and the continuous features, and the decoding result based on the codebook can be optimized and adjusted to improve the quality of the decoded image. The above scheme is convenient for improving the problem of severe loss of image details and significant reduction in image reconstruction quality in the decoded image under extremely low bit rate transmission caused by bandwidth limitation.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A flow chart of an image transmission method at an extremely low bit rate provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of the structure of an image transmission device at an extremely low bit rate provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0025] Figure 4 This is a diagram of the internal structure of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of this article described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0028] In this article, the term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship.

[0029] Embodiment 1

[0030] Figure 1 This is a flowchart of an image transmission method at an extremely low bit rate provided in Example 1 of the present application, refer to Figure 1 The method may be performed by a device for performing the method, and the device may be implemented by software and / or hardware. The method includes:

[0031] S110, performing quantization encoding on the acquired image to be compressed to obtain an initial index set.

[0032] Among them, with the popularity of high-resolution image formats, the resolution of videos transmitted by many real-time video transmission software is also improving. For example, the resolution of some important videos transmitted by real-time video transmission software may reach 4K or 8K, among which important videos include telemedicine videos, drone monitoring videos, etc. The video to be transmitted is composed of multiple video frames. When the transmission bandwidth is limited, in order to improve the transmission efficiency, the video frames need to be compressed first and then transmitted. The video frame is a frame of image in the corresponding video, and the video frame to be compressed is recorded as the image to be compressed. In other embodiments, the image to be compressed may also be an image that needs to be compressed before transmission, which is not specifically limited.

[0033] It should be noted that, in order to retain the main visual features and patterns of the image to be compressed after compression, it is necessary to first quantize and encode each image pixel in the image to be compressed, so as to obtain an initial index corresponding to each image pixel one by one. The initial index is discrete data and exists in a preset codebook. Some clustering algorithms are usually used, such as the Lloyd algorithm (also known as a variant of the K-means algorithm), to cluster a large number of training vectors, each cluster center is an initial index, and each initial index constitutes a complete codebook; by quantizing and encoding each image pixel in the image to be compressed, an initial index corresponding to each image pixel in the codebook can be determined.

[0034] The initial index corresponding to each image pixel may include the main visual features and patterns of the image to be compressed. By retaining the main visual features and patterns of the image to be compressed, a reconstruction basis can be provided for the subsequent reconstruction of the compressed image to be compressed, and the reconstruction quality of the reconstructed image can be guaranteed. Taking an image pixel in the image to be compressed as an example, the result obtained after quantization encoding of the image pixel is recorded as the initial index, and the set of initial indexes corresponding to each image pixel is recorded as the initial index set.

[0035] S120: compress and encode the image to be compressed to obtain continuous features.

[0036] The compression coding means first reducing the data volume of the image to be compressed to obtain a low-data-volume image, and then encoding the low-data-volume image, and recording the result obtained by encoding the low-data-volume image as a continuous feature.

[0037] It should be noted that by reducing the data volume of the image to be compressed first, it is convenient to reduce the amount of data required to encode the low-data-volume image without encoding the entire image to be compressed, which not only facilitates improving the efficiency of compression coding, but also makes the continuous features obtained by compression coding of the image to be compressed more suitable for transmission under limited transmission bandwidth. In addition, the continuous representation obtained by compression coding of the image to be compressed contains the main visual content of the image to be compressed, which is used for subsequent image reconstruction.

[0038] S130: Obtain a target reconstruction index based on the initial index set and the continuous features.

[0039] Among them, in this embodiment, a deep learning model for processing the initial index set and the continuous feature is preset. The deep learning model can map the initial index set and the continuous feature into corresponding target reconstruction indexes. The target reconstruction indexes are used to be transmitted to the image receiving end so that the image receiving end can decode the target reconstruction indexes, thereby reconstructing a decompressed image corresponding to the image to be compressed.

[0040] It should be noted that the target reconstruction index is used to retrieve the corresponding codeword from the codebook. The codeword is encoded and stored in the compression stage and is used to reconstruct the key visual content of the image.

[0041] S140: Obtain a reconstructed image based on the target reconstruction index and the continuous features.

[0042] Among them, the target reconstruction index corresponds to the initial index set and the continuous features. The initial index set contains the main visual features and patterns of the image to be compressed, and the continuous features contain the main visual content of the image to be compressed. After receiving the target reconstruction index, the image receiving end further obtains the continuous features sent by the image sending end, and then reconstructs the image through the target reconstruction index and the continuous features, and the image obtained after image reconstruction is recorded as the reconstructed image.

[0043] It should be noted that the reconstructed image not only contains the main visual features and patterns of the image to be compressed, but also contains the main visual content of the image to be compressed, that is, the reconstructed image contains the main key features and details of the image to be compressed and has a very high reconstruction quality.

[0044] It should also be noted that, in this embodiment, by quantizing and encoding the acquired image to be compressed to obtain an initial index set, high-dimensional image data can be converted into discrete indexes of lower dimensions, and the amount of data can be reduced by vector quantization, laying the foundation for subsequent compression and transmission; the image to be compressed is compressed and encoded to obtain continuous features, which can retain the detailed texture and structural information of the image. These continuous features provide necessary supplementary information for high-fidelity reconstruction of the image, especially at extremely low bit rates, continuous features can help restore details and texture information that may be lost in discrete streams; based on the initial index set and the continuous features, a target reconstruction index is obtained, and the information in the continuous feature stream can be used to predict and reconstruct the untransmitted index part. This step is the key to improving compression and transmission efficiency, because it allows the system to maintain the integrity and accuracy of image reconstruction while transmitting a very small amount of data. Based on the target reconstruction index and the continuous features, a reconstructed image is obtained, which can optimize and adjust the decoding result based on the codebook and improve the quality of the decoded image; the above scheme is convenient for improving the problem of severe loss of image details and significant reduction in image reconstruction quality in the decoded image under extremely low bit rate transmission conditions caused by bandwidth limitation.

[0045] Embodiment 2

[0046] Embodiment 2 of the present application provides an image transmission method at an extremely low bit rate, which optimizes the "quantization encoding of the acquired image to be compressed to obtain an initial index set" in Embodiment 1; it should be noted that for the parts not described in detail in this embodiment, reference may be made to the descriptions of other embodiments, and the method includes:

[0047] S211 , performing low-dimensional mapping on the acquired image pixels to be compressed to obtain a continuous representation.

[0048] Among them, in this embodiment, the form of quantization encoding for the image to be compressed is specifically vector quantization encoding; the image pixels of the image to be compressed are high-dimensional data. In order to facilitate the implementation of vector quantization encoding for the image pixels of the compressed image, it is necessary to first perform low-dimensional mapping on the high-dimensional image pixels, that is, to map the high-dimensional image pixels to low-dimensional data.

[0049] Specifically, this embodiment uses a vector quantization encoder (VQ-Encoder) to perform low-dimensional mapping on high-dimensional image pixels. The vector quantization encoder is a special neural network used to map high-dimensional data (such as the above-mentioned image pixel value) to a low-dimensional, finite codebook; taking an image pixel in the image to be compressed as an example, the image pixel is input into the vector quantization encoder (VQ-Encoder) for processing to obtain low-dimensional data corresponding to the image pixel one by one, and the low-dimensional data is recorded as a continuous representation z, and the continuous representation z represents the important features in the corresponding image to be compressed; , VQ-Encoder represents the corresponding low-dimensional mapping function in the vector quantization encoder, and x is the image pixel in the image to be compressed.

[0050] S212: Calculate the distance between the continuous representation and each codeword in a preset codebook to obtain a distance set.

[0051] The steps of generating the preset codebook include: usually using some clustering algorithms, such as the Lloyd algorithm (also known as a variant of the K-means algorithm), clustering a large number of training vectors, and the cluster center of each cluster is a codeword , collect all codewords Constitute a complete codebook, where i represents the i-th codeword in the codebook.

[0052] Specifically, the continuous representation z and the codeword The distance between the codewords is preferably the Euclidean distance in this embodiment, and the continuous representation z and the codeword The distance between In other embodiments, the continuous representation z and the codeword There is no specific limitation on the form of the distance between them.

[0053] The codebook contains multiple codewords, and the continuous representation z is calculated by comparing it with the codewords in the codebook. The distance between each codeword can be obtained One-to-one corresponding distances, and the set of each distance is recorded as a distance set.

[0054] S213: Obtain an initial index set based on the codebook and the distance set corresponding to each of the image pixels.

[0055] By implementing steps S211-S212, a distance set corresponding to each image pixel can be calculated, and the distance set includes multiple Euclidean distances.

[0056] Specifically, taking one of the image pixels as an example, after the distance set corresponding to the image pixel is calculated through the implementation of steps S211-S212, the smallest Euclidean distance in the distance set is further screened out, and the smallest Euclidean distance is used as the initial index corresponding to the image pixel; the initial index corresponding to each image pixel can be calculated through the above steps, and each initial index is collectively referred to as an initial index set.

[0057] It should be noted that the initial index corresponding to each image pixel is calculated through step S213, and the initial index is discrete, so that the mapping from the continuous representation z to the discrete space is realized, and the initial index is mapped. The initial index not only has a small amount of data, which is convenient for transmission in a bandwidth-constrained environment, but also can be used to reconstruct a high-quality reconstructed image later.

[0058] In another embodiment, the specific step of performing quantization coding on the acquired image to be compressed to obtain the initial index set may also be: performing adaptive quantization coding and / or hybrid quantization coding on the acquired image to be compressed to obtain the initial index set.

[0059] S220: compress and encode the image to be compressed to obtain continuous features.

[0060] S230: Obtain a target reconstruction index based on the initial index set and the continuous features.

[0061] S240: Obtain a reconstructed image based on the target reconstruction index and the continuous features.

[0062] It should be noted that this application adopts an innovative dual-stream compression framework, combining a discrete codebook stream (initial index set) and a continuous feature stream (continuous features). The discrete codebook stream is used to process the main structure and content of the image, while the continuous feature stream is used to maintain image details and texture information, especially in extremely low bit rate environments. This method improves the quality of image reconstruction, especially at extremely low bit rates, and can better maintain image details and reduce distortion, significantly improving the image compression effect compared to existing technologies.

[0063] Embodiment 3

[0064] Embodiment 3 of the present application provides an image transmission method at an extremely low bit rate, which optimizes the "compressing and encoding the image to be compressed to obtain continuous features" in Embodiment 1; it should be noted that for the parts not described in detail in this embodiment, reference may be made to the descriptions of other embodiments, and the method includes:

[0065] S310: Perform quantization encoding on the acquired image to be compressed to obtain an initial index set.

[0066] S321. Downsample the image to be compressed to obtain a downsampled image.

[0067] Among them, the image resolution of the image to be compressed is relatively high, resulting in a relatively high data volume of the image to be compressed. In order to reduce the data volume of the image to be compressed, so as to reduce the amount of data to be processed when the image to be compressed is subsequently compressed and encoded, so that the data obtained after compression and encoding can be conveniently transmitted in an environment with limited bandwidth. To this end, this embodiment downsamples the image to be compressed to reduce the image size of the image to be compressed (that is, to reduce the data volume of the image to be compressed) and retain important visual information in the image to be compressed.

[0068] Specifically, in this implementation, Gaussian blur is performed on the compressed image to achieve downsampling of the compressed image; in another embodiment, pooling operation (average pooling or maximum pooling) can be performed on the compressed image to achieve downsampling of the compressed image; in other embodiments, there is no specific limitation.

[0069] The image obtained after downsampling the image to be compressed is recorded as a downsampled image.

[0070] S322: Process the downsampled image based on a preset low bit rate image compression encoder to obtain continuous features.

[0071] Among them, the low bit rate image compression coder (MLIC) is a specially designed coder that optimizes parameters to adapt to the reconstruction quality of compressed images at very low bit rates; the coder uses a set of training parameters (theta) that are optimized during the training process to maximize the data compression rate while minimizing information loss; the low bit rate image compression coder (MLIC) is used to process downsampled images and output the corresponding continuous features y, continuous features , where X is the image to be compressed, is the downsampled image obtained by downsampling the image to be compressed, theta is the parameter set learned by the low bit rate image compression coder (MLIC) during the training process, which determines the behavior and performance of the low bit rate image compression coder (MLIC), including how to effectively extract and encode information from the input downsampled image. The continuous feature y is the continuous potential feature obtained by the low bit rate image compression coder (MLIC) after processing the downsampled image. The continuous feature y is part of the data stream during the image transmission process from the image sender to the image receiver, which retains the important visual and structural information of the image to be compressed.

[0072] In another embodiment, the specific steps of compressing and encoding the image to be compressed to obtain continuous features may also be: determining the importance and visual sensitivity of the image content in the image to be compressed, determining the compression rate of the image area where the image content is located based on the importance and visual sensitivity of the image content, and compressing and encoding the image to be compressed based on the compression rate to obtain continuous features.

[0073] S330: Obtain a target reconstruction index based on the initial index set and the continuous features.

[0074] S340: Obtain a reconstructed image based on the target reconstruction index and the continuous features.

[0075] Embodiment 4

[0076] Embodiment 4 of the present application provides an image transmission method at an extremely low bit rate, which optimizes the "obtaining a target reconstruction index based on the initial index set and the continuous features" in Embodiment 1; it should be noted that for the parts not described in detail in this embodiment, reference may be made to the descriptions of other embodiments, and the method includes:

[0077] S410: Perform quantization encoding on the acquired image to be compressed to obtain an initial index set.

[0078] S420: compress and encode the image to be compressed to obtain continuous features.

[0079] S431, selectively shielding the initial indexes in the initial index set to obtain a target index set;

[0080] Among them, the data volume of the initial index set is still relatively large when the bandwidth is limited. If the initial index set is transmitted through the limited bandwidth, the transmission efficiency will be reduced. Therefore, the data volume of the initial index set needs to be reduced before transmission. This embodiment adopts a method of selectively shielding the initial indexes in the initial index set to screen out some initial indexes from the initial index set, thereby reducing the data volume of the initial index set.

[0081] Specifically, in this embodiment, a preset masking ratio p is used to randomly mask some of the initial indexes in the initial index set, wherein the masking ratio p=the number of randomly masked initial indexes in the initial index set / the number of all initial indexes in the initial index set; in another embodiment, some of the initial indexes in the initial index set may be masked according to a preset masking mode (such as p=1 / 4 masking); in other embodiments, there is no specific limitation;

[0082] Among them, the new initial index set formed after the initial index set is selectively masked is recorded as the target index set , , where d is the initial index set and p is the masking ratio.

[0083] S432. Obtain a target reconstruction index based on the target index set, the continuous features and a preset deep learning model.

[0084] Among them, the preset deep learning model Transformer is used to generate the target index set Predict with continuous feature y to predict the target index set The target reconstruction index corresponding to the continuous feature y , target reindex Used to restore complete image information during subsequent decoding; , where phi is the parameter of the deep learning model, which is optimized during the training process so that the deep learning model can effectively index the target set And the complete index is restored from the continuous feature y, that is, the target reconstruction index.

[0085] S440: Obtain a reconstructed image based on the target reconstruction index and the continuous features.

[0086] Embodiment 5

[0087] Embodiment 5 of the present application provides a method for transmitting an image at an extremely low bit rate, which optimizes the "obtaining a reconstructed image based on the target reconstruction index and the continuous feature" in Embodiment 1; it should be noted that for the part not described in detail in this embodiment, reference may be made to the description of other embodiments, and the method includes:

[0088] S510: Perform quantization encoding on the acquired image to be compressed to obtain an initial index set.

[0089] S520: compress and encode the image to be compressed to obtain continuous features.

[0090] S530: Obtain a target reconstruction index based on the initial index set and the continuous features.

[0091] S541. Decode based on the target reconstructed index to obtain a decoding result.

[0092] Among them, the target reconstruction index is used to retrieve the corresponding codeword from a preset codebook, and the codeword corresponding to the target reconstruction index is a key feature for reconstructing the image to be compressed; in this embodiment, a preset decoder VQ-Decoder is used for decoding, and the decoder VQ-Decoder is used to process the target reconstruction index to obtain a decoding result, and the decoding result is a preliminary reconstructed image.

[0093] Specifically, the decoder VQ-Decoder retrieves the codeword corresponding to the target reconstruction index from the preset codebook, and constructs the decoding result based on the retrieved codeword , where d_full is the target reconstruction index and gamma is the parameter of VQ-Decoder, which determines the specific implementation details of the decoding process, such as the activation function, the depth and width of the network layer, etc.

[0094] S542: Obtain a reconstructed image based on the decoding result and the continuous feature.

[0095] It should be noted that the decoding result is a preliminarily reconstructed image. When the transmission bandwidth is limited, there is a certain difference between the decoding result and the corresponding image to be compressed, which results in the image quality of the preliminarily reconstructed image to be improved.

[0096] Among them, the continuous feature y retains important visual and structural information of the image to be compressed, and the decoding result can be further optimized through the continuous feature y to obtain an image that is closer to the corresponding image to be compressed; and the image obtained after the continuous feature y optimizes the decoding result is recorded as the reconstructed image.

[0097] Embodiment 6

[0098] Embodiment 6 of the present application provides a method for transmitting an image at an extremely low bit rate, which optimizes the "obtaining a reconstructed image based on the decoding result and the continuous feature" in Embodiment 5; it should be noted that for the part not described in detail in this embodiment, reference may be made to the description of other embodiments, and the method includes:

[0099] S610: Perform quantization encoding on the acquired image to be compressed to obtain an initial index set.

[0100] S620: compress and encode the image to be compressed to obtain continuous features.

[0101] S630: Obtain a target reconstruction index based on the initial index set and the continuous features.

[0102] S641. Decode based on the target reconstructed index to obtain a decoding result.

[0103] S642A, obtaining a correction result based on the decoding result, the continuous feature and a preset correction network.

[0104] Among them, there is a certain difference between the decoding result obtained by decoding based on the target reconstruction index and the corresponding image to be compressed. In order to calculate the difference, this embodiment presets a correction network Correction, which is used to process the decoding result And the continuous feature y, so as to obtain the corresponding correction result , where psi is the parameter of the correction network, which is optimized during the network training process to most effectively combine continuous features and codebook features to improve the quality of the decoded image; the correction result is used to characterize the difference between the decoding result and the corresponding image to be compressed.

[0105] S642B, obtaining a reconstructed image based on the correction result and the decoding result.

[0106] Among them, the reconstructed image X_recon is the correction result With the decoding result The sum of + .

[0107] It should be noted that the decoder ensures high-quality image reconstruction even at very low bit rates by integrating discrete codebook features and continuous image features, as well as adjusting the correction network. The advantage of the method shown in this implementation is that it can utilize the efficient compression capability of the codebook and the detailed information of the continuous features, and significantly improve the visual quality and fidelity of the decoded image through fine correction processing. In addition, the parameterized processing method (through gamma and psi) allows the decoder and correction network to flexibly respond to different image content and compression requirements, thereby achieving efficient data compression while maintaining image quality.

[0108] Existing image compression technologies (including deep learning-based methods) often have difficulty maintaining good image reconstruction quality at extremely low bit rates, which can easily lead to loss of image details and texture. By combining discrete codebook streams and continuous feature streams, this application can effectively maintain image details and texture information at extremely low bit rates. The discrete codebook stream provides efficient encoding of the structure and main content of the image, while the continuous feature stream supplements the detail information, together achieving higher quality image reconstruction.

[0109] Many compression algorithms based on deep learning have high computational complexity during compression and decompression, resulting in slow processing speed and are not suitable for real-time applications. This application optimizes data transmission by introducing a shielded predictor, which only requires transmitting part of the index and uses a continuous feature stream to predict the untransmitted index part, significantly reducing the amount of processed data and improving the processing speed, making the compression and decompression process more efficient and suitable for real-time or resource-constrained applications.

[0110] Existing deep learning models have high requirements for computing resources and relatively high energy consumption, which limits their application on mobile devices. This application significantly reduces the demand for computing resources and energy consumption by optimizing algorithms and simplifying network structures. This makes this application more suitable for deployment on devices with limited computing power, broadening its application areas.

[0111] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0112] Embodiment 7

[0113] Based on the same inventive concept, this embodiment also provides an extremely low bit rate image transmission device for implementing the above-mentioned image transmission method at an extremely low bit rate. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more embodiments of the image transmission device at an extremely low bit rate provided below can refer to the limitations of the image transmission method at an extremely low bit rate above, and will not be repeated here.

[0114] In this embodiment, Figure 2 As shown, a device for transmitting an image at an extremely low bit rate is provided, comprising:

[0115] A quantization encoding module, used for performing quantization encoding on the acquired image to be compressed to obtain an initial index set;

[0116] A compression coding module, used for performing compression coding on the image to be compressed to obtain continuous features;

[0117] An index recovery module, used to obtain a target reconstructed index based on the initial index set and the continuous features;

[0118] The image reconstruction module is used to obtain a reconstructed image based on the target reconstruction index and the continuous features.

[0119] Each module in the above-mentioned image transmission device at an extremely low bit rate can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0120] It should be noted that, in this embodiment, by quantizing and encoding the acquired image to be compressed to obtain an initial index set, high-dimensional image data can be converted into discrete indexes of lower dimensions, and the amount of data can be reduced by vector quantization, laying the foundation for subsequent compression and transmission; the image to be compressed is compressed and encoded to obtain continuous features, which can retain the detailed texture and structural information of the image. These continuous features provide necessary supplementary information for high-fidelity reconstruction of the image, especially at extremely low bit rates, continuous features can help restore details and texture information that may be lost in discrete streams; based on the initial index set and the continuous features, a target reconstruction index is obtained, and the information in the continuous feature stream can be used to predict and reconstruct the untransmitted index part. This step is the key to improving compression and transmission efficiency, because it allows the system to maintain the integrity and accuracy of image reconstruction while transmitting a very small amount of data. Based on the target reconstruction index and the continuous features, a reconstructed image is obtained, which can optimize and adjust the decoding result based on the codebook and improve the quality of the decoded image; the above scheme is convenient for improving the problem of severe loss of image details and significant reduction in image reconstruction quality in the decoded image under extremely low bit rate transmission conditions caused by bandwidth limitation.

[0121] In one embodiment, in performing quantization encoding on the acquired image to be compressed to obtain an initial index set, the quantization encoding module is specifically used to:

[0122] Performing low-dimensional mapping on the image pixels of the acquired image to be compressed to obtain a continuous representation;

[0123] Calculating the distance between the continuous representation and each codeword in a preset codebook to obtain a distance set;

[0124] An initial index set is obtained based on the distance set corresponding to the codebook and each of the image pixels.

[0125] In one embodiment, in terms of compressing and encoding the image to be compressed to obtain continuous features, the compression encoding module is specifically used to:

[0126] Downsampling the image to be compressed to obtain a downsampled image;

[0127] The downsampled image is processed based on a preset low bit rate image compression encoder to obtain continuous features.

[0128] In one embodiment, in obtaining a target reconstructed index based on the initial index set and the continuous feature, the index recovery module is specifically used to:

[0129] Selectively shielding initial indexes in the initial index set to obtain a target index set;

[0130] Based on the target index set, the continuous features and a preset deep learning model, a target reconstruction index is obtained.

[0131] In one embodiment, in obtaining a reconstructed image based on the target reconstruction index and the continuous feature, the image reconstruction module is specifically used to:

[0132] Decoding is performed based on the target reconstructed index to obtain a decoding result;

[0133] A reconstructed image is obtained based on the decoding result and the continuous features.

[0134] In one embodiment, in obtaining a reconstructed image based on the decoding result and the continuous feature, the image reconstruction module is specifically used to:

[0135] Obtaining a correction result based on the decoding result, the continuous feature and a preset correction network;

[0136] A reconstructed image is obtained based on the correction result and the decoding result.

[0137] Embodiment 8

[0138] In this embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for image transmission at an extremely low bit rate is implemented.

[0139] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present disclosure, and does not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] Embodiment 9

[0141] In this embodiment, a computer readable storage medium is provided. Figure 4 As shown, a computer program is stored thereon, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0142] Embodiment 10

[0143] In this embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0144] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0145] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present disclosure may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided by the present disclosure may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but are not limited to this.

[0146] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] The above-described embodiments only express several implementation methods of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the attached claims.

Claims

1. A method for transmitting an image at an extremely low bit rate, characterized in that: include: Performing quantization encoding on the acquired image to be compressed to obtain an initial index set; Performing compression encoding on the image to be compressed to obtain continuous features; Obtaining a target reconstruction index based on the initial index set and the continuous features; A reconstructed image is obtained based on the target reconstruction index and the continuous features.

2. The method according to claim 1, characterized in that The step of quantizing and encoding the acquired image to be compressed to obtain an initial index set includes: Performing low-dimensional mapping on the image pixels of the acquired image to be compressed to obtain a continuous representation; Calculating the distance between the continuous representation and each codeword in a preset codebook to obtain a distance set; An initial index set is obtained based on the distance set corresponding to the codebook and each of the image pixels.

3. The method according to claim 1, characterized in that The step of compressing and encoding the image to be compressed to obtain continuous features comprises: Downsampling the image to be compressed to obtain a downsampled image; The downsampled image is processed based on a preset low bit rate image compression encoder to obtain continuous features.

4. The method according to claim 1, characterized in that: The obtaining a target reconstruction index based on the initial index set and the continuous features includes: Selectively shielding initial indexes in the initial index set to obtain a target index set; Based on the target index set, the continuous features and a preset deep learning model, a target reconstruction index is obtained.

5. The method according to claim 1, characterized in that The step of obtaining a reconstructed image based on the target reconstruction index and the continuous features includes: Decoding is performed based on the target reconstructed index to obtain a decoding result; A reconstructed image is obtained based on the decoding result and the continuous features.

6. The method according to claim 5, characterized in that The step of obtaining a reconstructed image based on the decoding result and the continuous feature comprises: Obtaining a correction result based on the decoding result, the continuous feature and a preset correction network; A reconstructed image is obtained based on the correction result and the decoding result.

7. An image transmission device at an extremely low bit rate, characterized in that: The device comprises: A quantization encoding module, used for performing quantization encoding on the acquired image to be compressed to obtain an initial index set; A compression coding module, used for performing compression coding on the image to be compressed to obtain continuous features; An index recovery module, used to obtain a target reconstructed index based on the initial index set and the continuous features; The image reconstruction module is used to obtain a reconstructed image based on the target reconstruction index and the continuous features.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images