Method, apparatus and recording medium for encoding / decoding image

Abstract

In the embodiments, a learning-based image compression technique is disclosed that improves encoding efficiency by integrating a Channel Attention (CA) block and a Probabilistic Principal Component Analysis (PPCA) block as core elements. PPCA can emphasize important features and effectively process noise. Unlike conventional Principal Component Analysis (PCA), PPCA can better preserve meaningful data structures by using a probabilistic approach. The CA mechanism can emphasize important image features by preferentially processing key pixel values, and can minimize information of low importance while maintaining essential information during the compression process.

Description

Technology Field

[0001] The present invention relates to a method, apparatus, and recording medium for image encoding / decoding. Specifically, it relates to a method, apparatus, and recording medium for end-to-end image processing. Background Technology

[0002] With the rapid growth of digital media and the increasing demand for high-resolution content, there is a growing need for more efficient compression and transmission methods.

[0003] Data compression technology is essential for optimizing storage space and information transmission.

[0004] The primary purpose of data compression is to reduce the file size representing the data, and in particular, image compression is achieved by removing unnecessary information while preserving important details.

[0005] Image Signal Processing (ISP) is the process of converting raw data into a format, such as an RGB image, that can be perceived by the human visual system. This process plays an important role in improving image quality through methods such as noise reduction and color adjustment.

[0006] One of the widely used techniques in such signal processing is compression, and the Discrete Cosine Transform (DCT) is one of the representative traditional compression methods.

[0007] DCT is widely used in image compression standards such as the Joint Photographic Experts Group (JPEG) and the Moving Picture Experts Group (MPEG).

[0008] DCT converts images into frequency components, saving storage space by retaining important frequencies and removing less important ones.

[0009] However, this method has several drawbacks. Examples include the occurrence of blocking artifacts, inefficiency in low-bitrate environments, sensitivity to gradient changes, and a lack of optimal performance in lossless compression. Additionally, JPEG's lossy compression method results in information loss, and due to this information loss, the reconstructed image may look significantly different from the original.

[0010] In recently developed technologies, the latest compression techniques utilizing deep learning, particularly Convolutional Neural Networks (CNNs), have been introduced, and the compression process is performed more efficiently by reducing information redundancy within images through deep learning.

[0011] This deep learning-based approach can save bandwidth and storage space in various applications while maintaining image quality.

[0012] In recent years, deep learning has made significant advancements in image compression performance compared to existing traditional image compression methods. While methods based on existing standards are still in use, they have limitations in handling repetitive patterns and complex calculations, which can cause problems during the image reconstruction process. The problem to be solved

[0013] One embodiment may provide an apparatus, a method, and a recording medium for performing end-to-end image compression.

[0014] One embodiment may provide an apparatus, a method, and a recording medium for performing end-to-end image compression utilizing stochastic principal component analysis and channel attention. means of solving the problem

[0015] In one aspect, a video encoding method is provided, comprising: a step of generating a bitstream containing encoded information of a video; and a step of transmitting said bitstream, wherein the bitstream includes information about a potential representation and a hyperprior.

[0016] In another aspect, an image encoding method performed by an image encoding device may include: a step of acquiring an input image; a step of extracting a feature map from the input image through an encoder network; a step of applying a Channel Attention (CA) block to the feature map to analyze the correlation between channels and emphasize the main feature channels; a step of applying a Probabilistic Principal Component Analysis (PPCA) block to the output of the CA block to perform probabilistic dimensionality reduction based on a Gaussian noise model; and a step of entropy encoding a latent representation generated from the features passed through the PPCA block to generate a bitstream.

[0017] The above PPCA block can perform uncorrelation of feature data in a direction that maximizes marginal likelihood based on a probabilistic generating model in which the observed data is defined as a combination of low-dimensional latent variables, a projection matrix, and Gaussian noise.

[0018] The above CA block can select visually important information by performing global mean pooling and global maximum pooling in parallel on the feature map and applying channel-specific weight vectors derived through a shared multilayer perceptron and a sigmoid function to the original feature map.

[0019] The above PPCA block can rearrange information in the order of principal components with large variance by utilizing energy compression characteristics during the principal component analysis process, and can improve robustness against data loss occurring during compression through a probabilistic approach.

[0020] The encoder network above may include four convolution layers and GDN layers, and to adjust the transition efficiency into the latent space, the CA block and the PPCA block may be placed in the final stage of feature extraction.

[0021] The above entropy encoding step may include a step of modeling the probability density function of the potential representation using mean and standard deviation parameters output from a hyperdecoder of a hyperprior structure.

[0022] The above encoding method can be performed by a model trained using a central view weight-based quality metric, which differentiates the quality contributions of the central and peripheral regions of an image, as the distortion term of the loss function.

[0023] In another aspect, an image decoding method performed by an image decoding device may include: a step of entropy decoding a latent representation and hyperprior information from a bitstream; a step of estimating probability distribution parameters of the latent representation based on the hyperprior information; a step of reconstructing a feature map by applying a PPCA block and a CA block to the latent representation; and a step of generating a reconstructed image from the reconstructed feature map through a synthetic network.

[0024] The above PPCA block can perform principal component recovery with minimized information loss by inversely applying the probabilistic characteristics of the projection matrix used during encoding.

[0025] The above synthetic network may include an Inverse Generalized Divisive Normalization (IGDN) layer for denormalization and a transposed convolution layer for resolution restoration.

[0026] In another aspect, an image encoding device may be provided that includes a processor and memory and is configured to perform an image encoding method.

[0027] In another aspect, an image decoding device configured to perform an image decoding method may be provided. Effects of the invention

[0028] An apparatus, a method, and a recording medium for performing end-to-end image compression are provided.

[0029] An apparatus, method, and recording medium are provided for performing end-to-end image compression utilizing stochastic principal component analysis and channel attention. Brief explanation of the drawing

[0030] FIG. 1 is a structural diagram of an encoding device according to one embodiment. FIG. 2 is a structural diagram of a decoding device according to one embodiment. FIG. 3 is a flowchart of an encoding method according to one embodiment. FIG. 4 is a flowchart of a decoding method according to one embodiment. Figure 5 shows an image compression network according to one example. Figure 6 shows an attention mechanism within a channel attention model according to one example. Figure 7 shows the central viewing areas and peripheral areas within an image according to one example. Specific details for implementing the invention

[0031] The present invention is capable of various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention.

[0032] The detailed description of the exemplary embodiments described below refers to the accompanying drawings, which illustrate specific embodiments by way of example. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that various embodiments are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein with respect to one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. It should also be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiment. Accordingly, the detailed description below is not intended to be taken in a limiting sense, and the scope of the exemplary embodiments is limited only by the appended claims, with all equivalents to those claimed therein, provided appropriately described.

[0033] In drawings, similar reference numerals refer to the same or similar functions across multiple aspects. The shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

[0034] In the present invention, terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" may include a combination of a plurality of related described items or any of a plurality of related described items.

[0035] When it is stated that a component is "connected" or "connected" to another component, it should be understood that the two components may be directly connected or connected to each other, or that another component may exist between the two components. On the other hand, when it is stated that a component is "directly connected" or "directly connected" to another component, it should be understood that no other component exists between the two components.

[0036] The components shown in the embodiments are depicted independently to represent different characteristic functions and do not imply that each component consists of separate hardware or a single software unit. That is, each component is listed and included as a separate component for convenience of explanation; however, at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform a function, and such integrated and separated embodiments of each component are included within the scope of the present invention as long as they do not depart from the essence of the invention.

[0037] The terms used in the embodiments are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the embodiments, terms such as "comprising" or "having" are intended to indicate the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. That is, the description in the embodiments that a specific configuration "comprising" does not exclude configurations other than that configuration, and means that additional configurations may also be included within the scope of the practice or technical concept of the invention.

[0038] In the embodiments, the term “at least one” may mean one of a number of 1 or more, such as 1, 2, 3, and 4. In the embodiments, the term “a plurality of” may mean one of a number of 2 or more, such as 2, 3, and 4.

[0039] Some components of the embodiments may not be essential components performing an essential function in the present invention, but merely optional components for enhancing performance. The embodiments may be implemented by including only the components essential for embodying the essence of the embodiments, excluding the components used solely for performance enhancement. A structure including only the essential components, excluding the optional components used solely for performance enhancement, is also included within the scope of the embodiments.

[0040] In the following, embodiments are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the embodiments. In describing the embodiments, detailed descriptions of related known components or functions are omitted if it is determined that such detailed descriptions may obscure the gist of this specification. Additionally, the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0041] In the following, "image" may refer to a picture constituting a video, or it may represent the video itself. For example, "encoding and / or decoding of an image" may mean "encoding and / or decoding of a video," and may also mean "encoding and / or decoding of one of the images constituting the video."

[0042] In the following, the terms "video" and "motion picture(s)" may be used interchangeably with the same meaning.

[0043] In the following, the terms "image," "picture," "frame," and "screen" may be used interchangeably with the same meaning.

[0044] FIG. 1 is a structural diagram of an encoding device according to one embodiment.

[0045] The encoding device (100) may include a processing unit (110), a memory (130), a user interface (UI) input device (150), a UI output device (160), and a storage (140) that communicate with each other via a bus (190). Additionally, the encoding device (100) may further include a communication unit (120) connected to a network (199).

[0046] The processing unit (110) may be a semiconductor device that executes processing instructions stored in a central processing unit (CPU), memory (130), or storage (140). The processing unit (110) may be at least one hardware processor.

[0047] The processing unit (110) can perform the generation and processing of signals, data, or information that are input to the encoding device (100), output from the encoding device (100), or used internally within the encoding device (100), and can perform inspection, comparison, and judgment related to the signals, data, or information. That is to say, in the embodiment, the generation and processing of data or information, and the inspection, comparison, and judgment related to the data or information can be performed by the processing unit (110).

[0048] The processing unit (110) can generate a bitstream. The processing unit (110) can perform operations for generating a bitstream as described in the embodiments.

[0049] At least some of the units described in the embodiments may be program modules and may communicate with an external device or system. The program modules may be included in the encoding device (100) in the form of an operating system, an application program module, and other program modules.

[0050] Program modules may be physically stored on various known memory devices. Additionally, at least some of these program modules may be stored in a remote memory device capable of communicating with the encoding device (100).

[0051] Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures that perform functions or operations according to one embodiment or implement abstract data types according to one embodiment.

[0052] Program modules may consist of instructions or code executed by at least one processor of the encoding device (100).

[0053] The processing unit (110) can execute the instructions or code of the units described in the embodiments.

[0054] The storage unit may represent memory (130) and / or storage (140). Memory (130) and storage (140) may be various forms of volatile or non-volatile storage media. For example, memory (130) may include at least one of ROM (131) and RAM (132).

[0055] The storage unit may store data or information used for the operation of the encoding device (100). In an embodiment, the data or information possessed by the encoding device (100) may be stored in the storage unit.

[0056] The encoding device (100) can be implemented in a computer system including a recording medium that can be read by a computer.

[0057] The recording medium can store at least one module required for the encoding device (100) to operate. The memory (130) can store at least one module and can be configured so that at least one module is executed by the processing unit (110).

[0058] Functions related to the communication of data or information of the encoding device (100) can be performed through the communication unit (120).

[0059] For example, the communication unit (120) can transmit the bitstream to the decoding device (200) to be described later.

[0060] FIG. 2 is a structural diagram of a decoding device according to one embodiment.

[0061] The decoding device (200) may include a processing unit (210), a memory (230), a user interface (UI) input device (250), a UI output device (260), and a storage (240) that communicate with each other via a bus (290). Additionally, the decoding device (200) may further include a communication unit (220) connected to a network (299).

[0062] The processing unit (210) may be a semiconductor device that executes processing instructions stored in a central processing unit (CPU), memory (230), or storage (240). The processing unit (210) may be at least one hardware processor.

[0063] The processing unit (210) can perform the generation and processing of signals, data, or information that are input to the decoding device (200), output from the decoding device (200), or used internally within the decoding device (200), and can perform inspection, comparison, and judgment related to the signals, data, or information. That is to say, in the embodiment, the generation and processing of data or information, and the inspection, comparison, and judgment related to the data or information can be performed by the processing unit (210). The processing unit (210) can use a bitstream. The processing unit (110) can perform operations using the bitstream described in the embodiments.

[0064] At least some of the units described in the embodiments may be program modules and may communicate with an external device or system. The program modules may be included in the decoding device (200) in the form of an operating system, an application program module, and other program modules.

[0065] The program modules may be physically stored on various known memory devices. Additionally, at least some of these program modules may be stored in a remote memory device capable of communicating with the decoding device (200).

[0066] Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures that perform functions or operations according to one embodiment or implement abstract data types according to one embodiment.

[0067] Program modules may consist of instructions or code executed by at least one processor of the decoding device (200).

[0068] The processing unit (210) can execute the instructions or code of the units described in the embodiments.

[0069] The storage unit may represent memory (230) and / or storage (240). Memory (230) and storage (240) may be various forms of volatile or non-volatile storage media. For example, memory (230) may include at least one of ROM (231) and RAM (232).

[0070] The storage unit may store data or information used for the operation of the decoding device (200). In an embodiment, the data or information possessed by the decoding device (200) may be stored in the storage unit.

[0071] The decoding device (200) can be implemented in a computer system including a recording medium that can be read by a computer.

[0072] The recording medium can store at least one module required for the decoding device (200) to operate. The memory (230) can store at least one module and can be configured so that at least one module is executed by the processing unit (210).

[0073] Functions related to the communication of data or information of the decoding device (200) can be performed through the communication unit (220).

[0074] For example, the communication unit (220) can receive a bitstream from the encoding device (100).

[0075] FIG. 3 is a flowchart of an image encoding method according to one embodiment.

[0076] In step (310), the processing unit (110) of the encoding device (100) can generate encoded information for the image by performing encoding for the image.

[0077] Encoded information regarding the image may include encoded information representing the image described in the embodiments.

[0078] For example, encoded information for an image may include information regarding a latent representation and a hyperprior, which will be described later. Encoded information for an image may include information regarding a quantized latent representation and a quantized hyperprior. Encoded information for an image may include information regarding an arithmetic-encoded quantized latent representation and an arithmetic-encoded quantized hyperprior.

[0079] Additionally, the encoded information for the image may include information generated by the encoding device (100) or encoding described in the embodiment.

[0080] Additionally, the encoded information for the image may include information contained within the bitstream described in the embodiment.

[0081] In step (320), the processing unit (110) can generate a bitstream. The bitstream can be recorded in a computer-readable recording medium.

[0082] The processing unit (110) can store the bitstream in the storage unit (140). The communication unit (120) can transmit the bitstream to the decoding device (200).

[0083] FIG. 4 is a flowchart of an image decoding method according to one embodiment.

[0084] In step (410), the processing unit (210) of the decoding device (200) can acquire a bitstream. The bitstream can be recorded in a computer-readable recording medium.

[0085] The processing unit (210) can read the bitstream from the storage (240). The communication unit (220) can receive the bitstream from the encoding device (100).

[0086] A bitstream may contain encoded information about an image.

[0087] Encoded information regarding the image may include encoded information representing the image described in the embodiments.

[0088] For example, encoded information for an image may include information regarding a latent representation and a hyperprior, which will be described later. Encoded information for an image may include information regarding a quantized latent representation and a quantized hyperprior. Encoded information for an image may include information regarding an arithmetic-encoded quantized latent representation and an arithmetic-encoded quantized hyperprior.

[0089] Additionally, the encoded information for the image may include information generated by the encoding device (100) or encoding described in the embodiment.

[0090] Additionally, the encoded information for the image may include information contained within the bitstream described in the embodiment.

[0091] In step (420), the processing unit (210) can restore the image by performing decoding of the image using encoded information about the image.

[0092] Characteristics of Deep Learning-Based Image Compression Methods

[0093] Deep learning-based image compression methods have demonstrated superior performance in terms of the efficiency of rate-distortion (RD).

[0094] In one example, for a model of an image compression method, an end-to-end compression framework using an autoencoder based on a Convolutional Neural Network (CNN) can be introduced, and non-linear transformation and quantization can be applied.

[0095] This approach can be improved by adding a hyperprior to better capture spatial dependencies.

[0096] These features can improve the performance of the image compression method model.

[0097] In another example, a model of an image compression method can be constructed based on the hyperprior method of the image compression method described above.

[0098] Here, spatial redundancy can be reduced by introducing discretized Gaussian mixture likelihoods and nonlocal attention modules.

[0099] Through these features, compression performance can be improved, especially in images with complex textures.

[0100] In one example, the hyperprior method can be further advanced by an image compression method. The image compression method can capture both local textures and global structures by integrating a window-block attention mechanism and a Symmetrical TransFormer (STF) model.

[0101] In one example, an approach to another image compression method can solve the problem that occurs during the process of applying a transformer model to image compression, and as a result, superior performance can be achieved compared to other image compression methods of the embodiments.

[0102] Deep learning-based compression techniques may have several limitations, such as unoptimized encoding efficiency and an imbalance between compression quality and restoration quality.

[0103] However, these problems can be resolved in areas containing important information, such as image objects.

[0104] Attention mechanisms can have a significant impact on RD performance and compression efficiency. The attention blocks used in the embodiments can enhance important information in the feature channel by highlighting important information and ignoring irrelevant details.

[0105] This technique can distinguish objects and backgrounds in compressed images and optimize the use of bandwidth and storage space.

[0106] Overview of Channel Attention and Stochastic Principal Component Analysis

[0107] In the embodiments, a learning-based image compression technique is disclosed that improves encoding efficiency by integrating a Channel Attention (CA) block and a Probabilistic Principal Component Analysis (PPCA) block as core elements.

[0108] PPCA blocks can emphasize important features by focusing on them and can effectively process noise.

[0109] Unlike traditional Principal Component Analysis (PCA), PPCA uses a probabilistic approach to more effectively preserve meaningful data structures, and through this preservation, it can improve compression performance and robustness.

[0110] In addition, the CA block can emphasize important image features by prioritizing the processing of key pixel values. CA can minimize information of low importance while preserving essential / essential information during the compression process.

[0111] Additionally, the embodiments may disclose an image quality evaluation index based on a foveated technique that prioritizes the evaluation of visually important areas.

[0112] These indicators can prioritize visually important areas within an image, such as objects, and through this consideration, key information derived from the attention mechanism can be evaluated more accurately.

[0113] These indicators can also be used to analyze the impact of CA blocks and PPCA blocks on the image restoration process and on the image restoration results.

[0114] Below, an example of an image compression method based on deep learning; an architecture of the example including a CA block, a PPCA block, and an evaluation metric; and a dataset, a learning setup, and an evaluation metric of the example are described.

[0115] Deep learning-based image compression methods

[0116] Traditional image compression

[0117] Generally, image compression aims to reduce spatial redundancy within an image.

[0118] For example, the traditional technique DCT can be used in the JPEG standard. On the other hand, the JPEG2000 standard can use the discrete wavelet transform, which is specifically designed for this purpose.

[0119] In other methods, redundancy is reduced by separating high-frequency and low-frequency information, and a method of adjusting bit allocation according to the importance of the signal may be used.

[0120] In addition, entropy coding techniques such as Huffman, run-length coding, hashing, and arithmetic coding can be used to improve compression efficiency without degrading image quality.

[0121] Intra-prediction techniques commonly used in image compression can also be applied to image compression. For example, there is Better Portable Graphics (BPG), based on the HEVC / H.265 image compression standard. BPG can provide superior image compression compared to existing methods such as JPEG and JPEG2000.

[0122] For BPG image formats, a predictive-transform method for restoration using 35 encoding options to reduce redundant data can be applied.

[0123] Another intra-prediction technique is Versatile Video Coding (VVC). VVC can support larger computational units and various prediction methods, transformation methods, and encoding methods.

[0124] In addition, hybrid techniques combining traditional and modern compression methods can be used for super-resolution learning.

[0125] In these techniques, the merging and compression of satellite images can be performed simultaneously, and a high final resolution and an optimal compression ratio can be provided.

[0126] Despite the various methods of these existing techniques, some limitations may still exist due to reliance on manually designed methods such as entropy encoding.

[0127] Learning-based image compression

[0128] Deep Neural Networks (DNNs) are one of the modern technologies being actively researched in the field of image compression.

[0129] DNNs can provide significant advantages in various application fields, including computer vision tasks such as object detection and image segmentation.

[0130] In addition, DNNs can be commonly utilized in research on super-resolution, object detection, and image compression.

[0131] In one example, an arithmetic coding scheme can be applied to DNNs to improve compression efficiency by separating the adaptive model from the encoding channel.

[0132] In other examples, encoders and decoders based on Recurrent Neural Networks (RNNs), and neural networks for binarization and entropy encoding may be used.

[0133] This method can provide a neural network architecture capable of achieving various compression levels without performing neural network retraining.

[0134] In one example, the Adversarial Image Compression method can improve the quality and efficiency of image data compression by using Adversarial Networks.

[0135] Adversarial image compression methods can perform more efficient encoding by utilizing adversarial neural networks to learn complex data patterns.

[0136] Here, compression efficiency can be improved by using a hyperprior layer to capture background information from latent data.

[0137] In addition, adding a Generalized Divisive Normalization (GDN) layer can reduce redundancy in data representation, and this reduction in data representation redundancy can improve model performance.

[0138] Furthermore, the adversarial approach may serve as the basis for various embodiments or be applied to various embodiments.

[0139] In the embodiments, a Gaussian model distribution module may be utilized to more accurately capture the data distribution, and compression efficiency may be improved by such utilization.

[0140] In one example, by applying an attention module, the model can focus on key features such as image boundary patterns and textures. This module can capture non-local correlations by referencing spatially distant pixel relationships, and through this capture, enable a more meaningful data representation during the compression process.

[0141] According to one example, non-local correlations can be captured more effectively by adding an attention mechanism to the model. This capture can improve feature representation in image compression.

[0142] In one example, distortion evaluation using a Generative Adversarial Network (GAN) can be performed. This distortion evaluation can enable the decoder to form more consistent structural patterns and generate high-quality reconstructed images.

[0143] However, GAN technology can have several drawbacks. For example, the generator may tend to produce limited variations or focus on specific features within the image map. Additionally, instability or mode collapse can occur in image restoration utilizing GANs, and due to this instability or mode collapse, the compressed result may not accurately reflect the overall data distribution.

[0144] In addition, in one example, noise can be removed more efficiently using a super-resolution technique, and the loss can be optimized based on this removal.

[0145] Object quality evaluation

[0146] Traditional evaluation metrics such as Peak Signal-to-Noise Ratio (PSNR), Multi-Scale Structural Similarity Index Measure (MS-SSIM), and Mean Squared Error (MSE) can be used to measure the difference between the restored image and the original image.

[0147] PSNR can be effective for identifying overall details of a reconstructed image. However, PSNR may have limitations in evaluating areas containing important information.

[0148] Even if attention blocks are added to a trained image compression model, effective evaluation may be impossible because PSNR only calculates the quality of the entire image and does not consider the difference between visually more important areas, such as main objects, and less important areas, such as the background. Therefore, it may be difficult to accurately measure the impact of CA blocks and PPCA blocks.

[0149] In one example, foveated techniques can be utilized to consider factors such as visual position and the location of the fovea that affect visual quality.

[0150] This technique can be applied to measure the visual quality of a reconstructed image generated by an end-to-end model. Additionally, this central field of view technique can divide the image into two regions: the background (so to speak, the periphery) and the foreground (so to speak, the center). By prioritizing the calculation of the region containing key information over the region containing less information, the evaluation result can more accurately reflect the desired information.

[0151] of the overall image compression network

[0152] Figure 5 shows an image compression network according to one example.

[0153] First, the image compression network illustrated in Fig. 5 will be briefly explained.

[0154] As illustrated in FIG. 5, the network of the embodiment may include a PPCA block and a CA block.

[0155] As illustrated in FIG. 5, the encoder transform and the decoder transform can use the same architectures. In FIG. 5, the architectures of these transforms are t a and t s It was displayed as.

[0156] t a can represent an encoder or an encoder network.

[0157] t s can represent a decoder or a decoder network.

[0158] h a can represent a hyper-encoder or a hyper-encoder network.

[0159] h s can represent a hyper-decoder or a hyper-decoder network.

[0160] In Fig. 5, Q can represent the quantization process.

[0161] AE can represent the arithmetic encoder of the entropy model. AD can represent the arithmetic decoder of the above entropy model. The arithmetic encoder can perform arithmetic encoding. The arithmetic decoder can perform arithmetic decoding.

[0162] A series of small squares between AE and AD can represent a bitstream.

[0163] The bitstream may include a first bitstream illustrated in the center of FIG. 5 and a second bitstream illustrated on the right side of FIG. 5. Alternatively, the first bitstream and the second bitstream may be physically or logically separated parts within a single bitstream.

[0164] The above image compression network can be divided into sides of the encoding device (100) and sides of the decoding device (200).

[0165] The encoding device (100) is an encoder network t a , hyperencoder network h a , can be configured to include a quantization processor Q and an arithmetic encoder AE. Here, the arithmetic encoder AE is a quantized latent representation First arithmetic encoder and quantized hyperprior for It may include a second arithmetic encoder for.

[0166] The decoding device (200) is a decoder network t s , hyperdecoder network h s and may be configured to include an arithmetic decoder AD. Here, the arithmetic decoder AD is a quantized latent representation First arithmetic decoder and quantized hyperprior for It may include a second arithmetic decoder for.

[0167] In the embodiments, the compression model is an encoder network t a and decoder network t s The composition of and quantized potential representation It may mean a configuration for. In other words, the compression model may be a model implemented with the components shown on the left side of FIG. 5.

[0168] In the embodiments, the hyperprior model is a hyperencoder network h a and hyperdecoder network h s The composition of and quantized hyperprior It may mean a configuration for. In other words, the hyperprior model may be a model implemented with the components shown on the right side of Fig. 5.

[0169] AE may be considered as arithmetic encoding by the arithmetic encoder AE. AD may be considered as arithmetic decoding by the arithmetic decoder AD.

[0170] In FIG. 5, convolution parameters are illustrated in the form "Conv, #filters × W × H / stride (down or up) arrow". "Conv" may represent a convolution operation. "#filters" may represent the number of filters. W may represent the width. H may represent the height. "stride" may represent the stride. "Down-arrow" may represent a down-sampling process. "Up-arrow" may represent an up-sampling process.

[0171] The values ​​of N and M can be selected based on λ parameter tests. For example, N can be 192 and M can be 320.

[0172] Next, the framework of the embodiment for image compression illustrated in FIG. 5 is described in more detail.

[0173] Compression techniques based on CNNs can generally use autoencoders. Variational Autoencoders (VAEs) can be models commonly used in image compression.

[0174] In one embodiment, compression performance can be improved by using an entropy model, and hyperprior parameters can be added to the extension model.

[0175] A hyperfrender network can be configured to include two main components.

[0176] The first of the key components may be the hyper-encoder. The hyper-encoder can provide additional information to the encoder by encoding the distribution of hidden variables.

[0177] The second major component may be a hyper-decoder. The hyper-decoder can improve image restoration performance by decoding these parameters.

[0178] When these two components are combined, a predicted image closer to the original image can be generated as double quantization and two bitstreams are created.

[0179] Based on the compression model of the embodiments, the attention mechanism proposed in the embodiments, and the foveated-based evaluation metric of the embodiments, the model of the embodiments can integrate the CA block and the PPCA block based on an autoencoder network.

[0180] The attention mechanism can be applied to each part of the encoder, decoder, and hyperprior.

[0181] In Fig. 5, the encoder network t a Is A latent representation y can be generated by processing the original image x.

[0182] Encoder network t a This processing can be performed using four convolution layers containing non-linear functions such as Generalized Divisive Normalization (GDN).

[0183] Subsequently, the latent expression y is the latent expression quantized by quantization Q It can be transformed into. That is, the quantized latent expression obtained by applying quantization Q to the latent expression y. This can be generated.

[0184] Subsequently, quantized potential representation is, after passing through arithmetic encoding AE and arithmetic decoding AD, decoder network t s It can be delivered as.

[0185] For example, quantized latent representation By applying an arithmetic encoding AE to it, an arithmetic encoded quantized latent representation can be generated.

[0186] The first bitstream may include an arithmetic-encoded quantized latent representation.

[0187] For example, by applying arithmetic decoding AD to an arithmetic encoded quantized latent representation, the quantized latent representation This can be restored. The restored quantized potential representation A decoder network t s It can be entered as.

[0188] Decoder network t s Is Quantized potential representation The final restored image by processing It can generate.

[0189] The framework of the example can finely improve the details of the potential representation by using the added CA block and PPCA block.

[0190] One of the main goals of these improvements may be to generate high-quality images at a specific bitrate.

[0191] To achieve this goal, the target bitrate of the bitstream can be predicted by utilizing an entropy model.

[0192] The entropy model can improve compression efficiency by utilizing hyperpriorities and factorized modules.

[0193] The entropy encoding process can begin with a hyperprior network that estimates a latent representation prior to quantization. Subsequently, the output of the hyper-encoder can be encoded into a bitstream. This information can be essential for improving overall compression quality by applying an appropriate entropy model during the decoding process.

[0194] In Fig. 5, hyperencoder h a It can obtain hyperprior information from a latent representation y and transform the hyperprior information into a hyperprior z. In other words, hyperencoder h a It can generate a hyperprior z from a latent representation y.

[0195] After that, hyperfryer z is a hyperfryer quantized by quantization Q It can be converted into. That is, the quantized hyperprior by applying quantization Q to hyperprior z can be generated.

[0196] Afterwards, quantized hyperprior is, after passing through arithmetic encoding AE and arithmetic decoding AD, hyperdecoder h s It can be delivered as.

[0197] For example, quantized hyperprior An arithmetic encoded quantized hyperprior can be generated by applying an arithmetic encoding AE to it.

[0198] The second bitstream may include an arithmetic-encoded quantized hyperfrector.

[0199] For example, by applying arithmetic decoding AD to an arithmetic encoded quantized hyperfryer, the quantized hyperfryer It can be restored. Restored quantized hyperprior hyperdecoder h s It can be entered as.

[0200] Hyperdecoder h s is a quantized hyperprior The parameters of the entropy model can be estimated using the information.

[0201] The parameter is the standard deviation of the Gaussian model. It may include.

[0202] Parameters are quantized latent representations It can be provided to the arithmetic encoder AE and arithmetic decoder AD for, and can be used in arithmetic encoding and arithmetic decoding.

[0203] CA blocks and PPCA blocks

[0204] Below, the CA block and PPCA block integrated into the framework of the embodiments are described in more detail.

[0205] In the previous embodiment, y represents the latent representation (of the previous quantization), and Although it represented a quantized potential representation, in this embodiment can be referred to as a latent representation. Additionally, z represents the hyperprior (prior to quantization), and Although represents a quantized hyperprior, in this embodiment It can be called a hyperfryer.

[0206] The following [Equation 1] can be used to calculate the loss L during the entire learning process of compression.

[0207]

[0208] Here, D can represent distortion, and x and It can be used to measure the difference between livers.

[0209] R can represent the bitrate of the bitstream.

[0210] The coefficient λ can be used to adjust the balance between distortion and bitrate.

[0211] Distortion can be measured using MS-SSIM, and d(x, It can be displayed as ).

[0212] Bitrate is a latent representation and hyperfryer It can be calculated using the entropy H.

[0213] An entropy estimation method can be used in the learning process, where latent features can be expressed as [Equation 2] below.

[0214]

[0215] Each expression The mean is 0 and the variance It can be modeled as a Gaussian distribution having.

[0216] Here, is a hyperprior network It can be predicted by.

[0217] Hyperfryer is a latent expression It can be used to further model the distribution of.

[0218] It can represent a uniform distribution.

[0219] The operation * can represent a convolution operation or a convolution process.

[0220] Hyperfryer The probability distribution of can be modeled using a non-parametric fully factorized density model as shown in [Equation 3] below.

[0221]

[0222] Here, each probability distribution is It can be expressed as. The parameters of such a probability distribution It can be estimated during the learning process.

[0223] As shown in [Equation 4] and [Equation 5] below, the bitrate is a latent representation and hyperfryer It can be calculated as the sum of the entropies of. In other words, potential expression and hyperfryer The sum of the entropies can be considered as the bitrate.

[0224]

[0225]

[0226] Channel Attention Block

[0227] Figure 6 shows an attention mechanism within a channel attention model according to one example.

[0228] The attention mechanism within a channel attention model can operate by assigning weight values ​​to the input (F) for each input element. The weight values ​​can be assigned based on the relevance of the input element to other input elements.

[0229] In one embodiment, an attention mechanism of the CA type may be used.

[0230] Attention mechanisms can be deployed in compression models and hyperprior models.

[0231] CA blocks can be designed to identify and focus important information in an image.

[0232] CA blocks can capture key pixel values ​​within an image while ignoring unimportant pixel values. Through this processing, CA blocks can preserve essential information during the compression process.

[0233] In Fig. 6, an attention block designed to allow a compressed neural network to process foveated regions can be shown.

[0234] Channel attention C for input feature F and element-wise multiplication a The process of deriving (F) can be defined as [Equation 6] below.

[0235]

[0236] First, spatial information can be collected from the input feature map by simultaneously applying average pooling Avgpool() and max pooling MaxPool().

[0237] Next, the two pooled feature vectors are shared fully connected (FC) layers and It can be delivered as.

[0238] and A linear transformation can be applied to the pooled feature vectors.

[0239] and It can share weights. and As weights are shared, a method to extract more general features can be learned without significantly increasing the number of parameters.

[0240] The size of It can be defined as follows. Here, r can represent the reduction ratio used to reduce parameter overhead. For example, r can be set to 64.

[0241] The size of is, To match the size of, It can be set to. It can be configured to match the dimension of the pooled feature vector.

[0242] Therefore, the weight parameters can be defined as shown in [Equation 7] and [Equation 8] below.

[0243]

[0244]

[0245] Weight parameters can be shared across two pooled feature vectors.

[0246] In fully connected layers, the Rectified Linear Unit (ReLU) activation function can be used.

[0247] The final output feature vectors passed through the shared fully connected layer can be merged using element-wise summation and the sigmoid activation function σ.

[0248] PPCA block

[0249] PPCA blocks can be used in statistical analysis to perform tasks such as dimensionality reduction, energy compression, and decorrelation on large datasets.

[0250] In one embodiment, this operation can be performed using PPCA, and the probabilistic characteristics of the data can be reflected through convergence values.

[0251] These convergence values ​​can represent stabilized estimates of model parameters following iterative optimization processes. Furthermore, these convergence values ​​can capture the most important features of the data while minimizing noise or less significant variations.

[0252] This approach can effectively balance dimensionality reduction and the preservation of important data characteristics, and through this balance, can yield more accurate and robust results.

[0253] In one example, PCA can identify principal components based on the largest variance in the data by calculating the eigenvectors and eigenvalues ​​of the covariance matrix.

[0254] On the other hand, PPCA can model data as random variables generated from a latent variable model including a normal distribution and Gaussian noise.

[0255] This probabilistic framework enables PPCA to handle uncertainty and missing data more effectively by estimating principal components and treating data as a combination of latent variables and Gaussian noise.

[0256] In PPCA, energy compression can concentrate data into a few principal components that capture the largest variance within the dataset, and through this concentration, the most important data in terms of energy or variance can be prioritized.

[0257] After energy compression, dimensionality reduction can be applied to extract probabilistic data based on the characteristics captured by these principal components.

[0258] In this process, data can be selected based on convergence values ​​and lower bounds, and can be sorted in order from high energy to low energy.

[0259] In PPCA, probability values ​​can represent data obtained after energy compression. This data can be sorted in order from high energy to low energy based on quality. This approach can ensure the preservation of the most meaningful and informative data representation while excluding information of low importance.

[0260] In the method of the embodiment, both the encoder and the decoder can project data into a lower-dimensional space using PPCA, and then restore the data projected into the lower-dimensional space to the original dimension.

[0261] PPCA can be advantageous for preserving the structural characteristics of data during the dimensionality reduction process and can effectively handle noise and missing values.

[0262] Encoder t a In this case, the PPCA block can perform uncorrelation of feature elements, reduce dimensionality, and improve encoding efficiency before proceeding to the CA attention step processing.

[0263] This dimensionality reduction can enable the compressed data to be processed more efficiently in subsequent processing steps by compressing the data.

[0264] On the other hand, decoder t s The use of PPCA in this context may not be intuitive. This difference may be because the primary goal on the decoder side is to restore the original data as accurately as possible.

[0265] However, the application of dimensionality reduction in decoders can provide several distinct advantages, such as those described below.

[0266] By reducing the dimensionality in the decoder, PPCA can remove noise and irrelevant details that may be introduced during the encoding process, and focus on restoring the most important components of the data.

[0267] In addition, due to the probabilistic characteristics of PPCA, the decoder network of the embodiment can reflect the uncertainty and variability of the data. Also, due to the probabilistic characteristics of PPCA, overfitting to details of low importance can be prevented while the restored output maintains the intrinsic characteristics of the original data.

[0268] Therefore, a cleaner and more generalized output can be generated by the PPCA block within the decoder, and the PPCA block can optimize the restoration process by focusing on the principal components that capture the largest variance within the dataset.

[0269] This approach is particularly advantageous when dealing with complex and high-dimensional data, and the quality of the restored image can be significantly improved while retaining only the most important features.

[0270] In particular, the PPCA block of the encoder and the PPCA block of the decoder can be independent and unrelated to each other. Similarly, the CA block of the encoder and the CA block of the decoder can also operate independently of each other.

[0271] Through this independent configuration, each step in the PPCA blocks can be free from constraints by the operations of other steps, and the independent function of each step can be optimized.

[0272] Due to this independence, each PPCA block can be finely tuned for efficient encoding or accurate restoration, and at the same time, each PPCA block can effectively perform data compression and restoration.

[0273] In the initial stage of the PPCA block, the size of the PPCA input can be adjusted to match the output of the previous block.

[0274] As illustrated in Fig. 5, the input to the PPCA in the PPCA block may be the output of the previous convolution layer. This output may have dimensions of (height × width × channel).

[0275] The input to PPCA can be denoted by Y, and the dimension of the input can be N × M. Here, N can be the number of samples. M can represent the dimension of the data.

[0276] Because the dimensions of the input to PPCA and the dimensions of the output of the previous convolution layer are different, the output of the convolution layer can be reconfigured to match the input to PPCA. As a result of this reconfiguration, Y can be expressed as a linear combination of latent variables X and weights W. Y, X, and W can be defined as shown in [Equation 9], [Equation 10], and [Equation 11] below, respectively.

[0277]

[0278]

[0279]

[0280] Here, N can represent height × width, D can represent the latent dimension, and M can represent the number of channels.

[0281] After reconstruction, PPCA can be performed according to the following [Equation 12] and [Equation 13].

[0282]

[0283]

[0284] In [Equation 12], Y can be data modeled in PPCA. Y can be obtained based on the projection matrix W, noise term E, mean μ of the data, and projected latent variable X.

[0285] According to [Equation 13], the noise E has a mean of 0 and the covariance matrix is ​​τ -1 It may follow a multivariate normal distribution where I. Here, τ -1 It can be used to control noise dispersion.

[0286] The goal of PPCA may be to learn the projection matrix W and latent variable X that best represent the observed data Y by minimizing information loss or variance.

[0287] After the input size is adjusted, the convergence values ​​for W and X can be determined next.

[0288] In PPCA, this convergence process can be performed as iterative processes that maximize the likelihood function.

[0289] As a result of these iterative processes, energy compressibility can be ensured, and feature data can be arranged in order from high energy to low energy.

[0290] Once convergence is reached, dimensionality reduction using a lower bound can be applied. Through these processes, the stability of the convergence process can be guaranteed.

[0291] Dimensionality reduction can be performed by identifying the principal component that captures the largest variance. Through dimensionality reduction, new features can be obtained in W and X, as shown in [Equation 14] below.

[0292]

[0293] The PPCA restoration process can utilize only converged data features extracted using lower bounds.

[0294] Data lower than this lower limit may be considered to have low importance and may be excluded from use.

[0295] Through such exclusion, noise and unnecessary details can be effectively removed. As a result, the restored data can be regarded as an estimate of the original data. Furthermore, in the restored data, patterns and fundamental relationships in the most important (high-energy) features are highlighted, while less important (low-energy) features can be filtered out.

[0296] Due to the probabilistic nature of PPCA, PPCA blocks can model and process missing data more effectively. However, this probabilistic nature can introduce variability compared to the original data Y.

[0297] Therefore, the restored output It can provide the optimal stochastic estimate for Y.

[0298] Restored data can be refined data of Y. This is because, compared to Y, In this case, unnecessary data including noise was removed, and This may be because only data with important principal component dispersion structures were prioritized in the generation.

[0299] The determination of the best result by the PPCA block can depend on the data. Therefore, the size of the dataset can have a significant impact on the probabilistic results obtained from the trained neural network model. On the other hand, increasing the size of the dataset can increase the cost, training time, and complexity of the neural network model.

[0300] After the restoration result is obtained, output It can be converted into the form of the original input (e.g., height × width × channels) and then input back into the neural network model.

[0301] Object quality evaluation

[0302] Evaluation indicators can be used to evaluate the quality of the restored results.

[0303] In addition to the existing indicators of the embodiments, the indicator described below according to one example may be used.

[0304] The developed evaluation metric of the example can be referred to as the Foveal Peak Signal-to-Noise Ratio (F_PSNR).

[0305] F_PSNR can prioritize important visual areas, such as objects, over surrounding areas.

[0306] Furthermore, F_PSNR can be extended to Multi-F_PSNR (so to speak, MF_PSNR), a new metric that supports processing multiple objects in an image by evaluating multiple central viewing regions.

[0307] Quality evaluation can be applied to the restoration results, and the final result of the restoration results can be evaluated by the quality evaluation.

[0308] Foveated-PSNR

[0309] Figure 7 shows the central viewing areas and peripheral areas within an image according to one example.

[0310] In Fig. 7, N f Φ is the central foveal region intended to expand the scope of calculation to include more regions than a single region. j It can represent the result of adding.

[0311] First, the image can be divided into multiple regions to measure the quality of the central field of view.

[0312] These divided regions are, as illustrated in FIG. 7, the central field of view region Φ j (1 ≤ j ≤ N f It can be classified into one of the ) and peripheral region Φ0.

[0313] Here, N f It can represent the total number of central visual fields.

[0314] The method for identifying the central field of view in this embodiment may differ from the methods of other embodiments.

[0315] When regions are separated, a mask can be generated for each region. The central field of view can be given priority over the periphery due to the mask.

[0316] Each region Φ j central vision mask for is Φ j It has a value of 1 for the area and 0 for the other areas.

[0317] These masks can be generated manually and obtained using a detection method.

[0318] Central visual field area Φ j F_MSE(Φ j )Is It can be calculated using . Here, the object of calculation is Φ as defined in [Equation 15] below. j It can be only the pixels inside.

[0319]

[0320] Here, I ref (i) and I test (i) can represent the i-th pixel value of the reference image and the test image, respectively.

[0321] N can represent the total number of pixels in the test image.

[0322] Region Φ j F_PSNR(Φ), which is the PSNR value for j The target of ) can be a single central field of view and can be measured according to [Equation 16] below.

[0323]

[0324] When multiple central vision regions exist, the average of F_PSNR for all central vision regions It can be calculated according to [Mathematical Formula 17] below.

[0325]

[0326] Multi-Foveated PSNR

[0327] In the final stage, image quality can be evaluated by distinguishing the quality of visually important areas and surrounding areas.

[0328] The measure of image quality of the example may be MF_PSNR. MF_PSNR can be defined according to [Equation 18] below.

[0329]

[0330] In [Equation 18], η may be a foveal factor for adjusting the central field of view and the peripheral field of view, respectively, using weights.

[0331] F_PSNR(Φ0) can represent the PSNR of the surrounding region.

[0332] The purpose of MF_PSNR may be to provide a more precise evaluation by emphasizing the quality of the central field of view area.

[0333] The parameter η can adjust the quality priority of the central field of view area and the peripheral area.

[0334] To calculate F_PSNR and MF_PSNR, the central field of view area may need to be identified using a central field of view mask. Therefore, detection and segmentation techniques can be used to automatically detect the central field of view area without manual selection. Through these techniques, objective quality evaluation can be ensured through calculation without subjective intervention.

[0335] The above embodiments may be performed in the same and / or corresponding way in the encoding device (100) and the decoding device (200). Additionally, a combination of one or more of the above embodiments may be used for encoding and / or decoding of an image.

[0336] The order in which the above embodiments are applied may differ from one another in the encoding device (100) and the decoding device (200). Alternatively, the order in which the above embodiments are applied may be (at least partially) the same in the encoding device (100) and the decoding device (200).

[0337] The above embodiments can be performed for each of the luminance signal and the chroma signal. The above embodiments can be performed identically for the luminance signal and the chroma signal.

[0338] In the embodiments described above, when applying a specific treatment to a specific target, a specific condition may be required, and if it is described that the specific treatment is performed under a specific decision, and if it is described that whether the specific condition is satisfied is determined based on a specific coding parameter, or that a specific decision is made based on a specific coding parameter, the specific coding parameter may be interpreted as being replaceable with another coding parameter. That is to say, the coding parameter affecting the specific condition or the specific decision may be considered merely exemplary, and it may be understood that a combination of one or more other coding parameters in addition to the specified coding parameter performs the role of the specified coding parameter.

[0339] In the embodiments described above, methods are described based on flowcharts as a series of steps or units; however, the present invention is not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps as described above. Furthermore, those skilled in the art will understand that the steps shown in the flowcharts are not exclusive, that other steps may be included, or that one or more steps of the flowcharts may be omitted without affecting the scope of the present invention.

[0340] The embodiments described above include examples of various aspects. While not all possible combinations for representing various aspects can be described, those skilled in the art will recognize that combinations other than those explicitly described are possible. Accordingly, the present invention shall be deemed to include all other substitutions, modifications, and changes falling within the scope of the following claims.

[0341] The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software.

[0342] A computer-readable recording medium may include information used in embodiments according to the present invention. For example, a computer-readable recording medium may include a bitstream, and the bitstream may include information described in embodiments according to the present invention.

[0343] The bitstream may include computer-executable code and / or programs. The computer-executable code and / or programs may include information described in the embodiments and syntax elements described in the embodiments. That is to say, the information and syntax elements described in the actual examples may be considered as computer-executable code within the bitstream and may be considered as at least part of the computer-executable code and / or programs represented by the bitstream.

[0344] Computer-readable recording media may include non-transitory computer-readable media.

[0345] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The above hardware devices may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0346] Although the present invention has been described above with specific details such as specific components, limited embodiments, and drawings, this is provided only to aid in a more comprehensive understanding of the invention, and the invention is not limited to the above embodiments, and a person skilled in the art to which the invention belongs can make various modifications and variations from this description.

[0347] Accordingly, the scope of the present invention should not be limited to the embodiments described above, and all modifications equivalent to or equivalent to the claims set forth below, as well as the claims described below, shall be considered to fall within the scope of the concept of the present invention.

Claims

Claim 1 A video encoding method comprising: a step of generating a bitstream containing encoded information of a video; and a step of transmitting said bitstream, wherein the bitstream contains information about a potential representation and a hyperprior.

Images