Methods, apparatuses, and media for visual data processing

Abstract

Embodiments of the present disclosure provide a solution for visual data processing. A method for visual data processing is proposed. The method comprises: performing, with a neural network (NN) based model, a conversion between visual data and a bitstream of the visual data, wherein the bitstream comprises a first indication indicating whether a set of values for a set of parameters of the NN based model is common for processing of a plurality of components of the visual data.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to visual data processing techniques, and more specifically, to visual data encoding and decoding based on neural networks. Background Technology

[0002] The past decade has witnessed the rapid development of deep learning across various fields, particularly in computer vision and image processing. Neural networks were initially invented as part of interdisciplinary research in neuroscience and mathematics. They have demonstrated powerful capabilities in the context of nonlinear transformations and classification. Neural network-based image / video compression technology has made significant progress in the past five years. It has been reported that the latest neural network-based image compression algorithms have achieved rate-distortion (RD) performance comparable to that of Multifunctional Video Coding (VVC). With the continuous improvement in the performance of neural image compression, neural network-based video compression has become an actively developed research area. However, overall, there is a strong expectation to further improve the encoding / decoding quality and efficiency of neural network-based image / video codecs. Summary of the Invention

[0003] Embodiments of this disclosure provide a solution for visual data processing.

[0004] In a first aspect, a method for visual data processing is proposed. The method includes: performing a transformation between visual data and a bitstream of visual data using a neural network (NN)-based model, wherein the bitstream includes a first indication indicating whether the processing of a set of values ​​for a set of parameters of the NN-based model is common to multiple components of the visual data.

[0005] According to the method of the first aspect of this disclosure, an indicator is included in the bitstream and is used to indicate whether the processing of a set of values ​​for a set of parameters of a neural network-based model is common to multiple components of the visual data. With this indicator, the transmission of the set of values ​​for each component of the visual data via a separate signal can be avoided. Therefore, the proposed method can advantageously improve encoding and decoding efficiency.

[0006] In a second aspect, an apparatus for visual data processing is proposed. The apparatus includes a processor and a non-transitory memory having instructions thereon. When executed by the processor, the instructions cause the processor to perform the method according to the first aspect of this disclosure.

[0007] In a third aspect, a non-transitory computer-readable storage medium is proposed. This non-transitory computer-readable storage medium stores instructions that cause a processor to perform the method according to the first aspect of this disclosure.

[0008] In a fourth aspect, another non-transitory computer-readable recording medium is proposed. This non-transitory computer-readable recording medium stores a bitstream of visual data generated by a method performed by an apparatus for visual data processing. The method includes performing a conversion between visual data and a bitstream using a neural network (NN)-based model, wherein the bitstream includes a first indication indicating whether the processing of multiple components of the visual data is common to a set of values ​​for a set of parameters of the NN-based model.

[0009] In a fifth aspect, a method for storing a bitstream of visual data is proposed. The method includes: performing a conversion between visual data and a bitstream using a neural network (NN)-based model, wherein the bitstream includes a first indication indicating whether the processing of a set of values ​​for a set of parameters of the NN-based model is common to multiple components of the visual data; and storing the bitstream in a non-transitory computer-readable recording medium.

[0010] The present invention is provided to present, in a simplified form, the selection of concepts further described below in the detailed description. The present invention is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0011] The above and other objects, features, and advantages of exemplary embodiments of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. In the exemplary embodiments of the present disclosure, the same reference numerals generally refer to the same components.

[0012] Figure 1 A block diagram illustrating an example visual data encoding / decoding system according to some embodiments of the present disclosure is shown;

[0013] Figure 2 A typical transform encoding / decoding scheme is shown;

[0014] Figure 3 Images from the Kodak dataset and their different representations are shown;

[0015] Figure 4 The network architecture of an autoencoder that implements a hyperprior model is shown.

[0016] Figure 5 A block diagram of the combined model is shown;

[0017] Figure 6 The encoding process of the combined model is shown;

[0018] Figure 7 The decoding process of the combined model is shown;

[0019] Figure 8 Examples of decoding processes according to some embodiments of this disclosure are shown;

[0020] Figure 9 A flowchart of a method for visual data processing according to embodiments of the present disclosure is shown; and

[0021] Figure 10 A block diagram of a computing device in which various embodiments of the present disclosure may be implemented is shown.

[0022] Throughout all the accompanying figures, the same or similar reference numerals generally refer to the same or similar elements. Detailed Implementation

[0023] The principles of this disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described for illustrative purposes only and to help those skilled in the art understand and implement this disclosure, and do not imply any limitation on the scope of this disclosure. In addition to the methods described below, the disclosure described herein can be implemented in various other ways.

[0024] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0025] The terms "an embodiment," "embodiment," "example embodiment," etc., used in this disclosure refer to embodiments that may include specific features, structures, or characteristics, but not every embodiment includes that specific feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Moreover, when a specific feature, structure, or characteristic is described in conjunction with an example embodiment, it is claimed that, whether explicitly described or not, such a feature, structure, or characteristic affecting its relation to other embodiments is within the knowledge of those skilled in the art.

[0026] It should be understood that although the terms “first” and “second”, etc., may be used herein to describe various elements, these elements should not be limited to these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising,” “including,” “having,” “containing,” and / or “comprising” as used herein indicate the presence of the said features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0028] Example Environment

[0029] Figure 1 This is a block diagram illustrating an example visual data encoding / decoding system 100 that can utilize the techniques of this disclosure. As shown, the visual data encoding / decoding system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a visual data encoding device, and the destination device 120 may also be referred to as a visual data decoding device. In operation, the source device 110 may be configured to generate encoded visual data, and the destination device 120 may be configured to decode the encoded visual data generated by the source device 110. The source device 110 may include a visual data source 112, a visual data encoder 114, and an input / output (I / O) interface 116.

[0030] Visual data source 112 may include sources such as visual data acquisition devices. Examples of visual data acquisition devices include, but are not limited to, interfaces for receiving visual data from visual data providers, computer graphics systems for generating visual data, and / or combinations thereof.

[0031] Visual data may include one or more pictures or images from a video. A visual data encoder 114 encodes the visual data from a visual data source 112 to generate a bitstream. The bitstream may include a sequence of bits forming an encoded representation of the visual data. The bitstream may include encoded pictures and associated visual data. The encoded / decoded pictures are encoded representations of the pictures. The associated visual data may include sequence parameter sets, picture parameter sets, and other syntax structures. An I / O interface 116 may include a modulator / demodulator and / or a transmitter. The encoded visual data can be directly transmitted to a destination device 120 via the I / O interface 116 through a network 130A. The encoded visual data may also be stored on a storage medium / server 130B for access by the destination device 120.

[0032] The destination device 120 may include an I / O interface 126, a visual data decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may acquire encoded visual data from the source device 110 or the storage medium / server 130B. The visual data decoder 124 may decode the encoded visual data. The display device 122 may display the decoded visual data to a user. The display device 122 may be integrated with the destination device 120, or it may be external to the destination device 120, which is configured to interface with an external display device.

[0033] The visual data encoder 114 and the visual data decoder 124 can operate according to visual data encoding and decoding standards (such as video encoding and decoding standards or still image encoding and decoding standards, as well as other existing and / or future standards).

[0034] Some exemplary embodiments of this disclosure will be described in detail below. It should be understood that section headings are used in this document for ease of understanding and not to limit the embodiments disclosed in a section to that section. Furthermore, while some embodiments are described with reference to multi-functional video codecs or other specific visual data codecs, the disclosed techniques are also applicable to other codec techniques. Additionally, although some embodiments describe encoding steps in detail, it should be understood that the corresponding decoding steps for de-encoding will be implemented by the decoder. Furthermore, the term visual data processing includes visual data encoding or compression, visual data decoding or decompression, and visual data transcoding, in which visual data is represented from one compressed format to another or at a different compression bit rate.

[0035] 1. Brief Overview

[0036] A neural network-based image and video compression method includes separate processing of the color components of an image, wherein control parameters used to process one component are also used for the other component.

[0037] 2. Introduction

[0038] The past decade has witnessed the rapid development of deep learning across various fields, particularly in computer vision and image processing. From the immense success of deep learning technologies to computer vision, many researchers have shifted their focus from traditional image / video compression techniques to neural image / video compression. Neural networks were initially invented as part of interdisciplinary research in neuroscience and mathematics. They have demonstrated powerful capabilities in the context of nonlinear transformations and classification. Neural network-based image / video compression techniques have made significant progress in the past five years. It has been reported that the latest neural network-based image compression algorithms have achieved RD performance comparable to Video Coding-Codec (VVC), the latest video codec standard developed by the Joint Video Experts Group (JVET), a group of experts from MPEG and VCEG. With the continuous improvement in the performance of neural image compression, neural network-based video compression has become an actively developed research area. However, due to the inherent difficulties of the problem, neural network-based video codecs are still in their early stages.

[0039] 2.1. Image / Video Compression

[0040] Image / video compression generally refers to the computational technique of compressing images / videos into binary codes for easier storage and transmission. Binary codes may or may not support lossless reconstruction of the original image / video, a process known as lossless compression and lossy compression. Most efforts focus on lossy compression because lossless reconstruction is unnecessary in most scenarios. Typically, the performance of image / video compression algorithms is evaluated from two aspects: compression ratio and reconstruction quality. The compression ratio is directly related to the number of binary codes; fewer binary codes are better. Reconstruction quality is measured by comparing the reconstructed image / video with the original image / video; higher reconstruction quality is better.

[0041] Image / video compression techniques can be divided into two branches: classical video codec methods and neural network-based video compression methods. Classical video codec schemes employ transform-based solutions, where researchers leverage statistical dependencies in latent variables (e.g., DCT or wavelet coefficients) by carefully manually designing entropy codes to model dependencies in the quantization domain. Neural network-based video compression takes two forms: neural network-based codec tools and end-to-end neural network-based video compression. The former is embedded in existing classical video codecs as codec tools and used only as part of the framework, while the latter is a separate framework developed based on neural networks, independent of classical video codecs.

[0042] Over the past three decades, a series of classic video codec standards have been developed to accommodate the increasing visual content. The international standards organization ISO / IEC has two expert groups: the Joint Picture Experts Group (JPEG) and the Moving Picture Experts Group (MPEG). ITU-T also has its own Video Codec Experts Group (VCEG), which is responsible for standardizing image / video codec technologies. Influential video codec standards released by these organizations include JPEG, JPEG 2000, H.262, H.264 / AVC, and H.265 / HEVC. Following H.265 / HEVC, the Joint Video Experts Group (JVET), formed by MPEG and VCEG, has been working on a new video codec standard, Multi-Functional Video Codec (VVC). The first version of VVC was released in July 2020. Compared to HEVC, VVC has been reported to have an average bitrate reduction of 50% while maintaining the same visual quality.

[0043] Neural network-based image / video compression is not a new invention, as many researchers have been working on neural network-based image encoding and decoding. However, the network architecture is relatively shallow, and the performance is unsatisfactory. Benefiting from the support of large amounts of data and powerful computing resources, neural network-based methods are being better utilized in various applications. Currently, neural network-based image / video compression has shown promising improvements, confirming its feasibility. However, the technology is still far from mature, and many challenges remain to be solved.

[0044] 2.2. Neural Networks

[0045] Neural networks (also known as artificial neural networks (ANNs)) are computational models used in machine learning techniques. They typically consist of multiple processing layers, each composed of several simple but non-linear basic computational units. One benefit of such deep networks is considered to be their ability to process data with multiple levels of abstraction and transform that data into different kinds of representations. Note that these representations are not manually designed; instead, deep networks, including processing layers, learn from massive amounts of data using general machine learning processes. Deep learning eliminates the need for hand-crafted representations and is therefore considered useful, particularly for processing raw, unstructured data such as acoustic and visual signals, which has been a long-standing challenge in the field of artificial intelligence.

[0046] 2.3. Neural Networks for Image Compression

[0047] Existing neural networks used for image compression methods can be divided into two categories: pixel probability modeling and autoencoders. The former belongs to predictive encoding / decoding strategies, while the latter is a transform-based solution. Sometimes, these two methods are combined in the literature.

[0048] 2.3.1. Pixel Probability Modeling

[0049] According to Shannon's information theory, the optimal method for lossless encoding and decoding can achieve a minimum coding rate of log₂p(x), where p(x) is the probability of symbol x. Many lossless encoding and decoding methods have been developed in the literature, and among them, arithmetic encoding and decoding is considered optimal. Given a probability distribution p(x), arithmetic encoding and decoding ensures that the coding rate is as close as possible to its theoretical limit—log₂p(x)—without considering rounding errors. Therefore, the remaining problem is how to determine the probabilities; however, this is very challenging for natural images / videos due to dimensionality reduction.

[0050] Following a predictive encoding / decoding strategy, one way to model p(x) is to predict the probability of each pixel in raster scan order based on previous observations, where x is the image.

[0051] p(x)=p(x1)p(x2|x1)...p(x i |x1, ...,x i-1 ...p(x) m×n |x1, ...,x m×n-1 (1)

[0052] Here, m and n are the height and width of the image, respectively. Previous observations are also referred to as the context of the current pixel. When the image is large, it can be difficult to estimate the conditional probability, so a simplification approach is to limit the scope of its context.

[0053] p(x)=p(x1)p(x2|x1)...p(x i |x i-k , ..., x i-1 ...p(x) m×n |x m×n-k , ..., x m×n-1 (2)

[0054] Where k is a predefined constant that controls the scope of the context.

[0055] It should be noted that the conditions can also consider the sample values ​​of other color components. For example, when encoding and decoding RGB color components, the R sample depends on the previously encoded pixels (including R / G / B samples), and the current G sample can be encoded and decoded based on the previously encoded pixels and the current R sample. When encoding and decoding the current B sample, the previously encoded pixels, the current R sample, and the current G sample can also be considered.

[0056] Neural networks were initially introduced for computer vision tasks and have proven effective in regression and classification problems. Therefore, it has been proposed to use neural networks to estimate the relative order of elements given their context x1,n2,…,x. i-1 The probability p(x) i In the existing design, for binary images (i.e., x...), i The pixel probability is proposed using the expression ∈{-1,+1}). A Neural Autoregressive Distribution Estimator (NADF) is designed for pixel probability modeling, where a feedforward network with a single hidden layer is used. Similar work is presented in existing designs where the feedforward network also has connections that skip hidden layers, and parameters are shared. Experiments are performed on a binarized MNIST dataset. In existing designs, NADF is extended to the real-valued model RNADE, where the probability p(x) is given by the expression ∈{-1,+1}. i |x1,…,x i-1 The Gaussian mixture is used to derive the algorithm. Their feedforward networks also have a single hidden layer, but this hidden layer is rescaled to avoid saturation and uses a modified linear unit (ReLU) instead of a sigmoid. In existing designs, NADF and RNADE are improved by reorganizing the order of pixels and utilizing deeper neural networks.

[0057] Most of the methods described above directly model the probability distribution in the pixel domain. Some researchers have also attempted to model the probability distribution as a condition of either an explicit representation or a latent representation. That is, it is possible to estimate:

[0058]

[0059] Where h is an additional condition, and p(x) = p(h)p(x|h) means that the modeling is split into an unconditional one and a conditional one. The additional condition can be image label information or a high-level representation.

[0060] 2.3.2. Automatic Encoder

[0061] Autoencoders originate from existing designs. This method is trained for dimensionality reduction and comprises two parts: encoding and decoding. The encoding part transforms a high-dimensional input signal into a low-dimensional representation, typically with a reduced spatial size but a larger number of channels. The decoding part attempts to recover the high-dimensional input from the low-dimensional representation. Autoencoders achieve automatic learning of representations and eliminate the need for hand-crafted features, which is considered one of the most significant advantages of neural networks.

[0062] Figure 2 This illustrates a typical transform encoding / decoding scheme. The original image x is processed by the analysis network g. aA transformation is performed to achieve the latent representation y. The latent representation y is quantized and compressed into bits. The number of bits R is used to measure the coding rate. The quantized latent representation is then... By the synthetic network g s Inverse transform to obtain the reconstructed image Distortion is achieved by transforming x and Using the function g p It is computed in the perceptual space.

[0063] Applying autoencoder networks to lossy compression is intuitive. It simply requires encoding the learned latent representation from a trained neural network. However, adapting the autoencoder to image compression is not crucial because the original autoencoder is not optimized for compression, making direct use of the trained autoencoder inefficient. Furthermore, other major challenges exist: First, the low-dimensional representation should be quantized before encoding, but quantization is not differentiable, which is necessary during backpropagation when training the neural network. Second, the objectives differ in compression scenarios because both distortion and bit rate need to be considered. Estimating the bit rate is challenging. Third, practical image encoding / decoding schemes need to support variable bit rates, scalability, encoding / decoding speeds, and interoperability. In response to these challenges, many researchers have actively contributed to this field.

[0064] A prototype autoencoder for image compression Figure 2 In this context, it can be viewed as a transformation encoding / decoding strategy. The original image x is processed using the analysis network y=g a (x) is transformed, where y is the latent representation to be quantized and encoded / decoded. The synthesis network will process the quantized latent representation... Perform an inverse transform to obtain the reconstructed image. The framework utilization distortion loss function is trained, i.e. Where D is x and The distortion between them, R is from the quantized representation The bit rate is calculated or estimated, and λ is the Lagrange multiplier. It should be noted that D can be calculated in either the pixel domain or the receptive domain. All existing research follows this prototype, and the differences may lie only in the network architecture or the loss function.

[0065] 2.3.3. Pre-Prior Model

[0066] In the transform encoding and decoding method for image compression, the encoder subnetwork (Section 2.3.2) uses parametric analysis of the transform. The image vector x is transformed into a latent representation y, which is then quantized to form because These are discrete values ​​that can be losslessly compressed and transmitted as bit sequences using entropy encoding / decoding techniques (such as arithmetic encoding / decoding).

[0067] As from Figure 3 The images that are slightly to the left and slightly to the right are obviously different. Significant spatial dependencies exist among the elements. Notably, their scales (center-right image) appear spatially coupled. Introducing another set of random variables into the existing design... To capture spatial dependencies and further reduce redundancy. In this case, the image compression network... Figure 4 As shown in the image.

[0068] exist Figure 4 In the model, the encoder g is on the left side. a and decoder g s (Explained in Section 2.3.2). The right-hand side is used to obtain... The encoder h that utilizes prior information a and decoder h that utilizes prior information s Network. In this architecture, the encoder sends data to g. a The input image x is used to generate a response y with a spatially varying standard deviation. The response y is then fed into h. a In the middle, the standard deviation distribution of z is summarized. Then z is quantized. Compression is achieved and transmitted as side information. The encoder then uses the quantized vector... To estimate the spatial distribution of σ, the standard deviation, and to use it to compress and transmit the quantized image representation. The decoder first recovers from the compressed signal Then it uses h s To obtain σ, which provides a correct probability estimate for equally successful recovery. Then to g s feed To obtain the reconstructed image.

[0069] When an encoder and a decoder utilizing prior information are added to an image compression network, the quantized latent value Spatial redundancy is reduced. Figure 3 The rightmost image corresponds to the quantized latent value when using an encoder / decoder that utilizes prior information. Compared to the image slightly to the right of center, spatial redundancy is significantly reduced because the samples of the quantized latent value are less correlated.

[0070] Figure 3 Images from the Kodak dataset and different representations of the images are shown. Figure 3 The leftmost image shows an image from the Kodak dataset. Figure 3The image in the middle left shows a visualization of the potential representation y of the image. Figure 3 The right-center plot in the image shows the standard deviation σ of the potential value. Figure 3 The rightmost image shows the latent value y after introducing a super-prior network (an encoder that utilizes super-prior information and a decoder that utilizes super-prior information).

[0071] Figure 4 The network architecture of an autoencoder implementing a priori model is shown. The left side shows the image autoencoder network, and the right side corresponds to the priori subnetwork. The analysis and synthesis transformations are represented as g. a and g a Q represents quantization, and AE and AD represent the arithmetic encoder and arithmetic decoder, respectively. The hyperprior model consists of two sub-networks: an encoder utilizing hyperprior information (using h...). a (representation) and decoders utilizing prior information (using h) s (Representation) Composition. The advanced prior model generates quantified advanced prior information potential values. This includes information about quantified potential values. Information about the probability distribution of the sample points. Included in the bitstream and with They are transmitted together to the receiver (decoder).

[0072] 2.3.4. Context Model

[0073] Although the prior model improves the quantified latent value Modeling the probability distribution of latent values ​​is possible, but additional improvements can be obtained by utilizing an autoregressive model that predicts quantified latent values ​​from the causal context of the latent values ​​(context model).

[0074] The term autoregressive means that the output of a process is later used as its input. For example, a contextual model subnetwork generates a sample of latent values, which is later used as input to obtain the next sample.

[0075] In the existing design, a joint architecture is used, which utilizes both a super-prior model subnetwork (an encoder and a decoder utilizing super-prior information) and a context model subnetwork. The super-prior and context models are combined to learn quantized latent values. The probability model was then used for entropy encoding and decoding. For example... Figure 5As shown, the outputs of the context subnetwork and the decoder subnetwork utilizing prior information are referred to as the entropy parameter subnetwork combination, which generates the mean μ and scale (or variance) σ parameters for the Gaussian probability model. The Gaussian probability model is then used to encode samples of the quantized latent values ​​into the bitstream via the arithmetic encoder (AE) module. In the decoder, the Gaussian probability model is used to obtain the quantized latent values ​​from the bitstream via the arithmetic decoder (AD) module.

[0076] Figure 5 A block diagram of the combined model is shown. The combined model jointly optimizes an autoregressive component, which, along with a super-prior and a low-level autoencoder, estimates the probability distribution of latent values ​​from the causal context of the latent values ​​(context model). Real-valued latent representations are quantized (Q) to create quantized latent values. and quantified potential value of prior information They are compressed into a bitstream using an arithmetic encoder (AE) and decompressed by an arithmetic decoder (AD). The highlighted areas correspond to the components performed by the receiver (i.e., the decoder) to recover the image from the compressed bitstream.

[0077] Typically, potential samples are modeled as Gaussian distributions or Gaussian mixture models (not limited to these). In existing designs, according to Figure 5 The context model and the super-prior are jointly used to estimate the probability distribution of potential samples. Since the Gaussian distribution can be defined by the mean and variance (also known as sigma or scale), the joint model is used to estimate the mean and variance (denoted as μ and σ).

[0078] 2.3.5. Gain Variational Autoencoder (G-VAE)

[0079] Typically, neural network-based image / video compression methods require training multiple models to adapt to different bit rates. Gain Variational Autoencoders (G-VAEs) are variational autoencoders with a pair of gain units. G-VAEs are designed to achieve continuously variable bit rate adaptation using a single model. A G-VAE consists of a pair of gain units, which are typically inserted into the encoder's output and the decoder's input. The encoder's output is defined as the latent representation y∈R. c*h*w , where c, h, w represent the number of channels, the height of the latent representation, and the width. Each channel of the latent representation is represented as y. (i) ∈R h*w Where i = 0, 1, ..., c-1. A pair of gain units includes a gain matrix M ∈ R. c*n And the inverse gain matrix, where n is the number of gain vectors. The gain vector can be represented as m s ={α s(0) ,α s(1) ,…,αs(c-1)}, α s(i) ∈R, where s represents the index of the gain vector in the gain matrix.

[0080] The motivation for the gain matrix is ​​similar to that in JPEG, where a quantization table controls the quantization loss based on the characteristics of different channels. To apply the gain matrix to the latent representation, the corresponding value in the gain vector for each channel is multiplied.

[0081]

[0082] Where ⊙ represents channel-wise multiplication, that is... And α s(i) It is the gain vector m s The i-th gain value in the matrix. The inverse gain matrix used on the decoder side can be represented as M′∈R. c*n It consists of n inverse gain vectors, i.e., M′={δ s(0) ,δ s(1) ,…,δ s(c-1)}, δ s(i) ∈R. The inverse gain process is represented as:

[0083]

[0084] in It is the decoded quantized latent representation, and y′ s It is the quantized latent representation of the inverse gain that will be fed into the synthesis network.

[0085] To achieve continuous variable bit rate adjustment, interpolation is used between vectors. Given two pairs of gain vectors {m t ,m′ t} and {m r ,m′ r}, and the interpolated gain vector can be obtained through the following equation.

[0086] m v =[(m r ) l ·(m t ) 1-l ]

[0087] m′ v =[(m′ r ) l ·(m′ t ) 1-l ]

[0088] Where l∈R are interpolation coefficients, which control the corresponding bit rate of the generated gain vector pairs. Since l is a real number, any bit rate between two given gain vector pairs can be achieved.

[0089] 2.3.6. Encoding process using a joint autoregressive hyperprior model

[0090] Figure 5 This corresponds to the state of the technical compression method proposed in the existing design. The encoding and decoding processes will be described in this section and the next section, respectively.

[0091] Figure 6 The encoding process is described. The input image is first processed using an encoder subnetwork. The encoder transforms the input image into a transformed representation called the latent value, denoted by y. y is then fed into a quantizer block, denoted by Q, to obtain the quantized latent value. Then The arithmetic coding module (denoted as AE) will be used to convert it into a bitstream (bits1). The arithmetic coding blocks are sequentially converted... Each sample is converted into a bitstream (bits1).

[0092] The module utilizes an encoder with prior information, context, a decoder with prior information, and an entropy parameter subnetwork to estimate the quantized latent value. The probability distribution of the sample points. The latent value y is input to an encoder that utilizes prior information, and the encoder outputs the prior information latent value (denoted by z). The prior information latent value is then quantized. The second bitstream (bits2) is generated using the arithmetic coding (AE) module. The decompositional entropy module generates a probability distribution, which is used to encode the quantized prior information latent values ​​into the bitstream. The quantized prior information latent values ​​include information about the quantized latent values. Information about the probability distribution.

[0093] Entropy parameter subnetworks are generated to encode quantized latent values. The probability distribution estimation. Information generated from the entropy parameter is typically used together to obtain the mean μ and scale (or variance) σ parameter of the Gaussian probability distribution. The Gaussian distribution of the random variable x is defined as... Here, parameter μ is the mean or expected value of the distribution (as well as its median and mode), while parameter σ is its standard deviation (or variance or scale). To define a Gaussian distribution, the mean and variance need to be determined. In the existing design, the entropy parameter module is used to estimate the mean and variance values.

[0094] The subnetwork uses a decoder with prior information to generate a portion of the information used by the entropy parameter subnetwork; the remaining portion is generated by an autoregressive module called the context module. The context module uses samples already encoded by the arithmetic encoding (AE) module to generate information about the probability distribution of the quantized latent values. Typically, it is a matrix composed of many sample points. Sample points can be used in various ways, such as... or The index is used to indicate this, which depends on the matrix. Dimensions. Sample points The samples are encoded sequentially by the AE (Automatic Image Processor), typically using a raster scan order. In the raster scan order, the matrix is ​​processed row by row from top to bottom, with samples in a row processed from left to right. In this case (where the raster scan order is used by the AE to encode the samples into the bitstream), the context module uses samples encoded before the raster scan order to generate a sample stream. The relevant information, generated by the context module and the decoder utilizing prior information, is combined by the entropy parameter module to generate the quantized latent value. The probability distribution encoded into the bit stream (bits1).

[0095] Finally, as a result of the encoding process, the first bitstream and the second bitstream are transmitted to the decoder.

[0096] Note that other names can be used for the modules described above. In the above description, Figure 6 All elements in the encoder are collectively referred to as encoders. The analytic transformation that converts the input image into a latent representation is also called an encoder (or autoencoder).

[0097] 2.3.7. Decoding process using a joint autoregressive superprior model

[0098] Figure 7 The decoding process is described separately. During decoding, the decoder first receives a first bitstream (bits1) and a second bitstream (bits2) generated by the corresponding encoder. Bits2 is first decoded by the arithmetic decoding (AD) module using a probability distribution generated by a decompositional entropy subnetwork. The decompositional entropy module typically generates the probability distribution using a predetermined template, such as a predetermined mean and variance in the case of a Gaussian distribution. The output of the arithmetic decoding process for bits2 is... This is the quantized potential value of the prior information. The AD process recovers to the AE process applied in the encoder. The AE and AD processes are lossless, which means that the quantized potential value of the prior information generated by the encoder is... It can be reconstructed at the decoder without any changes.

[0099] In obtaining after, The data is processed by a decoder utilizing prior information, and the output of the decoder is fed into the entropy parameter module. The three sub-networks, context, decoder utilizing prior information, and entropy parameters used in the decoder are the same as those in the encoder. Therefore, the exact same probability distribution can be obtained in the decoder (as in the encoder), which is beneficial for reconstructing the quantized latent value without any loss. It is necessary. As a result, the quantized latent value obtained in the encoder... The same version can be obtained in the decoder.

[0100] After obtaining the probability distribution (e.g., mean and variance parameters) through the entropy parameter subnetwork, the arithmetic decoding module decodes the samples of quantized latent values ​​one by one from the bitstream bits1. From a practical perspective, autoregressive models (contextual models) are inherently serial and therefore cannot be accelerated using techniques such as parallelization.

[0101] Finally, the fully reconstructed quantified potential value Input to synthesis transformation (in) Figure 7 The module (represented as decoder) is used to obtain the reconstructed image.

[0102] In the above description, Figure 7 All elements in the array are collectively referred to as the decoder. The synthetic transformation that converts the quantized latent values ​​into the reconstructed image is also called the decoder (or autodecoder).

[0103] 2.4. Neural Networks for Video Compression

[0104] Similar to conventional video encoding and decoding techniques, neural image compression serves as the foundation for intra-frame compression in neural network-based video compression. Therefore, the development of neural network-based video compression technology lagged behind that of neural network-based image compression, but more effort is needed to address the challenges arising from its complexity. Since 2017, some researchers have been working on neural network-based video compression schemes. Compared to image compression, video compression requires effective methods to remove redundancy between images. Subsequently, inter-image prediction is a key step in these works. Motion estimation and compensation have been widely adopted, but only recently have they been implemented using trained neural networks.

[0105] Research on neural network-based video compression can be divided into two categories based on the target scenario: random access and low latency. In the random access case, decoding can begin at any point in the sequence, typically dividing the entire sequence into multiple separate segments, each of which can be decoded independently. In the low latency case, the aim is to reduce decoding time, so usually only temporally preceding frames can be used as reference frames to decode subsequent frames.

[0106] 2.5. Basic Knowledge

[0107] Almost all natural images / videos are in digital format. Grayscale digital images can be generated by... It means that, among them Let m be a set of pixel values, m be the image height, and n be the image width. For example, It's a general setting, and in this case... Therefore, a pixel can be represented by an 8-bit integer. An uncompressed grayscale digital image has 8 bits per pixel (bpp), while compressed images necessarily have fewer bits.

[0108] Color images are typically represented in multiple channels to record color information. For example, in the RGB color space, an image can be represented by... This means that red, green, and blue information are stored using three separate channels. Similar to an 8-bit grayscale image, an uncompressed 8-bit RGB image has 24 bpp. Digital images / videos can be represented in different color spaces. Neural network-based video compression schemes are primarily developed in the RGB color space, while traditional codecs typically use the YUV color space to represent video sequences. In the YUV color space, an image is decomposed into three channels: Y, Cb, and Cr, where Y is the luminance component and Cb / Cr are the chrominance components. Because the human visual system is less sensitive to the chrominance component, Cb and Cr are often downsampled for pre-compression.

[0109] A color video sequence consists of multiple color images (called frames) to record scenes at different timestamps. For example, in the RGB color space, a color video can be composed of X = {x0, x1, ..., x...} t ,…,x T-1} represents, where T is the number of frames in the video sequence. If m = 1080 and n = 1920, Furthermore, if the video has 50 frames per second (fps), then the data bit rate of the uncompressed video is 1920×1080×8×3×50=2,488,320,000 bits per second (bps), approximately 2.32Gbps. This requires a large amount of storage, so it must be compressed before being transmitted over the Internet.

[0110] Typically, for natural images, lossless methods can achieve compression ratios of around 1.5 to 3, which is clearly below the required level. Therefore, lossy compression is developed to achieve further compression ratios, but at the cost of introducing distortion. Distortion can be measured by calculating the mean squared difference (i.e., mean squared error (MSE)) between the original and reconstructed images. For grayscale images, MSE can be calculated using the following equation.

[0111]

[0112] Therefore, the quality of the reconstructed image compared to the original image can be measured by the peak signal-to-noise ratio (PSNR):

[0113]

[0114] in for The maximum value in the range is 255, for example, for an 8-bit grayscale image. Other quality assessment metrics exist, such as structural similarity (SSIM) and multi-scale SSIM (MS-SSIM).

[0115] To compare different lossless compression schemes, comparing the compression ratio at a given result bitrate is sufficient, or vice versa. However, to compare different lossy compression methods, both bitrate and reconstruction quality must be considered. For example, a common method is to calculate the relative bitrates for several different quality levels and then average these bitrates; the average relative bitrate is called the Bjontegaard delta-rate (BD-rate). Other important aspects for evaluating image / video encoding / decoding schemes include encoding / decoding complexity, scalability, and robustness.

[0116] 2.5.1. Separate processing of the luminance and chrominance components of an image

[0117] According to one implementation, a separate sub-network can be used to decode the luminance and chrominance components of an image. Figure 8 Examples of decoding processes according to some embodiments of this disclosure are shown. Figure 8 In this process, the luminance component of the image is processed by sub-networks such as "synthesis", "predictive fusion", "mask convolution", "Hyper Decoder using advanced prior information", and "Hyper Scale Decoder using advanced prior information". The chrominance component is processed by sub-networks such as "synthesis UV", "predictive fusion UV", "mask convolution UV", "decoder UV using advanced prior information", and "variance decoder UV using advanced prior information".

[0118] The advantage of this separate processing is that the computational complexity of image processing is reduced by applying individual processing steps. Typically, in neural network-based image and video decoding, computational complexity is proportional to the square of the number of feature maps. For example, if the total number of feature maps is 192, the computational complexity will be proportional to 192 x 192. On the other hand, if the feature maps are divided for luminance of 128 and chrominance of 64 (in the case of separate processing), the computational complexity is proportional to 128 x 128 + 64 x 64, which corresponds to a 45% reduction in complexity. Generally, separate processing of the luminance and chrominance components of an image does not lead to excessive performance degradation because the correlation between the luminance and chrominance components is usually very small.

[0119] Figure 8 The processing (decoding process) in the code can be explained as follows:

[0120] 1. First, a decompositional entropy model is used to decode the quantized latent values ​​for luminance and chrominance, i.e., Figure 8 In and

[0121] 2. The probability parameters (e.g., variance) generated by the second network are used to generate quantized residual latent values ​​by performing an arithmetic decoding process.

[0122] 3. The quantized residual potential value is inversely amplified using an inverse gain unit (iGain), such as... Figure 8 The orange color is shown in the figure. The outputs of the inverse gain unit for the luminance and chrominance components are expressed as follows: and

[0123] 4. For the luminance component, perform the following steps in a loop until all elements are obtained. until:

[0124] a. The first subnetwork was used to utilize the already obtained quantized latent values. To estimate the sample points The mean parameter.

[0125] b. Quantified residual potential value The mean was used to obtain The next element.

[0126] 5. In obtaining After all the samples are processed, a synthetic transformation can be applied to obtain the reconstructed image.

[0127] 6. For the chromaticity component, steps 4 and 5 are the same, but a separate set of networks is used.

[0128] 7. The decoded luminance component is used as additional information to obtain the chrominance component. Specifically, an inter-channel correlated information (ICCI) filter subnetwork is used for chrominance component recovery. Luminance is fed as additional information into the ICCI subnetwork to assist in chrominance component decoding.

[0129] 8. After the luminance and chrominance components are reconstructed, adaptive color transformation (ACT) is performed.

[0130] The module named ICCI is a neural network-based post-processing module. This disclosure is not limited to the UCCI subnetwork; any other neural network-based post-processing module may also be used.

[0131] exist Figure 8 (Decoding Process) illustrates example implementations of some embodiments of this disclosure. The framework includes two branches, one for the luma component and the other for the chroma component. In each branch, the first sub-network includes a context, a prediction, and an optional decoder module utilizing prior information. The second network includes a variance decoder module utilizing prior information. The quantized potential value of the prior information is... and The arithmetic decoding process generates quantized residual latent values, which are then fed into the iGain unit to obtain gain-quantized residual latent values. and

[0132] After the residual latent value is obtained, a recursive prediction operation is performed to obtain the latent value. and The following steps describe how to obtain samples of potential values. Furthermore, the chromaticity components are processed in the same way but using different networks.

[0133] 1. The autoregressive context module is used when using samples. The first input to the prediction module is generated, where the (m, n) pairs are the indices of the already obtained samples of the potential values.

[0134] 2. Optionally, the second input to the prediction module is obtained by using a decoder that utilizes prior information and quantized potential values ​​of the prior information. And thus obtained.

[0135] 3. Using the first and second inputs, the prediction module generates the mean [:,i,j].

[0136] 4. The mean [:,i,j] and the quantized residual latent values ​​are summed together to obtain the latent value.

[0137] 5. Repeat steps 1 to 4 for the next sample point.

[0138] Whether and / or how to apply at least one of the methods disclosed in this document can, for example, be transmitted from the encoder to the decoder in a bitstream via signal transmission.

[0139] Alternatively, whether and / or how to apply at least one method disclosed in the document can be determined by the decoder based on encoding / decoding information such as size, color format, etc.

[0140] Alternate or additional land, (in) Figure 8 Modules named MS1, MS2, or MS3+0 (in Chinese) can be included in the processing flow. These modules can perform operations on their inputs, obtaining an output by multiplying the input by a scalar or adding an additive component to the input. The scalar or additive component used by the module can be indicated in the bitstream.

[0141] Modules named RD or Figure 8 The module named AD in the code can be an entropy decoding module. An entropy decoding module can be a range decoder or an arithmetic decoder, etc.

[0142] The solutions described in this article are not limited to Figure 8 The examples illustrate specific combinations of units. Some modules may be missing, and some modules may be shifted according to processing order. Additional modules may also be included. For example:

[0143] 1. The ICCI module can be removed. In this case, the outputs of the synthesis module and the synthesis UV module can be combined by another module, which can be based on a neural network.

[0144] 2. One or more modules named MS1, MS2, or MS3+O can be removed. The core of the solution is unaffected by the removal of one or more of the aforementioned scaling and adding modules.

[0145] exist Figure 8 In the bitstream, star symbols are also used to indicate other operations performed during the processing of the luminance and chrominance components. These operations are represented as MS1, MS2, MS3+0. These operations can be, but are not limited to, adaptive quantization, latent sample scaling, and latent sample compensation operations. For example, in adaptive quantization, this can correspond to scaling the samples using a multiplier before the prediction process, where the multiplier is predefined or its value is indicated in the bitstream. Latent scaling can correspond to scaling the samples using a multiplier after the prediction process, where the multiplier value is predefined or indicated in the bitstream. Compensation operations can correspond to adding an additive element to the sample, again where the value of the additive element can be indicated, estimated, or predetermined in the bitstream.

[0146] Another operation can be slicing, where samples are first sliced ​​(grouped) into overlapping or non-overlapping regions, each of which is processed independently. For example, samples corresponding to the luminance component can be divided into slices with a slice height of 20 samples, while chrominance components can be divided into slices with a slice height of 10 samples for processing.

[0147] Another application is wavefront parallel processing. In wavefront parallel processing, multiple samples can be processed in parallel, and the number of samples that can be processed in parallel can be indicated by control parameters. These control parameters can be indicated, estimated, or predetermined in the bitstream. In the case of separate luma and chroma processing, the number of samples that can be processed in parallel can be different, so different indicators can be transmitted via signals in the bitstream to control the operation of luma and chroma processing separately.

[0148] 3. Problem

[0149] 3.1. Core Issues

[0150] Section 2.5.1 illustrates different processes for the separate processing of the luma and chroma components. Since the luma and chroma components are processed separately, different sets of control parameters need to be transmitted via signals in the bitstream to control different processing stages. For example, the adaptive quantization process for the luma component will require the transmission of one set of indicators via signals, and the adaptive quantization process for the chroma component will require the transmission of a second set of parameters via signals. Similarly, for sample scaling, sample compensation, sample scrambling, etc., two sets of control parameters will be required: one set to control the behavior of the processing steps for the luma component, and a second set to control the behavior of the processing steps for the chroma component.

[0151] The necessity of transmitting two sets of control parameters for the luminance and chrominance components separately via signals leads to an increase in the bit rate, and consequently a decrease in compression performance.

[0152] 4. Detailed Solution

[0153] The detailed solutions below should be considered as examples for explaining general concepts. These solutions should not be interpreted in a narrow way. Furthermore, these solutions can be combined in any way.

[0154] The proposed solution relates to the individual processing of image components using neural networks. A mechanism for sharing control parameters is disclosed, wherein the processing of the luminance and chrominance components can share some or all of a set of control parameters.

[0155] 4.1. Core of the Solution

[0156] The proposed solution aims to provide a mechanism for transmitting control parameters via signals to an encoding / decoding tool that can be used to process at least two components of an image. In one example, one component could be a luminance component, and the second component could be a chrominance component.

[0157] 4.2. Details of the Solution

[0158] exist Figure 8 The image depicts a compression network where the luminance and chrominance components of an image are reconstructed separately. Furthermore, example operations for processing residual latent samples, such as MS2, are described. The MS2 process can be, for example, an adaptive quantization process, where the input to the MS2 process (the quantized residual samples) is multiplied by a multiplier transmitted through the signal in the bitstream. Adaptive quantization processes typically include:

[0159] • At the encoder, the scaling process of residual samples before quantization.

[0160] • Quantization of scaled residual samples at the encoder.

[0161] • Reception of quantized residual samples at the decoder.

[0162] • Descaling operation at the decoder.

[0163] The value of the scalar (used in descaling) needs to be included in the bitstream so that the decoder can successfully perform the descaling operation. In this case, the control parameter of the adaptive quantization process is a scalar value that controls the magnitude of the scaling operation.

[0164] Based on the proposed solution:

[0165] A neural network-based image or video decoding method, wherein the image or video comprises at least multiple components, includes the following steps:

[0166] - Obtain the indicator from the bitstream.

[0167] - Obtain a set of control parameters,

[0168] - If the value of the indicator is equal to a predefined value, then this set of control parameters is used in the processing of both components of the image or video.

[0169] - If the value of the indicator is equal to the second predefined value, then the set of control parameters is used in the processing of only one of the multiple components.

[0170] Combine at least two processed components to reconstruct an image or video.

[0171] According to the proposed solution, an indicator is included in the bitstream. The indicator controls the use of control parameters associated with the encoding / decoding tools (e.g., adaptive quantization illustrated above). If the indicator assumes a predefined value “A”, for example, “A” can be equal to 1, then the same set of control parameters is used in the processing of both components of the image. The multiple components can be a luminance component and a chrominance component. In another example, the multiple components can be a red component and / or a green component and / or a blue component. On the other hand, if the value of the indicator is equal to a second predefined value “B”, for example, “B” can be equal to 0, then the set of control parameters is applied in the processing of only one of the multiple components. In this case, the other component may not be processed using the process utilizing the set of control parameters, or the second set of control parameters may be used.

[0172] Based on an example implementation of the proposed solution, three indicators can be included in the bitstream, with the following functionality:

[0173] 1. The first indicator indicates whether the encoding / decoding tool has been used to process a component of the image.

[0174] 2. The second indicator indicates whether the encoding / decoding tool is used for processing other components of the image.

[0175] 3. If two indicators suggest that the encoding / decoding tools were used for both components, a third indicator is included in the bitstream to indicate whether the control parameters used to process the two components are the same. If the control parameters are indicated to be the same, only a single set of control parameters is included in the bitstream to process both components.

[0176] The encoding / decoding tools controlled by the control parameter set can be (but are not limited to) adaptive quantization, sample scaling, skip mode, slicing, sample compensation, wavefront parallel processing, etc. For example, skip mode corresponds to processing samples in a way that determines whether a sample is included in the bitstream based on a threshold. The control parameter set corresponding to skip mode can include the value of the threshold. Slicing corresponds to grouping samples into at least two groups and processing them independently or in parallel. The control parameter set corresponding to slicing can include slice size or the number of slices or slice partitioning mode.

[0177] According to another example implementation of the proposed solution, the indicator can be included in the bitstream to indicate one of the following options:

[0178] 1. Encoding and decoding tools are disabled for processing all components.

[0179] 2. For processing a single component, the encoding / decoding tool is enabled. For this encoding / decoding tool, only one set of control parameters is included in the bitstream.

[0180] 3. For the processing of the two components, the encoding / decoding tool is activated. For this encoding / decoding tool, only one set of control parameters is included in the bitstream and applied to the processing of the two components.

[0181] 4. For processing the two components, the encoding / decoding tool is activated. For this tool, only one set of control parameters is included in the bitstream. The first component and / or the second component uses control parameters transmitted via signal transmission. The second component and / or the first component uses a derived set of control parameters. The derived operations can be, but are not limited to, rescaling, resampling, etc.

[0182] 5. For the processing of the two components, the encoding / decoding tool is activated. For this encoding / decoding tool, two sets of control parameters are included in the bitstream, and one set of control parameters is used in the processing of the first component, while the second set of control parameters is used in the processing of the second component.

[0183] According to another example implementation of the proposed solution, an indicator can be included in the bitstream to indicate whether a set of control parameters for one component is the same as that for other components. As an example:

[0184] 1. The first set of control parameters is included in the bitstream and used for the processing of the first component.

[0185] 2. An indicator is included in the bitstream to indicate whether the processing of the second component uses the same set of control parameters as the first component. If so, the same set of control parameters is used to process the second component. Otherwise, the second set of control parameters is included in the bitstream and used in the processing of the second component. Alternatively, the second set of control parameters can be derived from the first set of control parameters, and the second set of control parameters is used in the processing of the second component.

[0186] According to another example implementation of the proposed solution, a first indicator can be included in the bitstream to indicate the number of control parameter groups, which can be represented as N. Following the first indicator, N second indicators can be included in the bitstream corresponding to each control parameter group. Each second indicator can indicate:

[0187] • Whether the corresponding control parameter group applies only to the first component.

[0188] • Whether the corresponding control parameter group applies only to the second component,

[0189] • Whether the corresponding control parameter group is applied to both components.

[0190] In the example above, N control parameter groups are included in the bitstream. These control parameter groups can control the same codec tool. If N control parameter groups are included in the bitstream to control the codec tool, this could correspond to the same codec tool being applied N times repeatedly, with a different group used in each application.

[0191] The bitstream includes N control parameter groups that can correspond to different encoding / decoding tools. One parameter group may correspond to adaptive quantization, while others may correspond to wavefront parallel processing, etc. In the example above, an indicator is included in the bitstream to correspond to at least one of the control parameter groups, and the indicator controls whether the control parameter group is applied to the first component, the second component, or both.

[0192] According to another example implementation of the proposed solution, a control parameter group is included in the bitstream to control the encoding / decoding tools. Corresponding to the control parameter group, an indicator is included in the bitstream for indication:

[0193] • Whether the corresponding control parameter group applies only to the first component.

[0194] • Whether the corresponding control parameter group applies only to the second component,

[0195] • Whether the corresponding control parameter group is applied to both components.

[0196] 4.3. Benefits of the Solution

[0197] The proposed solution reduces the side information required to process multiple components of an image.

[0198] 5. Examples

[0199] 1. Decoder Implementation Example:

[0200] An image or video decoding method, including a neural network, includes the following steps:

[0201] - Obtain the control parameter set from the bitstream.

[0202] - Obtain an indicator from the bitstream that indicates whether the control parameter group is used during the processing of the first component, the processing of the second component, or the processing of both the first and second components.

[0203] - Process one component using the first neural subnetwork and process the second component using the second neural subnetwork.

[0204] The reconstructed image is obtained by combining the outputs of the first neural subnetwork and the second neural subnetwork.

[0205] Further details of embodiments of this disclosure relating to neural network-based visual data encoding and decoding will now be described. As used herein, the term "visual data" may refer to video, images, pictures in video, or any other visual data suitable for encoding and decoding.

[0206] As mentioned above, in existing designs, the values ​​of the control parameter set are transmitted via signals for the luma and chroma components separately. For example, even if the scaling factor used in the sample scaling process is the same for both the luma and chroma components, the value of this scaling factor needs to be transmitted via signal twice—once for the luma component and once for the chroma component. This leads to an increase in bit rate and consequently, a decrease in compression performance and encoding / decoding efficiency.

[0207] To address the aforementioned problems and some unmentioned issues, a visual data processing solution is disclosed as described below. The embodiments of this disclosure should be considered as examples for explaining general concepts and should not be interpreted in a narrow manner. Furthermore, these embodiments can be applied individually or in combination in any way.

[0208] Figure 9 A flowchart of a method 900 for visual data processing according to some embodiments of the present disclosure is shown. Figure 9 As shown, at 902, a conversion between visual data and a bitstream of visual data is performed using a neural network (NN)-based model. In some embodiments, the conversion may include encoding the visual data into a bitstream. Additionally or alternatively, the conversion may include decoding the visual data from the bitstream. For example, Figure 8 The decoding model shown can be used to decode visual data from a bitstream.

[0209] In some embodiments, the bitstream includes a first indicator indicating whether a set of values ​​for a set of parameters of a neural network-based model is common to the processing of multiple components of the visual data. As an example, and not a limitation, the first indicator could be a flag, a syntax element, etc. For example, if a set of values ​​for a set of parameters is common to the processing of multiple components, then each of the multiple components can be processed using the same set of values ​​for the set of parameters. In other words, the set of values ​​for the set of parameters is the same for processing multiple components.

[0210] In some embodiments, the multiple components may include a luminance component, a chroma component, a red component, a green component, a blue component, a Y component, a U component, a V component, a chroma blue (Cb) component, and / or a chroma red (Cr) component. It should be understood that the multiple components may include any other suitable components, such as an alpha component for transparency. The scope of this disclosure is not limited in this respect.

[0211] In some embodiments, multiple components of the visual data can be processed at least partially and individually. For example, the multiple components may include the luminance and chrominance components of the visual data. (See reference) Figure 8 The luminance and chrominance components can be processed separately using modules consisting of the same sequence of neural network layers, with differences in the size of the input tensors and the number of tensor channels. A first indicator can be used to indicate whether the value of at least one parameter of the NN-based model is common to the processing of the luminance and chrominance components. If the value of at least one parameter is common to the processing of the luminance and chrominance components, then the value of at least one parameter can be transmitted via signal only once, instead of being transmitted via signal for the luminance and chrominance components separately. This reduces the side information required to process multiple components of visual data.

[0212] In light of the above, an indication is included in the bitstream, and this indication is used to indicate whether the processing of a set of values ​​for a set of parameters of a neural network-based model is common to multiple components of the visual data. With this indication, the transmission of the set of values ​​for each component of the visual data via separate signals can be avoided. Therefore, the proposed method can advantageously improve encoding and decoding efficiency.

[0213] In some embodiments, a set of parameters may be used in a first encoder-decoder tool for a neural network-based model. As used herein, the term "encoder-decoder tool" may refer to any suitable subprocess for processing visual data, and the encoder-decoder tool may be implemented as a tool, unit, module, etc. For example, the first encoder-decoder tool may include adaptive quantization, sample scaling, skip mode, slicing, sample compensation, gain unit, inverse gain unit, filter, inter-channel correlation information (ICCI) filter, gain process, inverse gain process, post-processing module, decoder module utilizing prior information, variance decoder module utilizing prior information, and / or wavefront parallel processing. It should be understood that the possible embodiments of the first encoder-decoder tool described herein are merely illustrative and should therefore not be construed as limiting this disclosure in any way.

[0214] In some embodiments, a set of parameters may include a single parameter. In this case, the set of values ​​may include a single value of the single parameter. Alternatively, the set of parameters may include multiple parameters. In this case, the set of parameters may also include multiple values, and each of the multiple values ​​corresponds to one of the multiple parameters. For example, the number of values ​​in the set of values ​​may be equal to the number of parameters in the set of parameters.

[0215] In some embodiments, a set of parameters may include any suitable parameters for the first codec tool. By way of example and not limitation, depending on the first codec tool, this set of parameters may include scalars, multipliers, vectors, scaling factors, thresholds, slice sizes, the number of slices, slice partitioning modes, indices, models, offsets, additive coefficients, subtractive coefficients, and / or the number of samples processed in parallel. For example, if the first codec tool is a skip mode, the set of parameters may include a threshold for determining whether a sample is included in the bitstream. In another example, if the first codec tool is a slicing process, the set of parameters may include slice size, the number of slices, and / or slice partitioning modes. In another example, the offset may be implemented as a displacement term. It should be understood that the above examples are described for illustrative purposes only. The scope of this disclosure is not limited in this respect.

[0216] In some embodiments, a set of values ​​for a set of parameters can be indicated in the bitstream. For example, on the decoder side, this set of values ​​can be obtained by decoding the bitstream. Alternatively, a set of values ​​for a set of parameters can be predefined. In this case, the set of values ​​does not need to be transmitted via signaling, and therefore the bit rate used for encoding and decoding visual data can be advantageously further reduced.

[0217] In some embodiments, the value of the first indication equal to the first value can be used to indicate that the processing of a set of values ​​for a set of parameters is common to multiple components. Alternatively, the value of the first indication equal to the second value can be used to indicate that the processing of a set of values ​​for a set of parameters is not common to multiple components. The second value differs from the first value. In one example, the first value can be 1 and the second value can be 0. In another example, the first value can be 0 and the second value can be 1. It should be understood that the specific values ​​described herein are intended to be exemplary and not to limit the scope of this disclosure.

[0218] In some embodiments, the bitstream may further include a second indicator indicating whether a first encoding / decoding tool is applied to a first component (e.g., a chroma component, etc.) among multiple components. Furthermore, the bitstream may include a third indicator indicating whether the first encoding / decoding tool is applied to a second component (e.g., a luma component, etc.) among multiple components. The second component is distinct from the first component. If both indicators are used to indicate that the encoding / decoding tool is used in two components, the first indicator may be included in the bitstream to indicate whether a set of values ​​for a set of parameters of a neural network-based model is common to the processing of multiple components.

[0219] In some embodiments, the first indication may also indicate at least one of the following: whether the first codec tool is applied to a plurality of components, at least one component to which the first codec tool is applied, or at least one component to which a set of values ​​of a set of parameters is applied.

[0220] As an example, and not a limitation, in a case involving multiple components including two components of visual data, the first indicator can be used to indicate one of the following options:

[0221] First option: The first encoding / decoding tool is not applied to these two components.

[0222] Second option: The first codec tool is applied to only one of these two components. For the first codec tool, only one set of parameter values ​​is included in the bitstream.

[0223] Third option: The first encoding / decoding tool is applied to both components. For the first encoding / decoding tool, only one set of parameter values ​​is included in the bitstream and used for processing both components.

[0224] Fourth option: The first encoding / decoding tool is applied to both components. For the first encoding / decoding tool, only one set of values ​​for a single set of parameters is included in the bitstream. One of the two components is processed using that set of values ​​transmitted via signaling. The other component is processed using another set of values ​​determined based on that set of values ​​transmitted via signaling.

[0225] Fifth option: The first codec tool is applied to both components. For the first codec tool, two sets of values ​​for a set of parameters are included in the bitstream. One of the two components is processed using the first set of values ​​from these two sets. The other component is processed using the second set of values ​​from these two sets.

[0226] It should be understood that the above options are described for descriptive purposes only. The scope of this disclosure is not limited in this respect.

[0227] In some embodiments, the bitstream may further include a fourth indication of the number of multiple sets of parameters for at least one encoding / decoding tool for the NN-based model. The multiple sets of parameters may include a set of parameters for a first encoding / decoding tool, and the at least one encoding / decoding tool may include the first encoding / decoding tool. In this case, for each set of values ​​in each of the multiple sets of parameters, an indication may be included in the bitstream to indicate whether the processing of that set of values ​​is common to multiple components of the visual data.

[0228] In some embodiments, the bitstream may further include a fifth indication for indicating the number of multiple sets of values ​​for a set of parameters of the first codec tool. In this case, for each of the multiple sets of values, an indication may be included in the bitstream to indicate whether the processing of that set of values ​​is common to multiple components of the visual data.

[0229] In some embodiments, if a set of values ​​for a set of parameters is not common to the processing of multiple components, the first indication may also be used to indicate one or more components to which a set of values ​​for a set of parameters among the multiple components is applied.

[0230] In some embodiments, a set of values ​​for a set of parameters can be used for processing multiple components in a common manner. In this case, each of the multiple components can be processed using a first codec tool that has a set of values ​​for a set of parameters.

[0231] In some embodiments, the processing of a set of parameter values ​​may not be common to multiple components, and this set of values ​​may be applied to a first component among the multiple components. In this case, a second component that differs from the first component may not be processed using the first codec tool. Alternatively, the second component may be processed using a first codec tool with another set of parameter values. In one example, the other set of values ​​may be determined based on this set of values, for example, by rescaling the set of values, resampling the set of values, etc. In another example, the other set of values ​​may be indicated in the bitstream. In yet another example, the other set of values ​​may be predefined.

[0232] According to another embodiment of this disclosure, a non-transitory computer-readable recording medium is provided. This non-transitory computer-readable recording medium stores a bitstream of visual data generated by a method performed by an apparatus for visual data processing. In this method, a conversion between visual data and a bitstream is performed using a neural network (NN)-based model. The bitstream includes a first indication indicating whether the processing of a set of values ​​for a set of parameters of the NN-based model is common to multiple components of the visual data.

[0233] According to further embodiments of this disclosure, a method for storing a bitstream of visual data is provided. According to this method, a conversion between visual data and a bitstream is performed using a neural network (NN)-based model. The bitstream includes a first indication indicating whether the processing of a set of values ​​for a set of parameters of the NN-based model is common to multiple components of the visual data. Furthermore, the bitstream is stored in a non-transitory computer-readable recording medium.

[0234] Embodiments of this disclosure can be described according to the following entries, the features of which can be combined in any reasonable manner.

[0235] Item 1. A method for visual data processing, comprising: performing a conversion between visual data and a bitstream of visual data using a neural network (NN)-based model, wherein the bitstream includes a first indication indicating whether the processing of a set of values ​​for a set of parameters of the NN-based model is common to multiple components of the visual data.

[0236] Item 2. According to the method described in Item 1, a set of parameters is used in the first encoder / decoder tool for the NN-based model.

[0237] Item 3. The method according to any one of Items 1 to 2, wherein the plurality of components includes at least one of the following: luminance component, chromaticity component, red component, green component, blue component, Y component, U component, V component, chromaticity blue (Cb) component, or chromaticity red (Cr) component.

[0238] Item 4. The method according to any one of Items 2 to 3, wherein the first encoding / decoding tool comprises at least one of the following: adaptive quantization, sample scaling, skip mode, slicing, sample compensation, gain unit, inverse gain unit, filter, inter-channel correlation information (ICCI) filter, gain process, inverse gain process, post-processing module, decoder module utilizing prior information, variance decoder module utilizing prior information, or wavefront parallel processing.

[0239] Item 5. The method according to any one of Items 1 to 4, wherein a set of parameters includes at least one of the following: scalar, multiplier, vector, scaling factor, threshold, slice size, number of slices, slice partitioning pattern, index, model, offset, additive coefficient, or number of samples processed in parallel.

[0240] Item 6. The method according to any one of items 1 to 5, wherein a set of values ​​for a set of parameters is indicated or predefined in the bitstream.

[0241] Item 7. The method according to any one of Items 1 to 6, wherein the value of the first indication is equal to a first value used to indicate that the processing of a set of values ​​of a set of parameters is common to multiple components, or the value of the first indication is equal to a second value used to indicate that the processing of a set of values ​​of a set of parameters is not common to multiple components, the second value being different from the first value.

[0242] Item 8. The method according to Item 7, wherein the first value is 1 and the second value is 0, or wherein the first value is 0 and the second value is 1.

[0243] Item 9. The method according to any one of Items 2 to 8, wherein the bit stream further comprises: a second indication for indicating whether a first encoding / decoding tool is applied to a first component of a plurality of components, and a third indication for indicating whether the first encoding / decoding tool is applied to a second component of a plurality of components, the second component being different from the first component.

[0244] Item 10. The method according to any one of items 2 to 9, wherein the first indication is further used to indicate at least one of the following: whether the first codec tool is applied to a plurality of components, at least one component to which the first codec tool is applied, or at least one component to which a set of values ​​of a set of parameters is applied.

[0245] Item 11. The method according to any one of Items 2 to 10, wherein the bit stream further includes a fourth indication for indicating the number of multiple sets of parameters for at least one encoding / decoding tool for an NN-based model, the multiple sets of parameters including a set of parameters for a first encoding / decoding tool, and the at least one encoding / decoding tool including the first encoding / decoding tool, or wherein the bit stream further includes a fifth indication for indicating the number of multiple sets of values ​​for a set of parameters for the first encoding / decoding tool.

[0246] Item 12. The method according to any one of items 1 to 11, wherein if a set of values ​​of a set of parameters is not common to the processing of multiple components, the first indication is further used to indicate the component to which a set of values ​​of a set of parameters among the multiple components is applied.

[0247] Item 13. The method according to any one of items 2 to 12, wherein a set of values ​​of a set of parameters is common to the processing of multiple components, and each of the multiple components is processed by using a first encoding / decoding tool having a set of values ​​of a set of parameters.

[0248] Item 14. The method according to any one of Items 2 to 12, wherein a set of values ​​of a set of parameters is not common to the processing of multiple components, and a set of values ​​of a set of parameters is applied to a first component of the multiple components, and a second component of the multiple components that is different from the first component is not processed using a first encoding / decoding tool.

[0249] Item 15. The method according to any one of Items 2 to 12, wherein a set of values ​​of a set of parameters is not common to the processing of multiple components, and a set of values ​​of a set of parameters is applied to a first component of the multiple components, and a second component of the multiple components that is different from the first component is processed by using a first encoding / decoding tool having a set of parameters that is different from the first set of values.

[0250] Item 16. The method according to Item 15, wherein another set of values ​​is determined based on a set of values, or another set of values ​​is indicated in the bitstream, or another set of values ​​is predefined.

[0251] Item 17. The method according to any one of items 1 to 16, wherein multiple components of the visual data are processed individually, at least in part.

[0252] Item 18. The method according to any one of Items 1 to 17, wherein a set of parameters comprises a single parameter, or wherein a set of parameters comprises multiple parameters.

[0253] Item 19. The method according to any one of items 1 to 18, wherein the visual data includes video, a picture of video, or an image.

[0254] Item 20. The method according to any one of items 1 to 19, wherein the conversion includes encoding visual data into a bitstream.

[0255] Item 21. The method according to any one of items 1 to 19, wherein the conversion includes decoding visual data from a bitstream.

[0256] Item 22. An apparatus for visual data processing, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform a method according to any one of items 1 to 21.

[0257] Item 23. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform a method according to any one of items 1 to 21.

[0258] Item 24. A non-transitory computer-readable recording medium storing a bitstream of visual data generated by a method performed by an apparatus for visual data processing, wherein the method includes: performing a conversion between the visual data and the bitstream using a neural network (NN)-based model, wherein the bitstream includes a first indication indicating whether the processing of a set of values ​​for a set of parameters of the NN-based model is common to multiple components of the visual data.

[0259] Item 25. A method for storing a bitstream of visual data, comprising: performing a conversion between the visual data and the bitstream using a neural network (NN)-based model, wherein the bitstream includes a first indication indicating whether the processing of a set of values ​​for a set of parameters of the NN-based model is common to multiple components of the visual data; and storing the bitstream in a non-transitory computer-readable recording medium.

[0260] Example device

[0261] Figure 10 A block diagram of a computing device 1000 in which various embodiments of the present disclosure may be implemented is shown. The computing device 1000 may be implemented as a source device 110 (or visual data encoder 114) or a destination device 120 (or visual data decoder 124) or may be included in the source device 110 (or visual data encoder 114) or the destination device 120 (or visual data decoder 124).

[0262] It should be understood that, Figure 10 The computing device 1000 shown is for illustrative purposes only and is not intended to imply any limitation on the functionality and scope of the embodiments of this disclosure.

[0263] like Figure 10 As shown, the computing device 1000 includes a general-purpose computing device 1000. The computing device 1000 may include at least one or more processors or processing units 1010, memory 1020, storage units 1030, one or more communication units 1040, one or more input devices 1050, and one or more output devices 1060.

[0264] In some embodiments, the computing device 1000 can be implemented as any user terminal or server terminal with computing capabilities. The server terminal can be a server, a large computing device, etc., provided by a service provider. The user terminal can be, for example, any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, stations, units, devices, multimedia computers, multimedia tablet computers, internet nodes, communicators, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof. It is conceivable that the computing device 1000 can support any type of interface to the user (such as "wearable" circuitry devices, etc.).

[0265] The processing unit 1010 can be a physical processor or a virtual processor, and can perform various processes based on programs stored in the memory 1020. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of the computing device 1000. The processing unit 1010 may also be referred to as a central processing unit (CPU), microprocessor, controller, or microcontroller.

[0266] Computing device 1000 typically includes various computer storage media. Such media can be any media accessible by computing device 1000, including but not limited to volatile and non-volatile media, or removable and non-removable media. Memory 1020 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), or flash memory) or any combination thereof. Storage cell 1030 can be any removable or non-removable media and may include machine-readable media, such as memory, flash drives, disks, or other media that can be used to store information and / or visual data and can be accessed within computing device 1000.

[0267] The computing device 1000 may also include additional removable / non-removable storage media, volatile / non-volatile storage media. Although in Figure 10 Not shown, but a disk drive for reading from and / or writing to a removable non-volatile disk, and an optical disc drive for reading from and / or writing to a removable non-volatile optical disc may be provided. In this case, each drive may be connected to the bus (not shown) via one or more visual data media interfaces.

[0268] The communication unit 1040 communicates with another computing device via a communication medium. Furthermore, the functionality of the components in the computing device 1000 can be implemented by a single computing cluster or multiple computing machines that can communicate via communication connections. Therefore, the computing device 1000 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general-purpose network nodes.

[0269] Input device 1050 can be one or more of various input devices, such as a mouse, keyboard, trackball, voice input device, etc. Output device 1060 can be one or more of various output devices, such as a monitor, speaker, printer, etc. With the aid of communication unit 1040, computing device 1000 can also communicate with one or more external devices (not shown), such as storage devices and display devices. Computing device 1000 can also communicate with one or more devices that enable a user to interact with computing device 1000, or, if necessary, with any device (e.g., network card, modem, etc.) that enables computing device 1000 to communicate with one or more other computing devices. Such communication can be performed via an input / output (I / O) interface (not shown).

[0270] In some embodiments, some or all of the components of computing device 1000 may be deployed in a cloud computing architecture, rather than being integrated into a single device. In a cloud computing architecture, components may be remotely provided and work together to achieve the functionality described herein. In some embodiments, cloud computing provides computing, software, visual data access, and storage services without requiring end users to know the physical location or configuration of the systems or hardware providing these services. In various embodiments, cloud computing provides services via a wide area network (WAN), such as the Internet, using suitable protocols. For example, a cloud computing provider provides applications via a WAN that can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture, along with the corresponding visual data, may be stored on servers at remote locations. Computing resources in a cloud computing environment may be consolidated or distributed across locations in remote visual data centers. Cloud computing infrastructure may provide services through a shared visual data center, although to the user they appear as a single access point. Therefore, a cloud computing architecture can be used to provide the components and functionality described herein from a service provider at a remote location. Alternatively, the components and functionality described herein may be provided by conventional servers or installed directly or otherwise on client devices.

[0271] In embodiments of this disclosure, computing device 1000 may be used to implement visual data encoding / decoding. Memory 1020 may include one or more visual data encoding / decoding modules 1025 having one or more program instructions. These modules are accessible and executable by processing unit 1010 to perform the functions of the various embodiments described herein.

[0272] In an example embodiment of performing visual data encoding, input device 1050 may receive visual data as input 1070 to be encoded. The visual data may be processed, for example, by visual data encoding / decoding module 1025 to generate an encoded bitstream. The encoded bitstream may be provided as output 1080 via output device 1060.

[0273] In an example embodiment of performing visual data decoding, input device 1050 may receive an encoded bitstream as input 1070. The encoded bitstream may be processed, for example, by visual data encoding / decoding module 1025 to generate decoded visual data. The decoded visual data may be provided as output 1080 via output device 1060.

[0274] While this disclosure has been specifically shown and described with reference to preferred embodiments, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of this application as defined by the appended claims. These variations are intended to be covered by the scope of this application. Therefore, the foregoing description of embodiments of this application is not intended to be limiting.

Claims

1. A method for visual data processing, comprising: A neural network (NN)-based model is used to perform a transformation between visual data and a bitstream of the visual data, wherein the bitstream includes a first indication indicating whether the processing of multiple components of the visual data is common to a value of a parameter of the NN-based model. The parameters are used in the first encoder / decoder tool of the NN-based model, and the values ​​of the parameters are indicated in the bitstream. If the value of the parameter is not common to the processing of the plurality of components, then the value of the parameter is applied to a first component of the plurality of components, and a second component of the plurality of components that is different from the first component is processed by the first codec tool having another value of the parameter that is different from the value of the parameter, the other value being indicated in the bitstream.

2. The method according to claim 1, wherein the plurality of components includes at least one of the following: a luminance component or a chromaticity component.

3. The method according to claim 1, wherein the first encoding / decoding tool comprises at least one of the following: Sample scaling unit, Sample compensation unit, or Gain unit.

4. The method of claim 1, wherein the parameter includes an offset.

5. The method of claim 1, wherein the value of the first indication is equal to a first value used to indicate that the value of the parameter is common to the processing of the plurality of components, or The first indicated value is equal to the second value used to indicate that the value of the parameter is not common to the processing of the plurality of components, and the second value is different from the first value.

6. The method of claim 5, wherein the first value is 0 and the second value is 1.

7. The method of claim 1, wherein the first indication is further used to indicate at least one component in which the value of the parameter among the plurality of components is applied.

8. The method of claim 1, wherein if the value of the parameter is not common to the processing of the plurality of components, the first indication is further used to indicate the component in which the value of the parameter is applied among the plurality of components.

9. The method of claim 1, wherein if the value of the parameter is common to the processing of the plurality of components, then each of the plurality of components is processed by using the first codec tool having the value of the parameter.

10. The method of claim 1, wherein the plurality of components of the visual data are processed individually, at least in part.

11. The method of claim 1, wherein the visual data includes video or images.

12. The method according to any one of claims 1 to 11, wherein the conversion comprises encoding the visual data to generate the bitstream.

13. The method according to any one of claims 1 to 11, wherein the conversion comprises decoding the visual data from the bitstream.

14. The method according to any one of claims 1 to 11, wherein the conversion comprises: The bitstream is generated from the visual data, and The method further includes storing the bit stream in a non-transitory computer-readable recording medium.

15. An apparatus for visual data processing, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 14.

16. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method according to any one of claims 1 to 14.

17. A method for storing a bitstream of visual data, comprising: The visual data is encoded using a neural network (NN)-based model to generate the bitstream, wherein the bitstream includes a first indication indicating whether the processing of multiple components of the visual data is common for the value of a parameter of the NN-based model. as well as The bitstream is stored in a non-transitory computer-readable recording medium. The parameters are used in the first encoder / decoder tool of the NN-based model, and the values ​​of the parameters are indicated in the bitstream. If the value of the parameter is not common to the processing of the plurality of components, then the value of the parameter is applied to a first component of the plurality of components, and a second component of the plurality of components that is different from the first component is processed by the first codec tool having another value of the parameter that is different from the value of the parameter, the other value being indicated in the bitstream.

Images