Method, apparatus and medium for visual data processing

Abstract

An embodiment of the present disclosure provides a solution for visual data processing. A method for visual data processing is proposed. The method includes obtaining statistical information related to a first representation of visual data, the first representation being generated based on applying a first neural network to the visual data, determining at least one sample from a second representation of the visual data based on the statistical information, a value of the at least one sample being absent in the bitstream, the second representation being obtained by quantizing the first representation, and performing the conversion based on the determination, for conversion between the visual data and a bitstream of the visual data.

Description

[Technical field]

[0001] TECHNICAL FIELD Embodiments of the present disclosure relate generally to visual data processing techniques, and more particularly to neural network-based visual data encoding. [Background technology]

[0002] In the past decade, deep learning has been rapidly developed in various fields, especially in computer vision and image processing. Neural networks were originally invented by interdisciplinary research of neuroscience and mathematics. It has shown powerful abilities in terms of nonlinear transformation and classification. Neural network-based image / video compression technology has made great progress in the past five years. It has been reported that the latest neural network-based image compression algorithms have achieved rate-distortion (RD) performance comparable to that of Versatile Video Coding (VVC). As the performance of neural image compression is continuously improved, neural network-based video compression has become an actively developing research field. However, the coding efficiency of neural network-based visual data coding is generally expected to be further improved. Summary of the Invention

[0003] SUMMARY OF THE DISCLOSURE Embodiments of the present disclosure provide a solution for visual data processing.

[0004] In a first aspect, a method for visual data processing is proposed, for conversion between the visual data and a bitstream of visual data, the method comprising the steps of obtaining statistical information related to a first representation of the visual data, the first representation being generated based on applying a first neural network to the visual data, determining at least one sample from a second representation of the visual data based on the statistical information, a value of the at least one sample being absent in the bitstream, the second representation being obtained by quantizing the first representation, and performing the conversion based on the determination.

[0005] According to the method according to the first aspect of the present disclosure, values ​​of at least one sample of the quantization of the representation of visual data are not included in the bitstream. The proposed method thereby allows skipping at least a part of the quantization during the entropy coding process, and can reduce the time spent on entropy coding. In this way, the proposed method can advantageously increase the coding speed and coding efficiency.

[0006] In a second aspect, an apparatus for visual data processing is proposed, the apparatus comprising a processor and a non-transitory memory having instructions thereon, the instructions, when executed by the processor, causing the processor to perform a method according to the first aspect of the present disclosure.

[0007] In a third aspect, a non-transitory computer-readable storage medium is proposed, the non-transitory computer-readable storage medium storing instructions for causing a processor to execute the method according to the first aspect of the present disclosure.

[0008] In a fourth aspect, another non-transitory computer-readable recording medium is proposed, which stores a bitstream of visual data generated by a method executed by an apparatus for visual data processing, the method comprising the steps of obtaining statistical information related to a first representation of the visual data, the first representation being generated based on applying a first neural network to the visual data, determining at least one sample from a second representation of the visual data based on the statistical information, a value of the at least one sample being absent from the bitstream, the second representation being obtained by quantizing the first representation, and generating the bitstream based on the determination, for conversion between the visual data and the bitstream of visual data.

[0009] In a fifth aspect, a method for storing a bitstream of visual data is proposed, the method comprising the steps of obtaining statistical information related to a first representation of the visual data, the first representation being generated based on applying a first neural network to the visual data, determining at least one sample from a second representation of the visual data based on the statistical information, a value of the at least one sample being absent in the bitstream, the second representation being obtained by quantizing the first representation, generating the bitstream based on the determination, and storing the bitstream on a non-transitory computer-readable recording medium.

[0010] This Summary is provided to introduce in a simplified form some of the concepts that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. [Brief description of the drawings]

[0011] The above and other objects, features, and advantages of exemplary embodiments of the present disclosure will become more apparent through the following detailed description taken in conjunction with the accompanying drawings, in which like reference numerals generally refer to like components. [Figure 1] FIG. 1 shows a block diagram illustrating an example visual data encoding system according to some embodiments of the present disclosure. [Diagram 2] A typical transform coding method is shown below. [Diagram 3] 1 shows an image from the Kodak dataset and different representations of that image. [Figure 4] We present an autoencoder network architecture that implements a hyperprior model. [Diagram 5] A block diagram of a composite model is shown. [Figure 6] 1 shows the encoding process of a composite model. [Figure 7] 4 shows a decoding process for a composite model. [Figure 8] Two possible implementations of the entropy coding and quantization process are shown. [Figure 9] Here is a decoder implementation that corresponds to the encoder implementation. [Figure 10] 1 illustrates two possible implementations of an encoder according to an embodiment of the present disclosure. [Figure 11] 1 illustrates two possible implementations of a decoder according to an embodiment of the present disclosure. [Figure 12] 1 illustrates two possible probability distributions according to an embodiment of the present disclosure. [Figure 13] 1 illustrates an example of an exemplary visual data encoding process according to some embodiments of the present disclosure. [Figure 14] 1 illustrates an example visual data decoding process according to some embodiments of the present disclosure. [Figure 15] 1 illustrates a flowchart of a method for visual data processing according to some embodiments of the present disclosure. [Figure 16]FIG. 1 illustrates a block diagram of a computing device in which various embodiments of the present disclosure may be implemented.

[0012] Throughout the drawings, the same or similar reference numbers typically refer to the same or similar elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Next, the principles of the present disclosure will be described with reference to some embodiments. It should be understood that these embodiments are described for illustrative purposes only, to help those skilled in the art understand and practice the present disclosure, and do not imply any limitation on the scope of the present disclosure. The disclosure described herein can be implemented in various forms other than those described below.

[0014] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0015] References in this disclosure to "one embodiment," "an embodiment," "an exemplary embodiment," and the like indicate that the described embodiment may include a particular feature, structure, or characteristic, but not all embodiments need to include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in connection with an exemplary embodiment, it is submitted that it is within the knowledge of one of ordinary skill in the art to affect such feature, structure, or characteristic in connection with other embodiments, whether or not explicitly described.

[0016] Although terms such as "first" and "second" may be used herein to describe various elements, it should be understood that these elements should not be limited by these terms. These terms are merely used to distinguish one element from another. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element, without departing from the scope of the exemplary embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0017] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the example embodiments. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It is further understood that as used herein, the terms "comprises," "comprising," "has," "having," "includes," and / or "including" specify the presence of stated features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0018] Example Environment 1 is a block diagram illustrating an example visual data encoding system 100 that may utilize techniques of this disclosure. As shown, the visual data encoding system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a visual data encoding device, and the destination device 120 may also be referred to as a visual data decoding device. In operation, the source device 110 may be configured to generate encoded visual data, and the destination device 120 may be configured to decode the encoded visual data generated by the source device 110. The source device 110 may include a visual data source 112, a visual data encoder 114, and an input / output (I / O) interface 116.

[0019] Visual data sources 112 may include sources such as visual data capture devices, examples of which include, but are not limited to, an interface for receiving visual data from a visual data provider, a computer graphics system for generating visual data, and / or combinations thereof.

[0020] The visual data may include one or more pictures or one or more images of a video. The visual data encoder 114 encodes the visual data from the visual data source 112 to generate a bitstream. The bitstream may include a bit sequence that forms an encoded representation of the visual data. The bitstream may include an encoded picture and associated visual data. An encoded picture is an encoded representation of a picture. The associated visual data may include a sequence parameter set, a picture parameter set, and other syntax structures. The I / O interface 116 may include a modulator / demodulator and / or a transmitter. The encoded visual data may be transmitted directly to the destination device 120 through the I / O interface 116 over the network 130A. The encoded visual data may also be stored in a storage medium / server 130B for access by the destination device 120.

[0021] The destination device 120 may include an I / O interface 126, a visual data decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain the encoded visual data from the source device 110 or the storage medium / server 130B. The visual data decoder 124 may decode the encoded visual data. The display device 122 may display the decoded visual data to a user. The display device 122 may be integrated with the destination device 120 or may be external to the destination device 120 configured to interface with an external display device.

[0022] The visual data encoder 114 and the visual data decoder 124 may operate in accordance with a visual data encoding standard, such as a video encoding standard or a still image encoding standard, and other current and / or additional standards.

[0023] Some exemplary embodiments of the present disclosure are described in detail below. It should be understood that section headings are used herein for ease of understanding and are not intended to limit the embodiments disclosed in a section to that section alone. Furthermore, although certain embodiments are described with reference to Versatile Video Coding or other specific visual data codecs, the disclosed techniques are applicable to other encoding techniques. Furthermore, although some embodiments describe an encoding step in detail, it should be understood that the corresponding step of decoding that undoes the encoding is performed by a decoder. Furthermore, the term visual data processing encompasses visual data encoding or compression, visual data decoding or decompression, and visual data transcoding, where visual data is represented from one compressed format to another compressed format or at a different compressed bit rate. [Table 1] TIFF2025506961000003.tif252162TIFF2025506961000004.tif252162TIFF20255069610 00005.tif252163TIFF2025506961000006.tif252162TIFF2025506961000007.tif252162 TIFF2025506961000008.tif252162TIFF2025506961000009.tif250162TIFF20255069610 00010.tif252162TIFF2025506961000011.tif252159TIFF2025506961000012.tif252160 TIFF2025506961000013.tif249161TIFF2025506961000014.tif241164TIFF20255069610 00015.tif242163TIFF2025506961000016.tif239163TIFF2025506961000017.tif252164 TIFF2025506961000018.tif252164TIFF2025506961000019.tif253163TIFF20255069610 00020.tif252163TIFF2025506961000021.tif252163TIFF2025506961000022.tif239163

[0024] In the following, details of embodiments of the present disclosure relating to neural network-based visual data encoding are described. As used herein, the term "visual data" may refer to images, pictures in a video, or any other visual data suitable for being encoded.

[0025] As mentioned above, most modules in existing architectures are massively parallelizable, so GPUs can greatly accelerate these processes. On the other hand, the process of entropy coding (encoding or decoding) is inherently sequential since it involves converting between input samples and bit sequences (i.e., bitstreams). Due to its sequential nature, entropy coding cannot be parallelized, making it a bottleneck process and slowing down the entire coding process.

[0026] To solve the above problems, and some other problems not mentioned, a visual data processing solution is disclosed, as described below.

[0027] 13 illustrates an example visual data encoding process 1300 according to some embodiments of the present disclosure. For example, the visual data encoding process 1300 may be performed by the visual data encoder 114 shown in FIG. It should be understood that the visual data encoding process 1300 may also include additional blocks not shown and / or blocks that are shown may be omitted. The scope of the present disclosure is not limited in this respect.

[0028] As shown in FIG. 13, an analytical transformation may be performed on the visual data 1310 in a first neural network 1312 to obtain a latent representation (denoted as y in FIG. 13) of the visual data 1310. The visual data 1310 may include one or more pictures in an image or video. In block 1314, a residual latent representation (denoted as w in FIG. 13) may be obtained by subtracting a statistical value from the latent representation. The statistical value may be generated by a second neural network 1322 and may indicate a prediction of the latent representation. As a non-limiting example, the statistical value may be a mean (denoted as μ in FIG. 13) of a probability distribution, such as a Gaussian probability distribution. For example, the probability distribution may describe a probability distribution of values ​​of one or more samples of the latent representation. It should be appreciated that the statistical value may also be referred to as a statistical parameter, a probability parameter, a probability distribution parameter, or the like. The scope of the present disclosure is not limited in this respect.

[0029] The statistical value may be included in the statistical information generated by the second neural network 1322. For example, the second neural network 1322 may be referred to as an estimation model. It should be noted that in this application, a model may also be referred to as a module. That is, the second neural network may also be referred to as a first module, and the estimation model may also be referred to as an estimation module. The input of the second neural network 1322 may include the latent representation, and the output of the second neural network 1322 may include the statistical information. In one example, the second neural network 1322 may include a neural network-based sub-network. In another example, the second neural network 1322 may include a first sub-network for generating the statistical value and a second sub-network for generating the further statistical value. As a non-limiting example, the further statistical value may be the variance (denoted as σ in FIG. 13 ) of a probability distribution such as a Gaussian probability distribution. For example, the probability distribution may describe the probability distribution of values ​​of one or more samples of the latent representation. The first sub-network may be a hyperdecoder sub-network and the second sub-network may be a hyperscale decoder sub-network. In one example, the second neural network 1322 may further include a hyperencoder sub-network. It should be understood that the above-mentioned mean and variance are only two illustrative examples of statistics that may be used. Any other suitable statistics, such as standard deviation, may also be used. The scope of the present disclosure is not limited in this respect.

[0030] In the quantization block 1316, the residual latent representation may be quantized to obtain a quantized residual latent representation. The quantized residual latent representation may also be referred to as a quantization of the residual latent representation. In one example, the residual latent representation may be quantized using a rounding function. In another example, the residual latent representation may be quantized using a floor function. Alternatively, the residual latent representation may be quantized using a ceiling function. It should be understood that the above examples are provided for illustrative purposes only. The scope of the present disclosure is not limited in this respect.

[0031] In the grouping block 1318, at least one sample may be determined from the quantization of the representation based on the statistical information. The at least one sample constitutes a group of samples whose values ​​are not coded into the bitstream. Thus, the process of determining the at least one sample may also be referred to as a grouping process, during which at least one sample set may be obtained. In one example, the grouping process may be performed at a sample level. In other words, the decision to at least one sample is made for each of the samples of the quantized residual latent representation. Alternatively, the grouping process may be performed at a block level. More specifically, the samples of the quantized residual latent representation may be divided into a number of blocks. Each of the number of blocks may have a predetermined size of N×M, such as 8×8. Each of N and M is a positive integer. N and M may be indicated in the bitstream. The samples may be grouped based on the number of blocks. In other words, the decision to at least one sample is made for each of the number of blocks, and samples in the same block are divided into the same set.

[0032] In some embodiments, for a sample of the quantization, if a first statistical value corresponding to the sample is less than a first threshold, the at least one sample is determined to be included in the sample, and the sample may be grouped into a second set. Otherwise, the sample is determined not to be included in the at least one sample, and the sample is grouped into a first set different from the second set. In one example, the first statistical value may be the variance described above.

[0033] Alternatively, for a sample of the quantization, if the first statistical value corresponding to the sample is greater than another threshold, which may be the same as or different from the first threshold, the at least one sample may be determined to be included in the sample, and the sample may be grouped into the second set. Otherwise, the sample may be determined not to be included in the at least one sample, and the sample may be grouped into the first set. The above comparison may be performed for each of the samples of the quantized residual latent representation to obtain at least one sample.

[0034] It should be understood that the at least one sample may be determined in any other suitable manner, for example based on a comparison of a value determined based on the first statistical value with a threshold value, or based on a comparison of a threshold value with a probability that the value of the sample is equal to a target value. The probability may be determined based on statistical information. The target value of the sample may be a first predetermined value (such as zero) or a second statistical value (such as an average) corresponding to the sample. This is described in detail in section 4.3 above.

[0035] In some alternative embodiments, the grouping process may be performed at the block level. In such a case, for example, for samples in a block of quantization, if the first statistical value corresponding to each sample in the block is smaller than the fourth threshold, it may be determined that at least one sample includes all samples in the block, and these samples may be grouped into a second set. Otherwise, it may be determined that these samples are not included in the at least one sample, and these samples may be grouped into a first set different from the second set. In one example, the first statistical value may be the variance described above.

[0036] Alternatively, for example, for samples in a block of quantization, if the first statistical value corresponding to each sample in the block is greater than a further threshold, which may be the same as or different from the fourth threshold, it may be determined that the at least one sample includes all samples in the block, and these samples may be grouped into a second set. Otherwise, it may be determined that the samples are not included in the at least one sample, and these samples may be grouped into a first set. The above comparison may be performed for each of the blocks of the quantized residual latent representation to obtain at least one sample.

[0037] It should be understood that the at least one sample may be determined in any other suitable manner, for example, based on a comparison between a threshold and a first metric determined based on a first statistical value corresponding to the samples in the block, or based on a comparison between a threshold and a probability that the value of each sample in the block is equal to a target value, or based on a comparison between a threshold and a second metric determined based on a probability that the value of the samples in the block is equal to a respective target value. As non-limiting examples, the first metric may be an average, minimum or maximum value of the first statistical value. The second metric may be an average, minimum or maximum value of the probability. At least one of the thresholds used may be indicated in the bitstream.

[0038] In some embodiments, the output of the grouping block 1318 may include remaining samples of the quantized residual latent representation other than the at least one sample. For example, the output of the grouping block 1318 may include the first set of samples mentioned above (in FIG. 13, ŵ 1The entropy encoder 1320 may include a first neural network (denoted as seq. 13A) for encoding the first sample set based on statistical information generated by the second neural network. In the example shown in FIG. 13, only the variance is used for the entropy encoding process, although any other suitable statistical value(s) may be used, such as the mean and / or standard deviation. The entropy encoding process performed by the entropy encoder 1320 may be an arithmetic encoding process, a Huffman encoding process, etc.

[0039] Furthermore, although not shown in FIG. 13, the statistical information generated by the second neural network may also be encoded into the bitstream by performing an entropy coding process on the statistical information.

[0040] In special cases, the at least one sample may include all samples of the quantized residual latent representation, in which case values ​​of all samples of the quantized residual latent representation are not coded into the bitstream, but statistical information is signaled in the bitstream.

[0041] As a result, the values ​​of at least one sample of the quantization of the representation of visual data are not included in the bitstream. This allows skipping at least a part of the quantization during the entropy coding process, thereby reducing the time spent on entropy coding. Thus, the proposed method can advantageously improve the coding speed and coding efficiency.

[0042] An exemplary visual data encoding process is described above with respect to Fig. 13, however, it should be understood that any other suitable variations of the visual data encoding process are also contemplated in view of the present disclosure. In another example, the latent representation may be adjusted rather than the residual latent representation, which is exemplarily shown in the left sub-diagram of Fig. 10.

[0043] 14 illustrates an example visual data decoding process 1400 according to some embodiments of the present disclosure. For example, the visual data decoding process 1400 may be performed by the visual data decoder 124 shown in FIG. It should be understood that the visual data decoding process 1400 may also include additional blocks not shown and / or blocks that are shown may be omitted. The scope of the present disclosure is not limited in this respect.

[0044] As shown in FIG. 14, the statistical information may be generated by the second neural network 1422. The statistical information may include a mean μ and / or a variance σ. For example, the second neural network 1422 may be referred to as an estimation model. The input of the second neural network 1422 may include a bit stream, and the output of the second neural network 1422 includes at least one statistical value. In one example, the second neural network 1422 may include a neural network-based sub-network. In another example, the second neural network 1422 may include a first sub-network for generating the first statistical value and a second sub-network for generating the second statistical value. In one example, the first statistical value may be the variance of a Gaussian probability distribution, and the second statistical value may be the mean of the Gaussian probability distribution. The first sub-network may be a hyperscale decoder sub-network, and the second sub-network may be a hyperscale decoder sub-network. It should be understood that the mean and variance described above are just two examples of statistics that may be used. Any other suitable statistic, such as standard deviation, may also be used. The scope of the present disclosure is not limited in this respect.

[0045] In the entropy decoder 1418, the quantized residual latent representation (in FIG. 14, ŵ 1 The entropy decoding process may be performed on the bitstream based on the statistical information to obtain a first sample set of values ​​of .times. ...

[0046] In grouping block 1416, at least one sample may be determined from the quantization of the residual latent representation based on the statistical information in a manner similar to the grouping process described with respect to grouping block 1318 of FIG. 13, and therefore will not be described in detail for brevity.

[0047] Since the value of the at least one sample is not signaled in the bitstream, the value of each of the at least one sample may be estimated. In one example, the value of each of the at least one sample may be determined to a predetermined value. The predetermined value may be a constant scalar number, such as zero. Alternatively, the value of each of the at least one sample may be determined to a third statistical value (such as an average) corresponding to the sample. It should be understood that the value of each of the at least one sample may be estimated in any other suitable manner. The scope of the present disclosure is not limited in this respect. The quantized residual latent representation may be reconstructed in the grouping block 1416 based on the obtained values ​​of the first sample set and the estimated value of the at least one sample. In the above special case, the quantized residual latent representation may be reconstructed in the grouping block 1416 based on the estimated value of the at least one sample.

[0048] In block 1414, a quantized latent representation (denoted as ŷ in FIG. 14) may be obtained by adding the statistic (e.g., the mean, μ, in FIG. 14) and the output of grouping block 1416, i.e., the quantized residual latent representation. In synthesis transformation subnetwork 1412, a synthesis transformation may be performed on the quantized latent representation to obtain reconstructed visual data 1410.

[0049] As a result, the values ​​of at least one sample of the quantization of the representation of visual data are not included in the bitstream. This allows the proposed method to skip at least a part of the quantization during the entropy coding process, thereby reducing the time spent on entropy coding. In this way, the proposed method can advantageously improve the coding speed and coding efficiency.

[0050] An exemplary visual data decoding process is described above with respect to Figure 14, however, it should be understood that any other suitable variations of the visual data decoding process are also contemplated in view of the present disclosure. In another example, a latent representation may be used rather than a residual latent representation, which is exemplarily shown in the lower sub-diagram of Figure 11.

[0051] It should be understood that the above illustrations and / or examples are merely set forth for illustrative purposes. The scope of the present disclosure is not limited in this respect. The embodiments of the present disclosure should be considered as examples for illustrating the general concept and should not be interpreted narrowly. Moreover, these embodiments can be applied individually or combined in any manner.

[0052] FIG. 15 illustrates a flowchart of a method 1500 for visual data processing according to some embodiments of the present disclosure. The method 1500 may be performed during conversion between visual data and a bitstream of visual data. As shown in FIG. 15, the method 1500 begins at 1502 with obtaining statistical information associated with a first representation of the visual data. The first representation is generated based on applying a first neural network to the visual data. In one example, the first neural network may be used to perform an analytical transformation on the visual data. In some embodiments, the first representation may be a latent representation of the visual data or a residual latent presentation of the visual data.

[0053] At 1504, at least one sample is determined from a second representation of the visual data based on the statistical information, the value of the at least one sample being absent from the bitstream, the second representation being obtained by quantizing the first representation.

[0054] At 1506, a transformation is performed based on the determination. In one example, the transformation may include encoding the visual data into a bitstream. Alternatively or additionally, the transformation may include decoding the visual data from the bitstream. It should be appreciated that the above illustrations are provided for illustrative purposes only. The scope of the present disclosure is not limited in this respect.

[0055] Therefore, values ​​of at least one sample of the first representation of the visual data in the second representation are not included in the bitstream. This allows the proposed method to skip at least a part of the second representation during the entropy coding process, thereby reducing the time spent on entropy coding. In this way, the proposed method can advantageously increase the coding speed and coding efficiency.

[0056] In some embodiments, at 1504, a determination is made that at least one sample includes a sample of the second representation if at least one of the following conditions is met: a first statistical value corresponding to the sample is less than a first threshold, a value determined based on the first statistical value is less than a second threshold, or a probability that the value of the sample is equal to the target value is less than a third threshold.

[0057] In some embodiments, at 1504, a determination is made that at least one sample includes a sample within a block of the second representation if at least one of the following conditions is met: a first statistical value corresponding to each sample in the block is less than a fourth threshold; a first metric that may be determined based on the first statistical value corresponding to the samples in the block is less than a fifth threshold; a probability that the value of each sample in the block is equal to a target value is less than a sixth threshold; or a second metric that may be determined based on the probability that the values ​​of the samples in the block are equal to their respective target values ​​is less than a seventh threshold.

[0058] In some embodiments, at 1504, a determination is made that at least one sample includes a sample of the second representation if at least one of the following conditions is met: a first statistical value corresponding to the sample is greater than an eighth threshold, a value determined based on the first statistical value is greater than a ninth threshold, or a probability that the value of the sample equals the target value is greater than a tenth threshold.

[0059] In some embodiments, at 1504, a determination is made that at least one sample includes a sample within a block of the second representation if at least one of the following conditions is met: a first statistical value corresponding to each sample in the block is greater than an eleventh threshold; a first metric that may be determined based on the first statistical value corresponding to the samples in the block is greater than a twelfth threshold; a probability that the value of each sample in the block is equal to a target value is greater than a thirteenth threshold; or a second metric that may be determined based on the probability that the values ​​of the samples in the block are equal to their respective target values ​​is greater than a fourteenth threshold.

[0060] In some embodiments, the first metric may be the average, minimum, or maximum of the first statistical value. Additionally or alternatively, the second metric may be the average, minimum, or maximum of the probability.

[0061] In some embodiments, the size of the block is N×M, where each of N and M is a positive integer. In one example, N and M may be indicated in the bitstream. In some embodiments, the first statistical value may be a variance. In some embodiments, the target value for the sample may include one of a first predetermined value or a second statistical value corresponding to the sample. In one example, the first predetermined value may be zero or the second statistical value may be an average. In some embodiments, the probability may be determined based on the statistical information.

[0062] In some embodiments, at least one of the following thresholds may be indicated in the bitstream: a first threshold, a second threshold, a third threshold, a fourth threshold, a fifth threshold, a sixth threshold, a seventh threshold, an eighth threshold, a ninth threshold, a tenth threshold, an eleventh threshold, a twelfth threshold, a thirteenth threshold, or a fourteenth threshold.

[0063] In some embodiments, the second representation may be reconstructed based on the at least one sample at 1506. Further, a transformation may be performed based on the reconstructed second representation.

[0064] In some embodiments, the at least one sample may include all samples of the second representation, in which case values ​​of the at least one sample may be determined to reconstruct the second representation.

[0065] In some embodiments, the second representation may include at least one sample and a first sample set different from the at least one sample. A value of the at least one sample may be determined. The value of the first sample set may be obtained by performing an entropy decoding process on the bitstream based on the statistical information. The second representation may be reconstructed based on the value of the first sample set and the determined value of the at least one sample.

[0066] In some embodiments, to determine the value of the at least one sample, the value of each sample of the at least one sample may be determined to be a second predetermined value or a third statistical value corresponding to the sample. In one example, the second predetermined value may be a constant scalar number, such as zero. For example, the third statistical value may be an average.

[0067] In some embodiments, the first representation may be a latent representation of the visual data. At 1506, the visual data may be reconstructed by performing a compositing transform on the reconstructed second representation.

[0068] In some embodiments, the first representation may be a residual latent representation of the visual data. At 1506, a quantized latent representation of the visual data may be generated based on the reconstructed second representation and a fourth statistical value included in the statistical information. The visual data may be reconstructed by performing a synthesis transform on the quantized latent representation.

[0069] In some embodiments, the at least one sample may include all samples of the second representation. At 1506, a bitstream may be generated by performing an entropy coding process on the statistical information.

[0070] In some embodiments, the second representation may include at least one sample and a first set of samples different from the at least one sample. At 1506, the first portion of the bitstream may be generated by performing an entropy coding process on the statistical information. The second portion of the bitstream may be generated by performing an entropy coding process on the first set of samples based on the statistical information.

[0071] In some embodiments, the first representation may be a latent representation of the visual data. The method may further include generating the latent representation by performing an analytical transformation on the visual data using the first neural network, and obtaining the second representation by quantizing the latent representation.

[0072] In some embodiments, the first representation may be a residual latent representation of the visual data. The method may further include generating a latent representation of the visual data by performing an analytical transformation on the visual data using a first neural network, generating a residual latent representation using the first neural network based on the latent representation and a fourth statistical value included in the statistical information, and obtaining the second representation by quantizing the residual latent representation. In some embodiments, the fourth statistical value may be an average.

[0073] In some embodiments, the statistical information may be generated using a second neural network. In one example, the second neural network may be an estimation model. The second neural network may include a neural network-based sub-network. An input of the second neural network may include a bitstream or a latent representation of visual data.

[0074] In some embodiments, the statistical information may include a first statistical value and a second statistical value. The second neural network may include a first sub-network for generating the first statistical value and a second sub-network for generating the second statistical value. The first statistical value may be a variance and the second statistical value may be a mean.

[0075] In some embodiments, the first statistic may be the variance of a Gaussian probability distribution and the second statistic may be the mean of the Gaussian probability distribution. Additionally or alternatively, the first sub-network may be a hyperscale decoder sub-network and the second sub-network may be a hyperdecoder sub-network.

[0076] In some embodiments, at least one of the information regarding whether to apply the method or how to apply the method may be indicated in the bitstream, hi some embodiments, at least one of the information regarding whether to apply the method or how to apply the method may depend on the color format and / or color components of the visual data.

[0077] In some embodiments, the values ​​included in the bitstream may be coded at one of a sequence level, a picture level, a slice level, or a block level. In some embodiments, the values ​​included in the bitstream may be binarized before being coded. In some embodiments, the values ​​included in the bitstream may be coded in at least one arithmetic coding context.

[0078] According to a further embodiment of the present disclosure, a non-transitory computer-readable recording medium is provided, which stores a bitstream of visual data generated by a method performed by an apparatus for visual data processing, the method including the steps of obtaining statistical information associated with a first representation of the visual data, the first representation being generated based on applying a first neural network to the visual data, determining at least one sample from a second representation of the visual data based on the statistical information, a value of the at least one sample being absent from the bitstream, the second representation being obtained by quantizing the first representation, and generating the bitstream based on the determination.

[0079] According to yet another embodiment of the present disclosure, there is provided a method for storing a bitstream of visual data, the method including obtaining statistical information associated with a first representation of the visual data for a conversion between the visual data and the bitstream of visual data, the first representation being generated based on applying a first neural network to the visual data, determining at least one sample from a second representation of the visual data based on the statistical information, a value of the at least one sample being absent in the bitstream, the second representation being obtained by quantizing the first representation, generating the bitstream based on the determination, and storing the bitstream on a non-transitory computer-readable recording medium.

[0080] Implementations of the present disclosure can be described in terms of the following clauses, the features of which can be combined in any reasonable manner.

[0081] Clause 1. A method for visual data processing, comprising: obtaining statistical information associated with a first representation of the visual data, the first representation being generated based on applying a first neural network to the visual data; determining at least one sample from a second representation of the visual data based on the statistical information, the value of the at least one sample being absent in the bitstream, the second representation being obtained by quantizing the first representation; and performing the conversion based on the determination.

[0082] Clause 2. The method of clause 1, wherein the first representation comprises a latent representation of the visual data or a residual latent presentation of the visual data.

[0083] Clause 3. The method of clause 1 or 2, wherein determining the at least one sample from the second representation includes determining that the at least one sample comprises a sample of the second representation in accordance with a determination that at least one of the following conditions is satisfied: a first statistical value corresponding to the sample is less than a first threshold, a value determined based on the first statistical value is less than a second threshold, or a probability that the value of the sample is equal to a target value is less than a third threshold.

[0084] Clause 4. The method of clause 1 or 2, wherein determining the at least one sample from the second representation includes determining that the at least one sample comprises a sample in a block of the second representation in accordance with a determination that at least one of the following conditions is met: a first statistical value corresponding to each sample in the block is less than a fourth threshold; a first metric determined based on the first statistical value corresponding to the samples in the block is less than a fifth threshold; a probability that a value of each sample in the block is equal to a target value is less than a sixth threshold; or a second metric determined based on the probability that a value of the samples in the block is equal to a respective target value is less than a seventh threshold.

[0085] Clause 5. The method of clause 1 or 2, wherein determining the at least one sample from the second representation includes determining that the at least one sample comprises a sample of the second representation in accordance with a determination that at least one of the following conditions is satisfied: a first statistical value corresponding to the sample is greater than an eighth threshold, a value determined based on the first statistical value is greater than a ninth threshold, or a probability that the value of the sample is equal to a target value is greater than a tenth threshold.

[0086] Clause 6. The method of clause 1 or 2, wherein determining the at least one sample from the second representation includes determining that the at least one sample comprises a sample in a block of the second representation in accordance with a determination that at least one of the following conditions is met: a first statistical value corresponding to each sample in the block is greater than an eleventh threshold; a first metric determined based on the first statistical value corresponding to the samples in the block is greater than a twelfth threshold; a probability that a value of each sample in the block is equal to a target value is greater than a thirteenth threshold; or a second metric determined based on the probability that a value of the samples in the block is equal to a respective target value is greater than a fourteenth threshold.

[0087] Clause 7. The method of clause 4 or clause 6, wherein the first metric is the average, minimum or maximum of the first statistical value, or the second metric is the average, minimum or maximum of the probability.

[0088] Clause 8. The method according to any one of clauses 4 or 7 to 8, wherein the size of the block is N×M, and each of N and M is a positive integer.

[0089] Item 9. The method of item 8, wherein N and M are indicated in the bitstream.

[0090] Clause 10. The method according to any one of clauses 3 to 9, wherein the first statistical value is a variance.

[0091] Clause 11. The method according to any one of clauses 3 to 10, wherein the target value for a sample is one of a first predetermined value or a second statistical value corresponding to the sample.

[0092] Clause 12. The method of clause 11, wherein the first predetermined value is zero or the second statistical value is an average.

[0093] Clause 13. The method according to any one of clauses 3 to 12, wherein the probability is determined based on the statistical information.

[0094] Clause 14. The method according to any one of clauses 3 to 13, wherein at least one of the first threshold, the second threshold, the third threshold, the fourth threshold, the fifth threshold, the sixth threshold, the seventh threshold, the eighth threshold, the ninth threshold, the tenth threshold, the eleventh threshold, the twelfth threshold, the thirteenth threshold, or the fourteenth threshold is indicated in the bitstream.

[0095] Clause 15. The method of any one of clauses 1 to 14, wherein performing the transformation includes reconstructing the second representation based on the at least one sample, and performing the transformation based on the reconstructed second representation.

[0096] Clause 16. The method of clause 15, wherein the at least one sample includes all samples of the second representation, and reconstructing the second representation includes determining values ​​of the at least one sample to reconstruct the second representation.

[0097] Clause 17. The method of clause 15, wherein the second representation includes the at least one sample and a first sample set different from the at least one sample, and reconstructing the second representation includes determining a value of the at least one sample, obtaining values ​​of the first sample set by performing an entropy decoding process on the bitstream based on the statistical information, and reconstructing the second representation based on the value of the first sample set and the determined value of the at least one sample.

[0098] Clause 18. The method of clause 16 or clause 17, wherein determining the value of the at least one sample determines the value of each sample of the at least one sample to a second predetermined value or a third statistical value corresponding to the sample.

[0099] Item 19. The method of item 18, wherein the second predetermined value is a constant scalar number.

[0100] Clause 20. The method of clause 18, wherein the second predetermined value is zero.

[0101] Clause 21. The method according to any one of clauses 18 to 20, wherein the third statistical value is an average.

[0102] Clause 22. The method of any one of clauses 15 to 21, wherein the first representation is a latent representation of the visual data, and performing the transformation based on the reconstructed second representation includes reconstructing the visual data by performing a synthesis transformation on the reconstructed second representation.

[0103] Clause 23. The method of any one of clauses 15 to 21, wherein the first representation is a residual latent representation of the visual data, and performing the transformation includes generating a quantized latent representation of the visual data based on the reconstructed second representation and a fourth statistical value included in the statistical information, and reconstructing the visual data by performing a synthesis transformation on the quantized latent representation.

[0104] Clause 24. A method according to any one of clauses 1 to 14, wherein the at least one sample comprises all samples of the second representation, and performing the conversion comprises generating the bitstream by performing an entropy coding process on the statistical information.

[0105] Clause 25. The method of any one of clauses 1 to 14, wherein the second representation includes the at least one sample and a first set of samples different from the at least one sample, and performing the conversion includes generating a first portion of the bitstream by performing an entropy coding process on the statistical information, and generating a second portion of the bitstream by performing an entropy coding process on the first set of samples based on the statistical information.

[0106] Clause 26. The method of clause 24 or clause 25, wherein the first representation is a latent representation of the visual data, the method further comprising: generating the latent representation by performing an analytical transformation on the visual data using the first neural network; and obtaining the second representation by quantizing the latent representation.

[0107] Clause 27. The method of clause 24 or clause 25, wherein the first representation is a residual latent representation of the visual data, the method further comprising: generating a latent representation of the visual data by performing an analytical transformation on the visual data using the first neural network; generating the residual latent representation based on the latent representation and a fourth statistical value included in the statistical information; and obtaining the second representation by quantizing the residual latent representation.

[0108] Item 28. The method of item 23 or 28, wherein the fourth statistical value is an average.

[0109] Clause 29. The method of any one of clauses 1 to 28, wherein the statistical information is generated using a second neural network.

[0110] Clause 30. The method of clause 29, wherein the second neural network is an estimation model.

[0111] Clause 31. The method of clause 29 or clause 30, wherein the second neural network comprises a neural network-based sub-network.

[0112] Clause 32. The method of any one of clauses 29 to 31, wherein an input of the second neural network includes the bitstream or a latent representation of the visual data.

[0113] Clause 33. The method of clause 29 or clause 30, wherein the statistical information includes a first statistical value and a second statistical value, and the second neural network includes a first sub-network for generating the first statistical value and a second sub-network for generating the second statistical value.

[0114] Clause 34. The method of clause 33, wherein the first statistical value is a variance and the second statistical value is a mean.

[0115] Clause 35. The method of clause 33, wherein the first statistical value is the variance of a Gaussian probability distribution and the second statistical value is the mean of the Gaussian probability distribution.

[0116] Clause 36. The method of any one of clauses 33 to 35, wherein the first sub-network is a hyperscale decoder sub-network and the second sub-network is a hyperscale decoder sub-network.

[0117] Clause 37. A method according to any one of clauses 1 to 36, wherein at least one of information regarding whether to apply the method or information regarding how to apply the method is indicated in the bitstream.

[0118] Clause 38. A method according to any one of clauses 1 to 36, wherein at least one of the information regarding whether to apply the method or the information regarding how to apply the method depends on the color format and / or color components of the visual data.

[0119] Clause 39. A method according to any one of clauses 1 to 38, wherein the values ​​included in the bitstream are coded at one of a sequence level, a picture level, a slice level, or a block level.

[0120] Clause 40. A method according to any one of clauses 1 to 39, wherein the values ​​contained in the bitstream are binarised before being encoded.

[0121] Clause 41. A method according to any one of clauses 1 to 40, wherein values ​​included in the bitstream are coded with at least one arithmetic coding context.

[0122] Clause 42. The method of any one of clauses 1 to 41, wherein the visual data includes an image or video picture.

[0123] Clause 43. The method of any one of clauses 1 to 42, wherein the transforming includes encoding the visual data into the bitstream.

[0124] Clause 44. The method of any one of clauses 1 to 42, wherein the converting includes decoding the visual data from the bitstream.

[0125] Clause 45. An apparatus for processing visual data comprising a processor and a non-transitory memory having instructions thereon, the instructions, when executed by the processor, causing the processor to perform a method according to any one of clauses 1 to 44.

[0126] Clause 46. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method of any one of clauses 1 to 44.

[0127] Clause 47. A non-transitory computer-readable recording medium storing a bitstream of visual data generated by a method performed by an apparatus for visual data processing, the method including: obtaining statistical information associated with a first representation of the visual data, the first representation being generated based on applying a first neural network to the visual data, for conversion between visual data and the bitstream of the visual data; determining at least one sample from a second representation of the visual data based on the statistical information, the value of the at least one sample being absent in the bitstream, the second representation being obtained by quantizing the first representation; and generating the bitstream based on the determination.

[0128] Clause 48. A method for storing a bitstream of visual data, comprising: obtaining statistical information associated with a first representation of the visual data, the first representation being generated based on applying a first neural network to the visual data, determining at least one sample from a second representation of the visual data based on the statistical information, the value of the at least one sample being absent from the bitstream, the second representation being obtained by quantizing the first representation, generating the bitstream based on the determination, and storing the bitstream on a non-transitory computer-readable recording medium.

[0129] Exemplary Devices 16 illustrates a block diagram of a computing device 1600 in which various embodiments of the present disclosure may be implemented. The computing device 1600 may be implemented as or included in the source device 110 (or the visual data encoder 114) or the destination device 120 (or the visual data decoder 124).

[0130] It should be understood that the computing device 1600 shown in FIG. 16 is for illustrative purposes only and is not intended to suggest any limitation on the functionality and scope of the embodiments of the present disclosure.

[0131] 16, the computing device 1600 includes a general-purpose computing device 1600. The computing device 1600 may include at least one or more processors or processing units 1610, a memory 1620, a storage unit 1630, one or more communication units 1640, one or more input devices 1650, and one or more output devices 1660.

[0132] In some embodiments, the computing device 1600 may be implemented as any user terminal or server terminal having computing capabilities. A server terminal may be a server provided by a service provider, a large-scale computing device, etc. A user terminal may be, for example, any type of mobile, fixed, or portable terminal, including a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a gaming device, or any combination thereof, and further including accessories and peripherals of these devices, or any combination thereof. It is contemplated that the computing device 1600 may support any type of interface to a user, such as "wearable" circuitry.

[0133] The processing unit 1610 may be a physical or virtual processor and may perform various processes based on programs stored in the memory 1620. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of the computing device 1600. The processing unit 1610 may also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0134] Computing device 1600 typically includes a variety of computer storage media. Such media may be any media accessible by computing device 1600, including but not limited to volatile and nonvolatile media, or removable and non-removable media. Memory 1620 may be volatile memory (e.g., registers, cache, Random Access Memory (RAM)), non-volatile memory (Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), or flash memory, etc.), or any combination thereof. Storage unit 1630 may be any removable or non-removable media that may be used to store information and / or data and may include a machine-readable medium such as a memory, a flash memory drive, a magnetic disk, or other media that is accessible within computing device 1600.

[0135] Computing device 1600 may further include other removable / non-removable, volatile / non-volatile memory media. Although not shown in FIG. 16, it is possible to provide a magnetic disk drive for reading from and / or writing to a removable, non-volatile magnetic disk, and an optical disk drive for reading from and / or writing to a removable, non-volatile optical disk. In such a case, each drive may be connected to a bus (not shown) via one or more data media interfaces.

[0136] The communication unit 1640 communicates with further computing devices via a communication medium. Furthermore, the functionality of the components in the computing device 1600 may be implemented by a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the computing device 1600 may operate in a networked environment using logical connections with one or more other servers, networked personal computers (PCs), or more general network nodes.

[0137] The input device(s) 1650 may be one or more of a variety of input devices such as a mouse, a keyboard, a tracking ball, a voice input device, etc. The output device(s) 1660 may be one or more of a variety of output devices such as a display, a speaker, a printer, etc. The communication unit 1640 allows the computing device 1600 to further communicate with one or more external devices (not shown), such as a storage device and a display device, with one or more devices that allow a user to interact with the computing device 1600, or with any device (such as a network card, a modem, etc.) that allows the computing device 1600 to communicate with one or more other computing devices, as needed. Such communication may occur via an input / output (I / O) interface (not shown).

[0138] In some embodiments, instead of being integrated into a single device, some or all of the components of computing device 1600 may be located in a cloud computing architecture. In a cloud computing architecture, the components may be provided remotely and may cooperate to implement the functionality described in this disclosure. In some embodiments, cloud computing provides computing, software, data access, and storage services without the end user needing to be aware of the physical location or configuration of the systems or hardware providing these services. In various embodiments, cloud computing provides services over a wide area network (such as the Internet) using appropriate protocols. For example, a cloud computing provider provides applications accessible through a web browser or other computing components over the wide area network. Software or components of the cloud computing architecture and corresponding data may be stored on servers in remote locations. Computing resources of a cloud computing environment may be consolidated or distributed across locations in remote data centers. The cloud computing infrastructure may act as a single access point for users, but provide services through a shared data center. Thus, a cloud computing architecture may be used to provide the components and functionality described herein from a remote service provider. Alternatively, it may be provided from a conventional server or installed directly or in other manner on the client device.

[0139] The computing device 1600 may be used to perform visual data encoding / decoding in embodiments of the present disclosure. The memory 1620 may include one or more visual data encoding modules 1625 having one or more program instructions. These modules are accessible and executable by the processing unit 1610 to perform the functions of the various embodiments described herein.

[0140] In an example embodiment performing visual data encoding, the input device 1650 may receive visual data as input to be encoded 1670. The visual data may be processed, for example, by the visual data encoding module 1625 to generate an encoded bitstream. The encoded bitstream may be provided as output 1680 via the output device 1660.

[0141] In an example embodiment performing visual data decoding, input device 1650 may receive an encoded bitstream as input 1670. The encoded bitstream may be processed, for example, by visual data encoding module 1625 to generate decoded visual data. The decoded visual data may be provided as output 1680 via output device 1660.

[0142] Although the present disclosure has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application as defined by the appended claims. Such variations are intended to be covered by the scope of the present application. Thus, the foregoing description of the embodiments of the present application is not intended to be limiting.

Claims

1. obtaining statistical information associated with a first representation of the visual data for conversion between visual data and a bitstream of the visual data, the first representation being generated based on applying a first neural network to the visual data; determining at least one sample from a second representation of the visual data based on the statistical information, a value of the at least one sample being absent from the bitstream, the second representation being obtained by quantizing the first representation; performing said conversion based on said determination; and 16. A method for visual data processing, comprising:

2. The method of claim 1 , wherein the first representation comprises a latent representation of the visual data or a residual latent presentation of the visual data.

3. Determining the at least one sample from the second representation comprises: determining that the at least one sample comprises a sample of the second representation in accordance with a determination that at least one of the following conditions is satisfied, the conditions including: a first statistical value corresponding to the sample is less than a first threshold; the value determined based on the first statistical value is less than a second threshold value; or 3. The method of claim 1, wherein the probability that the value of the sample is equal to the target value is less than a third threshold.

4. Determining the at least one sample from the second representation comprises: determining that the at least one sample comprises a sample in a block of the second representation in accordance with a determination that at least one of the following conditions is satisfied, the conditions including: a first statistical value corresponding to each sample in the block is less than a fourth threshold; a first metric determined based on a first statistical value corresponding to samples in the block is less than a fifth threshold; the probability that the value of each sample in the block is equal to the target value is less than a sixth threshold; or 3. The method of claim 1, wherein a second metric, determined based on the probability that the values ​​of the samples in the block are equal to the respective target values, is less than a seventh threshold.

5. Determining the at least one sample from the second representation comprises: determining that the at least one sample comprises a sample of the second representation in accordance with a determination that at least one of the following conditions is satisfied, the conditions including: a first statistical value corresponding to the sample is greater than an eighth threshold; the value determined based on the first statistical value is greater than a ninth threshold value; or The method of claim 1 or 2, wherein the probability that the value of the sample is equal to the target value is greater than a tenth threshold.

6. Determining the at least one sample from the second representation comprises: determining that the at least one sample comprises a sample in a block of the second representation in accordance with a determination that at least one of the following conditions is satisfied, the conditions including: a first statistical value corresponding to each sample in the block is greater than an eleventh threshold; a first metric determined based on a first statistical value corresponding to samples in the block is greater than a twelfth threshold; the probability that the value of each sample in the block is equal to the target value is greater than a thirteenth threshold; or 3. The method of claim 1, wherein a second metric, determined based on the probability that the values ​​of the samples in the block are equal to the respective target values, is greater than a fourteenth threshold.

7. the first metric is an average, minimum, or maximum value of the first statistic; or The method of claim 4 or 6, wherein the second metric is a mean, a minimum or a maximum of the probability.

8. The method according to any one of claims 4 and 7 to 8, wherein the size of the block is NxM, where each of N and M is a positive integer.

9. The method of claim 8 , wherein N and M are indicated in the bitstream.

10. The method of any one of claims 3 to 9, wherein the first statistic is a variance.

11. The method of any one of claims 3 to 10, wherein the target value for a sample is one of a first predetermined value or a second statistical value corresponding to said sample.

12. The method of claim 11 , wherein the first predetermined value is zero or the second statistical value is an average.

13. The method according to any one of claims 3 to 12, wherein the probability is determined based on the statistical information.

14. 14. The method of claim 3, wherein at least one of the first threshold, the second threshold, the third threshold, the fourth threshold, the fifth threshold, the sixth threshold, the seventh threshold, the eighth threshold, the ninth threshold, the tenth threshold, the eleventh threshold, the twelfth threshold, the thirteenth threshold, or the fourteenth threshold is indicated in the bitstream.

15. Performing the conversion includes: reconstructing the second representation based on the at least one sample; and performing the transformation based on the reconstructed second representation; and The method according to any one of claims 1 to 14, comprising:

16. The at least one sample includes all samples of the second representation, and reconstructing the second representation includes: The method of claim 15 , further comprising determining a value of the at least one sample to reconstruct the second representation.

17. The second representation includes the at least one sample and a first set of samples different from the at least one sample, and reconstructing the second representation includes: determining a value of the at least one sample; performing an entropy decoding process on the bitstream based on the statistical information to obtain the first sample set of values; reconstructing the second representation based on the values ​​of the first sample set and the determined value of the at least one sample; 16. The method of claim 15, comprising:

18. Determining the value of the at least one sample includes:

18. The method according to claim 16 or 17, further comprising determining a value for each sample of the at least one sample to a second predetermined value or a third statistical value corresponding to the sample.

19. 20. The method of claim 18, wherein the second predetermined value is a constant scalar number.

20. 20. The method of claim 18, wherein the second predetermined value is zero.

21. The method of any one of claims 18 to 20, wherein the third statistical value is an average.

22. The first representation is a latent representation of the visual data, and performing the transformation based on the reconstructed second representation includes: The method of any one of claims 15 to 21, comprising reconstructing the visual data by performing a synthesis transform on the reconstructed second representation.

23. The first representation is a residual latent representation of the visual data, and performing the transformation includes: generating a quantized latent representation of the visual data based on the reconstructed second representation and a fourth statistical value included in the statistical information; and reconstructing the visual data by performing a synthesis transform on the quantized latent representation; The method according to any one of claims 15 to 21, comprising:

24. The at least one sample includes all samples of the second representation, and performing the transform includes: The method of any one of claims 1 to 14, comprising generating the bitstream by performing an entropy coding operation on the statistical information.

25. The second representation includes the at least one sample and a first set of samples different from the at least one sample, and performing the transformation includes: generating a first portion of the bitstream by performing an entropy coding process on the statistical information; generating a second portion of the bitstream by performing an entropy coding process on the first sample set based on the statistical information; and The method according to any one of claims 1 to 14, comprising:

26. the first representation is a latent representation of the visual data; The method comprises: generating the latent representation by performing an analytical transformation on the visual data using the first neural network; quantizing the latent representation to obtain the second representation; and 26. The method of claim 24 or 25, further comprising:

27. the first representation is a residual latent representation of the visual data; The method comprises: generating a latent representation of the visual data by performing an analytical transformation on the visual data using the first neural network; generating the residual latent representation based on the latent representation and a fourth statistical value included in the statistical information; quantizing the residual latent representation to obtain the second representation; 26. The method of claim 24 or 25, further comprising:

28. 30. The method of claim 23 or 28, wherein the fourth statistic is an average.

29. The method of any one of claims 1 to 28, wherein the statistical information is generated using a second neural network.

30. 30. The method of claim 29, wherein the second neural network is an estimation model.

31. 31. The method of claim 29 or 30, wherein the second neural network comprises a neural network-based sub-network.

32. A method according to any one of claims 29 to 31, wherein an input of the second neural network comprises a latent representation of the bitstream or the visual data.

33. 31. The method of claim 29 or 30, wherein the statistical information comprises a first statistical value and a second statistical value, and the second neural network comprises a first sub-network for generating the first statistical value and a second sub-network for generating the second statistical value.

34. 34. The method of claim 33, wherein the first statistical value is a variance and the second statistical value is a mean.

35. 34. The method of claim 33, wherein the first statistical value is the variance of a Gaussian probability distribution and the second statistical value is the mean of the Gaussian probability distribution.

36. The method of any one of claims 33 to 35, wherein the first sub-network is a hyperscale decoder sub-network and the second sub-network is a hyperdecoder sub-network.

37. A method according to any preceding claim, wherein at least one of information regarding whether to apply the method or information regarding how to apply the method is indicated in the bitstream.

38. 37. The method of any one of claims 1 to 36, wherein at least one of the information regarding whether to apply the method or information regarding how to apply the method depends on the colour format and / or colour components of the visual data.

39. The method according to any one of the preceding claims, wherein values ​​included in the bitstream are coded at one of a sequence level, a picture level, a slice level or a block level.

40. A method according to any one of claims 1 to 39, wherein values ​​contained in the bitstream are binarised before being encoded.

41. The method according to any one of claims 1 to 40, wherein values ​​comprised in the bitstream are coded with at least one arithmetic coding context.

42. A method according to any preceding claim, wherein the visual data comprises an image or a video picture.

43. A method according to any preceding claim, wherein the transforming comprises encoding the visual data into the bitstream.

44. A method according to any preceding claim, wherein the converting comprises decoding the visual data from the bitstream.

45. An apparatus for visual data processing comprising a processor and a non-transitory memory having instructions thereon, the instructions, when executed by the processor, causing the processor to perform a method according to any one of claims 1 to 44.

46. A non-transitory computer readable storage medium storing instructions that cause a processor to perform the method of any one of claims 1 to 44.

47. 1. A non-transitory computer readable recording medium storing a bitstream of visual data generated by a method performed by an apparatus for visual data processing, the method comprising: obtaining statistical information associated with a first representation of the visual data for conversion between visual data and a bitstream of the visual data, the first representation being generated based on applying a first neural network to the visual data; determining at least one sample from a second representation of the visual data based on the statistical information, a value of the at least one sample being absent from the bitstream, the second representation being obtained by quantizing the first representation; generating the bitstream based on the determination; and A non-transitory computer-readable recording medium comprising:

48. 1. A method for storing a bitstream of visual data, comprising the steps of: obtaining statistical information associated with a first representation of the visual data for conversion between visual data and a bitstream of the visual data, the first representation being generated based on applying a first neural network to the visual data; determining at least one sample from a second representation of the visual data based on the statistical information, a value of the at least one sample being absent from the bitstream, the second representation being obtained by quantizing the first representation; generating the bitstream based on said determination; storing the bitstream on a non-transitory computer readable recording medium; 16. A method for storing a bitstream of visual data, comprising:

Images