Encoding method and apparatus, decoding method and apparatus, device, storage medium, and computer program product

Abstract

The present invention provides encoding methods and apparatuses, decoding methods and apparatuses, devices, storage media, and computer program products. [Solution] When probability estimation is performed in the encoding method, the probability distribution of the unquantized image features is estimated based on the hyperplier features of the unquantized image features via a first probability distribution estimation network, and then the probability distribution of the quantized image features is obtained through quantization. Alternatively, the probability distribution of the quantized image features is directly estimated based on the hyperplier features of the unquantized image features via a second probability distribution estimation network (a network obtained by performing a simple operation on the network parameters of the first probability distribution estimation network).

Description

[Technical Field]

[0001] This application claims priority to Chinese Patent Application No. 202210234190.8, filed on 10 March 2022, entitled “Encoding Method and Apparatus, Decoding Method and Apparatus, Device, Storage Medium, and Computer Program Product,” which is incorporated herein by reference in its entirety. Technical field This application relates to the field of encoding and decoding technology, and more particularly to encoding methods and apparatus, decoding methods and apparatus, devices, storage media, and computer program products. [Background technology]

[0002] Image compression technology enables the effective transmission and storage of image information and plays a crucial role in the media age where the types and amounts of image information are increasing. Image compression technology includes image encoding and decoding. Encoding and decoding performance reflects image quality and is a factor that must be considered in image compression technology.

[0003] In the encoding process of related technologies, image features y are extracted from an image via an image feature extraction network, and image features y are quantized based on a quantization step q to obtain image features ys. Image features ys are input to a hyperencoder network to determine hyperplier features zs, and hyperplier features zs are encoded into a bitstream through entropy encoding. Entropy decoding is performed on the hyperplier features zs in the bitstream to obtain hyperplier features zs', and based on hyperplier features zs', the probability distribution parameters of image features ys are obtained via a probability distribution estimation network. Based on the probability distribution parameters of image features ys, image features ys are encoded into a bitstream through entropy encoding. The decoding process is related to the encoding process. Most image compression is implemented by using quantization operations, and quantization operations have a significant impact on encoding and decoding performance.

[0004] However, quantization operations in the encoding and decoding processes must match the bitrate. In multi-bitrate scenarios, different quantization steps are typically required in the encoding and decoding processes to match different bitrates. However, different quantization steps result in significant differences in the numerical range of image features ys obtained through quantization. To obtain a probability distribution estimation network for estimating the probability distribution parameters of image features ys at different bitrates, the probability distribution estimation network must be trained using image features ys with different numerical ranges. However, the numerical range of image features ys changes significantly at different bitrates, making it difficult to train the probability distribution estimation network, resulting in unstable network training and difficulty in obtaining a probability distribution estimation network with good performance through training. Consequently, encoding and decoding performance is affected. [Overview of the project]

Problems to be Solved by the Invention

[0005] Embodiments of the present application provide an encoding method and apparatus, a decoding method and apparatus, a device, a storage medium, and a computer program product. In a multi-bitrate scenario, the training of a probability distribution estimation network can be simplified, as a result, the network training becomes stable, a network with good performance is trained, and thereby the encoding and decoding performance is improved.

Means for Solving the Problems

[0006] According to a first aspect, the present application provides an encoding method, and the method includes: determining a first image feature and a second image feature of an image to be encoded, wherein the first image feature is an image feature obtained by quantizing the second image feature based on a first quantization step; determining a first hyperprior feature of the second image feature; encoding the first hyperprior feature into a bitstream; determining a first probability distribution parameter based on the first hyperprior feature via a probability distribution estimation network; quantizing the first probability distribution parameter based on the first quantization step to obtain a second probability distribution parameter; and encoding the first image feature into the bitstream based on the second probability distribution parameter.

[0007] When probability estimation is performed in the encoding process, the probability distribution of the unquantized image features [image features that have not been quantized] is estimated based on the hyperprior features of the unquantized image features via the probability distribution estimation network, and then the probability distribution of the quantized image features is obtained through quantization. It can be seen that even in a multi-bitrate scenario, the numerical range of the unquantized image features is not affected by the quantization step and is stable. Therefore, training the probability distribution estimation network by using the unquantized image features is not very difficult, is stable, and a network with good performance can be trained. This helps to improve the encoding and decoding performance.

[0008] Optionally, the step of determining the first hyperprior feature of the second image feature includes: inputting the second image feature into a hyperencoder network to obtain the first hyperprior feature. That is, on the encoder side, the unquantized image features are input into the hyperencoder network to obtain the first hyperprior feature of the unquantized image features.

[0009] Optionally, the step of determining the first hyperprior feature of the second image feature includes: inverse-quantizing the first image feature based on the first quantization step to obtain a third image feature of the image; and inputting the third image feature into a hyperencoder network to obtain the first hyperprior feature. That is, on the encoder side, the inverse-quantized image features are input into the hyperencoder network, and the obtained first hyperprior feature of the inverse-quantized image features is regarded as the first hyperprior feature of the unquantized image features.

[0010] Optionally, the step of determining a first probability distribution parameter based on a first hyperprior feature via a probability distribution estimation network includes: inputting a third image feature of an image into a context network to obtain a context feature of a third image feature, the third image feature being an image feature obtained by dequantizing the first image feature based on a first quantization step; determining a first plier feature based on a first hyperprior feature; and inputting the first plier feature and context feature into a probability distribution estimation network to obtain a first probability distribution parameter. That is, the encoder side extracts the context feature from the dequantized image feature and determines the first probability distribution parameter based on the context feature and the first plier feature. This helps to improve the accuracy of the probability estimation.

[0011] Optionally, the step of determining a first probability distribution parameter via a probability distribution estimation network based on the first hyperplier feature includes: inputting the first image feature into a context network to obtain a context feature of the first image feature; determining a first plier feature based on the first hyperplier feature; quantizing the first plier feature based on a second quantization step to obtain a second plier feature; and inputting the second plier feature and the context feature into the probability distribution estimation network to obtain the first probability distribution parameter. That is, the encoder side may alternatively extract a context feature from the inversely quantized image feature, then obtain a second plier feature by applying a quantization operation to the first plier feature, and determine the first probability distribution parameter based on the second plier feature and the context feature. This also improves the accuracy of the probability estimation to some extent.

[0012] Optionally, the first quantization step is acquired via a gain network based on the image bitrate, and the gain network is used to determine the quantization step corresponding to each of several types of bitrates. That is, the quantization step is acquired through network learning, and the quantization step can be better matched to the bitrate. This helps to improve encoding and decoding performance.

[0013] According to the second aspect, a decoding method is provided. This method is: The process includes: parsing a bitstream to obtain a first hyperplier feature of an image to be decoded; determining a first probability distribution parameter representing the probability distribution of the unquantized image features of the image based on the first hyperplier feature via a probability distribution estimation network; quantizing the first probability distribution parameter based on the first quantization step to obtain a second probability distribution parameter; parsing the bitstream based on the second probability distribution parameter to obtain a first image feature of the image; and inversely quantizing the first image feature based on the first quantization step to reconstruct the image.

[0014] When probability estimation is performed during the decoding process, the probability distribution of unquantized image features is estimated via a probability distribution estimation network based on the hyperprior features of the unquantized image features, and then the probability distribution of quantized image features is obtained through quantization. Even in multi-bitrate scenarios, the numerical range of unquantized image features remains stable and is not affected by the quantization step. Therefore, training a probability distribution estimation network using unquantized image features is not particularly difficult, and a stable network with good performance can be trained. This helps improve encoding and decoding performance.

[0015] Optionally, the first image feature is an image feature obtained by quantizing the second image feature of the image based on the first quantization step.

[0016] Optionally, the step of reconstructing an image by dequantizing a first image feature based on a first quantization step includes: obtaining a third image feature of the image by dequantizing a first image feature based on a first quantization step, and reconstructing the image based on the third image feature.

[0017] Optionally, the first probability distribution parameter includes the probability distribution parameters of a plurality of feature points, and the first hyperplier feature includes the hyperplier features of the plurality of feature points. The step of determining the first probability distribution parameter via a probability distribution estimation network based on the first hyperplier features includes: performing the following operations on the first feature point in order to determine the probability distribution parameter of the first feature point, where the first feature point is one of the plurality of feature points: determining the contextual features of the first feature point based on the image features of the decoded feature point in the first image feature; determining the first plier features of the first feature point based on the hyperplier features of the first feature point; and determining the probability distribution parameter of the first feature point via a probability distribution estimation network based on the first plier features and the contextual features of the first feature point, i.e., determining the probability distribution parameter of the first feature point in the first probability distribution parameter. That is, the probability distribution is estimated based on the contextual features. This helps to improve the accuracy of the probability estimation.

[0018] Optionally, the step of determining the context features of the first feature point based on the image features of the decoded feature point in the first image feature includes: determining the peripheral feature points of the first feature point from the decoded feature point; dequantizing the image features of the peripheral feature points in the first image feature based on the first quantization step to obtain the peripheral features of the first feature point; and inputting the peripheral features of the first feature point into a context network to obtain the context features of the first feature point. The step of determining the probability distribution parameters of the first feature point via a probability distribution estimation network based on the first plier features of the first feature point and the context features of the first feature point includes: inputting the first plier features of the first feature point and the context features of the first feature point into a probability distribution estimation network to obtain the probability distribution parameters of the first feature point. That is, the decoder side extracts context features from the dequantized image features and determines the first probability distribution parameters based on the context features and the first plier features.

[0019] Optionally, the step of determining the context features of a first feature point based on the image features of a decoded feature point in a first image feature includes: determining the peripheral feature points of the first feature point from the decoded feature points; and inputting the image features of the peripheral feature points in the first image feature into a context network to obtain the context features of the first feature point. The step of determining the probability distribution parameters of the first feature point via the probability distribution estimation network based on the first plier features and the context features of the first feature point includes: quantizing the first plier features of the first feature point based on a second quantization step to obtain the second plier features of the first feature point; and inputting the second plier features and the context features of the first feature point into the probability distribution estimation network to obtain the probability distribution parameters of the first feature point. In other words, the decoder extracts context features from the quantized image features, then obtains a second plier feature by applying a quantization operation to the first plier feature, and determines the first probability distribution parameter based on the second plier feature and the context features.

[0020] According to the third aspect, an encoding method is provided. This method is: The method includes: determining a first image feature and a second image feature of an image to be encoded, wherein the first image feature is an image feature obtained by quantizing the second image feature based on a first quantization step; determining a first hyperplier feature of the second image feature; encoding the first hyperplier feature into a bitstream; determining a second probability distribution parameter based on the first hyperplier feature via a second probability distribution estimation network, wherein the network parameter of the second probability distribution estimation network is obtained based on the network parameter of the first probability distribution estimation network and the first quantization step, and the first probability distribution estimation network is used to determine the probability distribution of the unquantized image feature; and encoding the first image feature into the bitstream based on the second probability distribution parameter.

[0021] During the encoding process, the hyperplier features of the unquantized image features are also determined, but the second probability distribution parameters are subsequently obtained directly via a second probability distribution estimation network. The second probability distribution estimation network is obtained by processing the network parameters in the first probability distribution estimation network based on the first quantization step. In this solution, it is found that only the first probability distribution estimation network needs to be trained. Even in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and unaffected by the quantization step. Therefore, training the first probability distribution estimation network is not very difficult, the network training is stable, and a first probability distribution estimation network with good performance can be trained. This helps to improve encoding and decoding performance.

[0022] Optionally, the first probability distribution estimation network is a probability distribution estimation network in the first aspect or the second aspect.

[0023] Optionally, a second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by a first quantization step.

[0024] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets.

[0025] Optionally, the step of determining the first hyperplier feature of a second image feature includes inputting the second image feature into a hyperencoder network to obtain the first hyperplier feature. That is, the encoder inputs the unquantized image feature into the hyperencoder network to obtain the first hyperplier feature of the unquantized image feature.

[0026] Optionally, the step of determining the first hyperplier feature of a second image feature includes: dequantizing the first image feature based on the first quantization step to obtain a third image feature of the image; and inputting the third image feature into a hyperencoder network to obtain the first hyperplier feature. That is, the encoder inputs the dequantized image feature into the hyperencoder network, and the obtained first hyperplier feature of the dequantized image feature is considered to be the first hyperplier feature of the unquantized image feature.

[0027] Optionally, the step of determining a second probability distribution parameter based on a first hyperplier feature via a second probability distribution estimation network includes: inputting a third image feature of an image into a context network to obtain a context feature of the third image feature, the third image feature being an image feature obtained by dequantizing the first image feature based on a first quantization step; determining a first plier feature based on the first hyperplier feature; and inputting the first plier feature and the context feature into a second probability distribution estimation network to obtain a second probability distribution parameter. That is, the encoder side extracts the context feature from the dequantized image feature and determines the first probability distribution parameter based on the context feature and the first plier feature. This helps to improve the accuracy of the probability estimation.

[0028] Optionally, the step of determining a second probability distribution parameter based on a first hyperplier feature via a second probability distribution estimation network includes: inputting a first image feature into a context network to obtain a context feature of the first image feature; determining a first plier feature based on the first hyperplier feature; obtaining a second plier feature by quantizing the first plier feature based on a second quantization step; and inputting the second plier feature and context feature into a second probability distribution estimation network to obtain a second probability distribution parameter. That is, the encoder side may alternatively extract context features from the inversely quantized image feature, then obtain a second plier feature by applying a quantization operation to the first plier feature, and determine the first probability distribution parameter based on the second plier feature and context feature. This, too, can improve the accuracy of the probability estimation to some extent.

[0029] According to the fourth aspect, a decoding method is provided. This method is: The process includes: parsing a bitstream to obtain a first hyperplier feature of an image to be decoded; determining a second probability distribution parameter based on the first hyperplier feature via a second probability distribution estimation network, wherein the network parameter of the second probability distribution estimation network is obtained based on the network parameter of the first probability distribution estimation network and a first quantization step, the first probability distribution estimation network being used to determine the probability distribution of unquantized image features; parsing the bitstream based on the second probability distribution parameter to obtain a first image feature of the image; and reconstructing the image by dequantizing the first image feature based on the first quantization step.

[0030] In the decoding process, the second probability distribution parameters are obtained directly through the second probability distribution estimation network, which in turn obtains them by processing the network parameters in the first probability distribution estimation network based on the first quantization step. In this solution, only the first probability distribution estimation network needs to be trained. Even in multi-bitrate scenarios, the numerical range of unquantized image features is stable and unaffected by the quantization step. In other words, the input numerical range of the first probability distribution estimation network does not change with the bitstream. Therefore, training the first probability distribution estimation network is not very difficult, the network training is stable, and a first probability distribution estimation network with good performance can be trained. This helps to improve encoding and decoding performance.

[0031] Optionally, the first probability distribution estimation network is a probability distribution estimation network in the first aspect or the second aspect.

[0032] Optionally, a second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by a first quantization step.

[0033] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets.

[0034] Optionally, the first image feature is an image feature obtained by quantizing the second image feature of the image based on the first quantization step.

[0035] Optionally, the step of reconstructing an image by dequantizing a first image feature based on a first quantization step includes: obtaining a third image feature of the image by dequantizing a first image feature based on a first quantization step; and reconstructing the image based on the third image feature.

[0036] Optionally, the second probability distribution parameter includes the probability distribution parameters of multiple feature points, and the first hyperplier feature includes the hyperplier features of multiple feature points. The step of determining the second probability distribution parameter based on the first hyperplier feature via the second probability distribution estimation network includes: performing the following operations on the first feature point to determine the probability distribution parameter of the first feature point, where the first feature point is one of multiple feature points: determining the contextual feature of the first feature point based on the image features of the decoded feature point in the first image feature; determining the first plier feature of the first feature point based on the hyperplier feature of the first feature point; and determining the probability distribution parameter of the first feature point based on the first plier feature and the contextual feature of the first feature point via the second probability distribution estimation network, i.e., determining the probability distribution parameter of the first feature point in the second probability distribution parameter. That is, the probability distribution is estimated based on the contextual feature. This helps to improve the accuracy of the probability estimation.

[0037] Optionally, the step of determining the context features of the first feature point based on the image features of the decoded feature point in the first image feature includes: determining the peripheral feature points of the first feature point from the decoded feature point; dequantizing the image features of the peripheral feature points in the first image feature based on the first quantization step to obtain the peripheral features of the first feature point; and inputting the peripheral features of the first feature point into a context network to obtain the context features of the first feature point. The step of determining the probability distribution parameters of the first feature point based on the first plier features and the context features of the first feature point via a second probability distribution estimation network includes: inputting the first plier features and the context features of the first feature point into a second probability distribution estimation network to obtain the probability distribution parameters of the first feature point. That is, the decoder side extracts context features from the dequantized image features and determines second probability distribution parameters based on the context features and the first plier features.

[0038] Optionally, the step of determining the context features of a first feature point based on the image features of a decoded feature point in a first image feature includes: determining the surrounding feature points of the first feature point from the decoded feature points; and inputting the image features of the surrounding feature points in the first image feature into a context network to obtain the context features of the first feature point. The step of determining the probability distribution parameters of a first feature point based on the first plier features and the context features of the first feature point via a second probability distribution estimation network includes: quantizing the first plier features of the first feature point based on a second quantization step to obtain the second plier features of the first feature point; and inputting the second plier features and the context features of the first feature point into a second probability distribution estimation network to obtain the probability distribution parameters of the first feature point. In other words, the decoder extracts context features from the quantized image features, then obtains a second plier feature by applying a quantization operation to the first plier feature, and determines a second probability distribution parameter based on the second plier feature and the context features.

[0039] According to the fifth aspect, an encoding device is provided. The encoding device has the function of realizing the behavior of the encoding method in the first aspect. The encoding device includes one or more modules, which are configured to implement the encoding method provided in the first aspect.

[0040] In other words, an encoding device is provided. The device is: A first determination module configured to determine a first image feature and a second image feature of an image to be encoded, wherein the first image feature is an image feature obtained by quantizing the second image feature based on a first quantization step; A second determination module configured to determine the first hyperplier feature of the second image feature; A first encoding module configured to encode the first hyperplier feature into a bitstream; A probability estimation module configured to determine a first probability distribution parameter based on the first hyperplier feature via a probability distribution estimation network; A quantization module configured to obtain a second probability distribution parameter by quantizing the first probability distribution parameter based on the first quantization step; A second encoding module configured to encode the first image features into the bitstream based on the second probability distribution parameter, and Includes.

[0041] Optionally, the second decision module is: The system includes a first hyperencoder submodule configured to input the second image feature into a hyperencoder network to acquire the first hyperplier feature.

[0042] Optionally, the second decision module is: An inverse quantization submodule configured to inversely quantize the first image feature based on the first quantization step to obtain a third image feature of the image; A second hyperencoder submodule configured to input the third image feature into a hyperencoder network to acquire the first hyperplier feature, and Includes.

[0043] Optionally, the probability estimation module is: A context submodule configured to input the aforementioned third image feature of the aforementioned image into a context network to obtain a context feature of the aforementioned third image feature, wherein the aforementioned third image feature is an image feature obtained by dequantizing the aforementioned first image feature based on the aforementioned first quantization step; A first determination submodule configured to determine a first plier feature based on the first hyperplier feature; A first probability estimation submodule is configured to input the first plier feature and the context feature into the probability distribution estimation network to obtain the first probability distribution parameter, and Includes.

[0044] Optionally, the probability estimation module is: A context submodule configured to input the aforementioned first image feature into a context network to obtain the context features of the aforementioned first image feature; A second determination submodule configured to determine a first plier feature based on the first hyperplier feature; A quantization submodule configured to obtain a second plier feature by quantizing the first plier feature based on a second quantization step; A second probability estimation submodule is configured to input the second pryor feature and the context feature into the probability distribution estimation network to obtain the first probability distribution parameter, and Includes.

[0045] According to the sixth aspect, a decoding device is provided. This decoding device has the function of implementing the behavior of the decoding method in the second aspect. The decoding device includes one or more modules, which are configured to implement the decoding method provided in the second aspect.

[0046] In other words, a decoding device is provided. This device is: A first parsing module configured to parse a bitstream and obtain a first hyperplier feature of the image to be decoded; A probability estimation module configured to determine a first probability distribution parameter based on the first hyperplier feature via a probability distribution estimation network, wherein the first probability distribution parameter represents the probability distribution of the unquantized image features of the image; A quantization module configured to obtain a second probability distribution parameter by quantizing the first probability distribution parameter based on a first quantization step; A second parsing module configured to parse the bitstream based on the second probability distribution parameter to obtain a first image feature of the image; A reconstruction module configured to reconstruct the image by dequantizing the first image features based on the first quantization step, and Includes.

[0047] Optionally, the first image feature is an image feature obtained by quantizing the second image feature of the image based on the first quantization step.

[0048] Optionally, the reconfiguration module is: An inverse quantization submodule configured to inversely quantize the first image feature based on the first quantization step to obtain a third image feature of the image; A reconstruction submodule configured to reconstruct the image based on the third image feature described above, Includes.

[0049] Optionally, the first probability distribution parameter includes the probability distribution parameters of multiple feature points, and the first hyperplier feature includes the hyperplier feature of multiple feature points. The probability estimation module includes a context submodule, a first decision submodule, and a probability estimation submodule.

[0050] For the first feature point, the probability distribution parameters of the first feature point are determined via the context submodule, the first decision submodule, and the probability estimation submodule, and the first feature point is one of several feature points.

[0051] The context submodule is configured to determine the context features of the first feature point based on the image features of the decoded feature point in the first image feature.

[0052] The first decision submodule is configured to determine the first plier feature of the first feature point based on the hyperplier feature of the first feature point.

[0053] The probability estimation submodule is configured to determine the probability distribution parameters of a first feature point based on the first pryor feature and the context features of the first feature point, via a probability distribution estimation network.

[0054] Optionally, the context submodule is: From the decoded feature points, the surrounding feature points of the first feature point are determined; Based on the first quantization step, the image features of the peripheral feature points in the first image feature are dequantized to obtain the peripheral features of the first feature point; The system is configured to input the surrounding features of the first feature point into a context network to obtain the context features of the first feature point. Determining the probability distribution parameters of the first feature point via the probability distribution estimation network, based on the first plier feature of the first feature point and the context feature of the first feature point, is as follows: This includes inputting the first plier feature and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameters of the first feature point.

[0055] Optionally, the context submodule is: From the decoded feature points, the surrounding feature points of the first feature point are determined; The system is configured to input the image features of the surrounding feature points in the first image feature into a context network to obtain the context features of the first feature point. Determining the probability distribution parameters of the first feature point via the probability distribution estimation network, based on the first plier feature of the first feature point and the context feature of the first feature point, is as follows: Based on the second quantization step, the first plier feature of the first feature point is quantized to obtain the second plier feature of the first feature point; This includes inputting the second plier feature of the first feature point and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameters of the first feature point.

[0056] According to the seventh aspect, an encoding device is provided. The encoding device has the function of implementing the encoding method in the third aspect. The encoding device includes one or more modules, which are configured to implement the encoding method provided in the third aspect.

[0057] In other words, an encoding device is provided. The device is: A first determination module configured to determine a first image feature and a second image feature of an image to be encoded, wherein the first image feature is an image feature obtained by quantizing the second image feature based on a first quantization step; A second determination module configured to determine the first hyperplier feature of the second image feature; A first encoding module configured to encode the first hyperplier feature into a bitstream; A probability estimation module configured to determine a second probability distribution parameter based on the first hyperplier feature via a second probability distribution estimation network, wherein the network parameter of the second probability distribution estimation network is obtained based on the network parameter of the first probability distribution estimation network and the first quantization step, and the first probability distribution estimation network is used to determine the probability distribution of the unquantized image feature; A second encoding module configured to encode the first image features into the bitstream based on the second probability distribution parameter, and Includes.

[0058] Optionally, a second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by a first quantization step.

[0059] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets.

[0060] Optionally, the second decision module is: The system includes a first hyperencoder submodule configured to input the second image feature into a hyperencoder network to acquire the first hyperplier feature.

[0061] Optionally, the second decision module is: An inverse quantization submodule configured to inversely quantize the first image feature based on the first quantization step to obtain a third image feature of the image; A second hyperencoder submodule configured to input the third image feature into a hyperencoder network to acquire the first hyperplier feature, and Includes.

[0062] According to the eighth aspect, a decoding device is provided. The decoding device has the function of implementing the operation of the decoding method in the fourth aspect. The decoding device includes one or more modules, which are configured to implement the decoding method provided in the fourth aspect.

[0063] In other words, a decoding device is provided. This device is: A first parsing module configured to parse a bitstream and obtain a first hyperplier feature of the image to be decoded; A probability estimation module configured to determine a second probability distribution parameter based on the first hyperplier feature via a second probability distribution estimation network, wherein the network parameter of the second probability distribution estimation network is obtained based on the network parameter of the first probability distribution estimation network and a first quantization step, and the first probability distribution estimation network is used to determine the probability distribution of the unquantized image feature; A second parsing module configured to parse the bitstream based on the second probability distribution parameter to obtain a first image feature of the image; A reconstruction module configured to reconstruct the image by dequantizing the first image features based on the first quantization step, and Includes.

[0064] Optionally, a second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by a first quantization step.

[0065] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets.

[0066] Optionally, the first image feature is an image feature obtained by quantizing the second image feature of the image based on the first quantization step.

[0067] Optionally, the reconfiguration module is: An inverse quantization submodule configured to inversely quantize the first image feature based on the first quantization step to obtain a third image feature of the image; A reconstruction submodule configured to reconstruct the image based on the third image feature described above, Includes.

[0068] According to the ninth aspect, an encoder-side device is provided. The encoder-side device includes a processor and memory. The memory is configured to store a program for performing the encoding method provided in the first and / or third aspects, and to store data for performing the encoding method provided in the first and / or third aspects. The processor is configured to execute the program stored in memory. The encoder-side device may further include a communication bus, which is configured to establish a connection between the processor and the memory.

[0069] According to the tenth aspect, a decoder-side device is provided. The decoder-side device includes a processor and memory. The memory is configured to store a program for performing the decoding method provided in the second and / or fourth aspect, and to store data for performing the decoding method provided in the second and / or fourth aspect. The processor is configured to execute the program stored in memory. The decoder-side device may further include a communication bus, which is configured to establish a connection between the processor and the memory.

[0070] According to the eleventh aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions. When the instructions are executed on a computer, the computer is made capable of performing an encoding method according to the first or third aspect, or a decoding method according to the second or fourth aspect.

[0071] According to the twelfth aspect, a computer program product containing instructions is provided. When the computer program product is executed on a computer, the computer is made capable of performing an encoding method according to the first or third aspect, or a decoding method according to the second or fourth aspect.

[0072] The technical effects obtained in Aspects 5 through 12 are similar to the technical effects obtained through the corresponding technical means in Aspects 1 through 4. Further details are not described herein.

[0073] The technical solution provided in this application can bring about at least the following beneficial effects.

[0074] To obtain the probability distribution parameters of the quantized image features, in the encoding process of the solution, a first probability distribution parameter is determined based on the hyperplier features of the unquantized image features via a probability distribution estimation network, and the first probability distribution parameter represents the probability distribution of the unquantized image features. Then, based on a first quantization step (i.e., a quantization step for quantizing the image features), the first probability distribution parameter is quantized to obtain a second probability distribution parameter that represents the probability distribution of the quantized image features. In the encoding process of another solution, the hyperplier features of the unquantized image features are also determined, but the second probability distribution parameter is then directly obtained via a second probability distribution estimation network. The second probability distribution estimation network is obtained by processing the network parameters in the first probability distribution estimation network based on the first quantization step, and the first probability distribution network parameters are the probability distribution estimation network in the first solution. The decoding process is related to the encoding process. In these two solutions, it is found that only the first probability distribution estimation network (for determining the probability distribution parameters of unquantized image features) needs to be trained. Even in multi-bitrate scenarios, the numerical range of unquantized image features is stable and unaffected by the quantization step. In other words, the input numerical range of the first probability distribution estimation network does not change with the bitrate. Therefore, training the first probability distribution estimation network is not very difficult, the network training is stable, and a first probability distribution estimation network with good performance can be trained. This helps to improve encoding and decoding performance. [Brief explanation of the drawing]

[0075] [Figure 1] This is a diagram of an implementation environment according to one embodiment of the present invention.

[0076] [Figure 2]This is a diagram of another implementation environment according to one embodiment of the present invention.

[0077] [Figure 3] This is a diagram of yet another implementation environment according to one embodiment of the present invention.

[0078] [Figure 4] This is a flowchart of an encoding method according to one embodiment of the present invention.

[0079] [Figure 5] This is a diagram showing the structure of an image feature extraction network according to one embodiment of the present invention.

[0080] [Figure 6] This is a flowchart of an encoding and decoding method according to one embodiment of the present invention.

[0081] [Figure 7] This is a flowchart of another encoding and decoding method according to one embodiment of the present invention.

[0082] [Figure 8] This is a flowchart of yet another encoding and decoding method according to one embodiment of the present invention.

[0083] [Figure 9] This is a flowchart of yet another encoding and decoding method according to one embodiment of the present invention.

[0084] [Figure 10] This is a flowchart of yet another encoding and decoding method according to one embodiment of the present invention.

[0085] [Figure 11] This is a flowchart of another encoding method according to one embodiment of the present invention.

[0086] [Figure 12] This is a flowchart of a decoding method according to one embodiment of the present invention.

[0087] [Figure 13] This is a flowchart of another decoding method according to one embodiment of the present invention.

[0088] [Figure 14] This is a structural diagram of an encoding device according to one embodiment of the present invention.

[0089] [Figure 15] This is a structural diagram of a decoding device according to one embodiment of the present invention.

[0090] [Figure 16] This is a structural diagram of another encoding device according to one embodiment of the present invention.

[0091] [Figure 17] This is a structural diagram of another decoding device according to one embodiment of the present invention.

[0092] [Figure 18] This is a block diagram of an encoding and decoding device according to one embodiment of the present invention. Embodiments of the present invention will be described below. [Modes for carrying out the invention]

[0093] To further clarify the purpose, technical solution, and advantages of this application, the implementation of this application will be described in more detail below with reference to the attached drawings.

[0094] The system architectures and service scenarios described in the embodiments of this application are intended to more clearly illustrate the technical solutions in the embodiments of this application and do not constitute a limitation on the technical solutions provided in the embodiments of this application. Those skilled in the art will recognize that, with the evolution of system architectures and the emergence of new service scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0095] Before describing in detail the encoding and decoding methods provided in the embodiments of this application, the terminology and implementation environment in the embodiments of this application will be described first.

[0096] To facilitate understanding, the terminology used in the embodiments of this application will be explained first.

[0097] Bitrate: In image compression, the bitrate is the code length required to encode pixels. A higher bitrate indicates better image reconstruction quality.

[0098] Convolutional neural networks (CNNs): Convolutional neural networks are feedforward neural networks with a deep structure that include convolutional operations, and are one of the representative algorithms of deep learning. CNNs include convolutional layers and may also include activation layers (e.g., rectified linear unit (ReLU) and parametric ReLU (PReLU)), pooling layers, batch normalization (BN) layers, fully connected layers, etc. Typical CNNs include, for example, LeNet, AlexNet, VGGNet, and ResNet. Basic CNNs may include a backbone network and a head network. Complex CNNs may include a backbone network, a neck network, and a head network.

[0099] Feature map: A feature map is three-dimensional data output by the convolutional, activation, pooling, and batch normalization layers in a convolutional neural network. The three dimensions are called width, height, and channel, respectively. A single feature map contains image features of multiple feature points.

[0100] Backbone network: A backbone network is the initial part of a convolutional neural network used to extract feature maps of multiple scales from an input image. Generally, a backbone network includes convolutional layers, pooling layers, activation layers, etc., but does not include fully connected layers. Generally, feature maps output by layers closer to the input image in a backbone network have high resolution (width and height) but a small number of channels. Typical backbone networks include, for example, VGG-16, ResNet-50, and ResNeXt-101.

[0101] Head network: The head network is the final component of a convolutional neural network and is used to process feature maps to obtain the prediction results output by the neural network. A typical head network includes fully connected layers, softmax modules, etc.

[0102] Neck network: The neck network is an intermediate part of a convolutional neural network, used to further integrate feature maps generated by the head network to obtain new feature maps. A common network is the feature pyramid network (FPN) in fast region convolutional neural networks (faster-RCNNs).

[0103] The implementation environment in the embodiments of this application will be described below.

[0104] Figure 1 is a diagram of an implementation environment according to one embodiment of the present application. Refer to Figure 1. This implementation environment includes an encoder side 101 and a decoder side 102. The encoder side 101 is used to compress an image according to an encoding method provided in an embodiment of the present application, and the decoder side 102 is used to decode an image according to a decoding method provided in an embodiment of the present application. Optionally, the encoder side 101 includes an encoder, which is configured to compress an image. The decoder side 102 includes a decoder, which is configured to decode an image. When the encoder side 101 and the decoder side 102 are located in the same device, the encoder side 101 communicates with the decoder side 102 via an internal device connection line or network. When the encoder side 101 and the decoder side 102 are located in different devices, the encoder side 101 communicates with the decoder side 102 via an external connection line or wireless network. The encoder side 101 is also called the source device, and the decoder side 102 is also called the destination device.

[0105] Figure 2 is a diagram of another implementation environment according to one embodiment of the present application. Refer to Figure 2. The implementation environment includes a source device 10, a destination device 20, a link 30, and a storage device 40. The source device 10 can generate an encoded image. Therefore, the source device 10 is also called the image encoding device or encoder side. The destination device 20 can decode the encoded image generated by the source device 10. Therefore, the destination device 20 is also called the image decoding device or decoder side. The link 30 can receive the encoded image generated by the source device 10 and transmit the encoded image to the destination device 20. The storage device 40 may receive the encoded image generated by the source device 10 and store the encoded image. In this case, the destination device 20 may retrieve the encoded image directly from the storage device 40. Alternatively, the storage device 40 may correspond to a file server or another intermediate storage device that can store the encoded image generated by the source device 10. In this case, the destination device 20 may stream or download the encoded image stored in the storage device 40.

[0106] Each of the source device 10 and destination device 20 may include one or more processors and memory connected to those one or more processors. The memory may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, and any other medium accessible to the computer that can be used to store the necessary program code in the form of instructions or data structures. For example, each of the source device 10 and destination device 20 may include mobile phones, smartphones, personal digital assistants (PDAs), wearable devices, pocket personal computers (pocket PCs, PPCs), tablet computers, smart head units, smart televisions, smart speakers, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-car computers, and similar devices.

[0107] Link 30 may include one or more media or devices capable of transmitting an encoded image from source device 10 to destination device 20. In one possible implementation, link 30 may include one or more communication media that enable source device 10 to directly transmit an encoded image to destination device 20 in real time. In this embodiment of the present application, source device 10 may modulate the encoded image according to a communication standard, which may be a wireless communication protocol, and transmit the modulated image to destination device 20. One or more communication media may include wireless communication media and / or wired communication media. For example, one or more communication media may include a radio frequency (RF) spectrum or one or more physical transmission lines. One or more communication media may be part of a packet-based network. The packet-based network may be a local area network, a wide area network, a global network (e.g., the Internet), etc. One or more communication media may include routers, switches, base stations, other devices that facilitate communication from source device 10 to destination device 20, etc. This is not particularly limited in the embodiments of the present application.

[0108] In one possible implementation, the storage device 40 may store the received encoded image transmitted by the source device 10, and the destination device 20 may directly retrieve the encoded image from the storage device 40. In this case, the storage device 40 may include one of several types of distributed or locally accessed data storage media. For example, one of several types of distributed or locally accessed data storage media may be a hard disk drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded images.

[0109] In one possible implementation, the storage device 40 may correspond to a file server or another intermediate storage device capable of storing encoded images generated by the source device 10, and the destination device 20 may stream or download the images stored in the storage device 40. The file server may be any type of server capable of storing encoded images and sending encoded images to the destination device 20. In one possible implementation, the file server may include a network server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, a local disk drive, etc. The destination device 20 may retrieve the encoded images through any standard data connection (including an internet connection). Such standard data connection may include a wireless channel (e.g., a Wi-Fi® connection), a wired connection (e.g., a digital subscriber line (DSL) or cable modem), or a combination of a wireless channel and a wired connection suitable for retrieving encoded images stored in the file server. The transmission of the encoded image from the storage device 40 may be via streaming transmission, download transmission, or a combination of both.

[0110] The implementation environment shown in Figure 2 is merely one possible implementation. Furthermore, the technology in the embodiments of this application is not only applicable to the source device 10 capable of encoding an image and the destination device 20 capable of decoding the encoded image shown in Figure 2, but can also be applied to other devices capable of encoding an image and other devices capable of decoding the encoded image. This is not particularly limited to the embodiments of this application.

[0111] In the implementation environment shown in Figure 2, the source device 10 includes a data source 120, an encoder 100, and an output interface 140. In some embodiments, the output interface 140 may include a modulator / demodulator (modem) and / or a transmitter. The transmitter is also called a broadcaster. The data source 120 may include an image acquisition device (e.g., a camera), an archive containing previously captured images, a feed-in interface for receiving images from an image content provider, and / or a computer graphics system for generating images, or a combination of these image sources.

[0112] The data source 120 may transmit an image to the encoder 100, which may encode the received image transmitted from the data source 120 to obtain an encoded image. The encoder can send the encoded image to an output interface. In some embodiments, the source device 10 sends the encoded image directly to the destination device 20 through the output interface 140. In other embodiments, the encoded image may instead be stored in a storage device 40, and the destination device 20 then retrieves the encoded image for decoding and / or display.

[0113] In the implementation environment shown in Figure 2, the destination device 20 includes an input interface 240, a decoder 200, and a display device 220. In some embodiments, the input interface 240 includes a receiver and / or a modem. The input interface 240 can receive encoded images via link 30 and / or from storage device 40, and then transmit the encoded images to the decoder 200. The decoder 200 can decode the received encoded image to obtain a decoded image. The decoder can transmit the decoded image to the display device 220. The display device 220 may be integrated with the destination device 20 or located outside the destination device 20. Generally, the display device 220 displays the decoded image. The display device 220 may be any one of several types of display devices. For example, the display device 220 may be a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0114] Although not shown in Figure 2, in some aspects the encoder 100 and decoder 200 may be integrated with the encoder and decoder, respectively, and may include a suitable multiplexer-demultiplexer (MUX-DEMUX) unit or other hardware and software for encoding both audio and video in a common data stream or separate data streams. In some embodiments, where applicable, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol or another protocol such as the user datagram protocol (UDP).

[0115] Each of the encoder 100 and decoder 200 may be one of the following circuits: one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the technology in the embodiments of this application is partially implemented in software, the device may store instructions for the software in a suitable non-volatile computer-readable storage medium, and may execute the instructions in hardware through one or more processors to implement the technology in the embodiments of this application. Any one of the foregoing (including hardware, software, and combinations of hardware and software) may be considered one or more processors. Each of the encoder 100 and decoder 200 may be included in one or more encoders or decoders. Either an encoder or decoder may be integrated in a corresponding device as part of a combined encoder / decoder (codec).

[0116] In this embodiment of the present application, the encoder 100 may generally be said to “signal” or “send” some information to another device, for example, the decoder 200. The terms “signal” or “send” may generally refer to the transmission of syntax elements and / or other data used to decode a compressed image. Such transmission may occur in real time or near real time. Alternatively, such communication may occur after a period of time, for example, when the syntax elements in the encoded bitstream are stored in a computer-readable storage medium during encoding. The decoding device may then retrieve the syntax elements at any time after they have been stored in the medium.

[0117] Figure 3 shows yet another implementation environment according to an embodiment of the present application. In this implementation environment, the encoding and decoding method provided in the embodiment of the present application is applied to a virtual reality streaming scenario. Refer to Figure 3. This implementation environment includes an encoder side and a decoder side. The encoder side includes a video acquisition and preprocessing module (also called a preprocessing module), a video encoding module, and a transmission module. The decoder side includes a receiving module, a bitstream decoding module, and a rendering and display module.

[0118] The encoder-side acquisition module acquires video, which includes multiple frames of an image to be encoded. The preprocessing module then preprocesses each frame of the image. The video encoding module then encodes each preprocessed frame of the image according to the encoding method provided in the embodiments of this application to obtain a bitstream. The transmission module transmits the bitstream to the decoder side via a transport network. The decoder-side receiving module first receives the bitstream. The decoding module then decodes the bitstream according to the decoding method provided in the embodiments of this application to obtain image information. Finally, the rendering and display module renders and displays the image information. In addition, after obtaining the bitstream, the encoder side may, alternatively, store the bitstream.

[0119] It should be noted that the encoding and decoding methods provided in the embodiments of this application may be applied to multiple scenarios, and in various scenarios, the images to be encoded may all be images contained in image files, or all be images contained in video files. The encoded images may be in formats such as RGB, YUV444, or YUV420. It should be noted that, based on the implementation environments shown in Figures 1, 2, and 3, one of the following encoding methods may be performed by the encoder, and one of the following decoding methods may be performed by the decoder.

[0120] The encoding method provided in the embodiments of this application will be described below.

[0121] Figure 4 is a flowchart of an encoding method according to one embodiment of the present invention. This method is applied to the encoder side. As shown in Figure 4, the method includes the following steps.

[0122] Step 401: Determine the first and second image features of the image to be encoded. The first image feature is obtained by quantizing the second image feature based on the first quantization step.

[0123] In this embodiment of the present application, the encoder inputs the image to be encoded into an image feature extraction network in order to obtain a second image feature of the image. Here, the second image feature is an unquantized image feature. The encoder quantizes the second image feature based on a first quantization step to obtain a first image feature, and the first image feature is a quantized image feature.

[0124] Note that each of the first and second image features includes image features of multiple feature points, and each image feature of a feature point in the first image feature may be called the first feature value of the corresponding feature point, and each image feature of a feature point in the second image feature may be called the second feature value of the corresponding feature point.

[0125] Optionally, the image feature extraction network is a convolutional neural network, the first image feature is represented by a first feature map, the second image feature is represented by a second feature map, and each of the first and second feature maps has multiple feature points. The image feature extraction network in this embodiment of the present application is acquired through pre-training, and it should be noted that the network structure, training method, etc., of the image feature extraction network are not limited to the embodiment of the present application. For example, the image feature extraction network may be a fully connected network or the aforementioned convolutional neural network, and the convolution in the convolutional neural network may be 2D convolution or 3D convolution. In addition, the number of network layers included in the image feature extraction network and the number of nodes in each layer are not limited to the embodiment of the present application.

[0126] Figure 5 shows the structure of an image feature extraction network according to one embodiment of the present invention. Refer to Figure 5. The image feature extraction network is a convolutional neural network, which includes four convolutional (Conv) layers and three interleaved and cascaded grab detection network (GDN) layers. The size of the convolutional kernel in each convolutional layer is 5 × 5, the number of channels in the output feature map is M, and downsampling is performed by a factor of 2 for both width and height in each convolutional layer. For example, for an input image of 16W × 16H × 3, the size of the feature map output by the convolutional neural network is W × H × M. Note that the structure of the convolutional neural network shown in Figure 5 is not intended to limit the embodiments of the present invention. For example, the size of the convolutional kernel, the number of channels in the feature map, the downsampling factor, the number of downsampling iterations, the number of convolutional layers, etc., may all be adjusted.

[0127] Optionally, in this embodiment of the present application, the first quantization step is obtained via a gain network based on the bitrate of the image, and the gain network is used to determine the quantization step corresponding to each of several types of bitrates. For example, the encoder determines a first quality factor based on the bitrate of the image and inputs the first quality factor into the gain network to obtain the first quantization step. Note that different bitrates may correspond to different quality factors, and different quantization steps may be obtained via the gain network. Alternatively, a mapping relationship between bitrate and quantization step may be stored in advance, and the corresponding quantization step may be obtained as the first quantization step from the mapping relationship based on the bitrate of the image. Optionally, in some other embodiments, the first quantization step is a first quantization step corresponding to the first quality factor obtained from the mapping relationship between the quality factor and the quantization step after the first quality factor has been determined based on the bitrate of the image to be encoded.

[0128] The quality factors may be replaced by quantization parameters. The aforementioned implementation process may include multiple quantization methods, such as uniform quantization or scalar quantization. Scalar quantization may further have an offset, specifically, scalar quantization is performed based on the quantization step after biasing is performed on the data to be quantized (e.g., a second image feature) based on the offset. Optionally, in this embodiment of the present application, the quantization process performed on the image feature includes quantization and rounding. For example, the second image feature is represented by a feature map y, the numerical range of the second image feature is within the interval [0, 100], the first quantization step is represented by q1, where q1 is 0.5, and the first image feature is represented by a feature map y s It is assumed that it will be represented by this. In this case, the encoder quantizes the feature value of each feature point in the feature map y and uses it to represent the feature map y s Obtain the feature map y s The feature values ​​of each feature point within the image are rounded to obtain a feature map ys', i.e., the first image features are obtained. The numerical range of the first image features falls within the interval [0, 50]. As an example, uniform quantization is used. The feature value obtained by quantizing an arbitrary feature value x based on the quantization step q is x' = x * q.

[0129] Optionally, the first quantization steps for quantizing the image features of feature points may be the same or different. For example, feature points in the same channel may use the same first quantization step, or feature values ​​in different channels at the same spatial location may use the same first quantization step. Assume the size of the second image feature to be quantized is W × H × M. Under an arbitrary quality factor i, the first quantization step for a feature point in the second image feature with coordinates (k,j,l) is q i (k,j,l) where q i(k,j,l) may be obtained through learning by a gain network, or based on a stored mapping relation. k∈[1,W], j∈[1,H], l∈[1,M]. It should be understood that different quantization parameters QP correspond to different quantization steps q, and there is a one-to-one correspondence between quantization parameters QP and quantization steps q. For example, in some standard solutions, the mapping relation between quantization parameters and quantization steps is q=(2 1 / 6 ) QP-4 It may also be expressed as follows. Of course, another function may be designed as an alternative to represent the mapping relationship between QP and q.

[0130] Please note that the following quantization processing methods are the same as those described herein. For the following quantization processing methods, please refer to the methods described herein. Details will not be described again below in the embodiments of this application.

[0131] Step 402: Determine the first hyperplier feature of the second image feature.

[0132] In this embodiment of the present application, in order to later obtain the probability distribution parameters of the unquantized image features (i.e., the first probability distribution parameters) by using step 404, the encoder determines the first hyperplier feature of the unquantized image features (for example, the first hyperplier feature of the second image feature) before step 404. There are several implementations in which the encoder determines the first hyperplier feature of the second image feature. Two of these implementations are described below.

[0133] A first implementation in which the encoder determines the first hyperplier feature of a second image feature is to input the second image feature into the hyperencoder network to obtain the first hyperplier feature. That is, the encoder inputs the unquantized image feature into the hyperencoder network to obtain the first hyperplier feature of the unquantized image feature. A second implementation in which the encoder determines the first hyperplier feature of a second image feature is to dequantize the first image feature based on the first quantization step to obtain the third image feature of the image; input the third image feature into the hyperencoder network to obtain the first hyperplier feature. The first hyperplier feature may also be considered the first hyperplier feature of the third image feature, or the first hyperplier feature of the second image feature. The second image feature is the image feature obtained before quantization, and the third image feature is the image feature obtained through dequantization. Therefore, although the first image feature and the third image feature are numerically different, the image information represented by the first and third image features is essentially equivalent.

[0134] Optionally, the hyperencoder network outputs a first hyperplier feature. Alternatively, the hyperencoder network outputs a second hyperplier feature. The encoder side quantizes the second hyperplier feature based on a third quantization step to obtain the first hyperplier feature, which is the quantized hyperplier feature. The third quantization step is the same as or different from the first quantization step. That is, the quantization operation may also be performed on the hyperplier feature to compress it. Optionally, the hyperplier feature may also be called side information, which may be understood as further extracting features from the image feature.

[0135] It should be noted that each of the first and second hyperplier features includes hyperplier features of multiple feature points. For example, in step 404, the image features of each feature point in the first image feature are input to the hyperencoder network to obtain the hyperplier features of the feature points in the first hyperplier feature. In addition, the hyperencoder network in this embodiment of the present application is acquired through pre-training. The network structure, training method, etc., of the hyperencoder network are not limited to the embodiment of the present application. For example, the hyperencoder network may be a convolutional neural network or a fully connected network. Optionally, the hyperencoder network as used herein may also be referred to as a hyperplier network.

[0136] Step 403: Encode the first hyperplier feature into a bitstream.

[0137] In this embodiment of the present application, the encoder encodes the first hyperplier feature into a bitstream. The decoder then performs decoding based on the first hyperplier feature.

[0138] Optionally, the encoder encodes the first hyperplier feature into a bitstream through entropy encoding. For example, the encoder encodes the first hyperplier feature into a bitstream through entropy encoding based on specified probability distribution parameters. The specified probability distribution parameters are probability distribution parameters determined in advance via a probability distribution estimation network. The network structure, training method, etc., of the probability distribution estimation network are not limited to the embodiments of this application.

[0139] Step 404: Determine the first probability distribution parameters based on the first hyperplier feature via the probability distribution estimation network.

[0140] In this embodiment of the present application, a probability distribution estimation network is used to determine the probability distribution parameters of an unquantized image feature. Based on this, the encoder side determines a first probability distribution parameter based on a first hyperplier feature via the probability distribution estimation network, and the first probability distribution parameter represents the probability distribution of the unquantized image feature (e.g., a second or third image feature). It should be noted that the probability distribution parameter herein may be any parameter for representing the probability distribution of an image feature, such as the mean and variance (or standard deviation) of a Gaussian distribution, the positional and scale parameters of a Laplace distribution, or the mean and scale parameters of a logistic distribution, and, in other examples, another model parameter.

[0141] Optionally, to align with the decoder's decoding process, the encoder parses the bitstream to obtain a first hyperplier feature, and based on the first hyperplier feature obtained through parsing, determines a first probability distribution parameter via a probability distribution estimation network.

[0142] From the above explanation, it can be seen that the first hyperplier feature may be a quantized hyperplier feature or an unquantized hyperplier feature. Based on this, in an implementation where the first hyperplier feature is a quantized hyperplier feature, the encoder side dequantizes the first hyperplier feature based on the third quantization step to obtain a second hyperplier feature, and inputs the second hyperplier feature into the probability distribution estimation network to obtain the first probability distribution parameter. In an implementation where the first hyperplier feature is an unquantized hyperplier feature, the encoder side inputs the first hyperplier feature into the probability distribution estimation network to obtain the first probability distribution parameter. The probability distribution estimation network may be considered a hyperdecoder network, which is used to determine the probability distribution parameter based on the hyperplier feature.

[0143] In addition to the aforementioned implementation for determining the first probability distribution parameter, the encoder may, alternatively, determine the first probability distribution parameter based on contextual features to improve its accuracy. This is described below.

[0144] Optionally, in one implementation, the encoder inputs a third image feature of the image into a context network to obtain the context feature of the third image feature. The third image feature is obtained by dequantizing the first image feature based on the first quantization step. The encoder determines the first plier feature based on the first hyperplier feature, and inputs the first plier feature and the context feature into a probability distribution estimation network to obtain the first probability distribution parameters. That is, the encoder extracts the context feature from the dequantized image feature and determines the first probability distribution parameters based on the context feature and the first plier feature.

[0145] In an implementation where the first hyperplier feature is a quantized hyperplier feature, the encoder parses the bitstream to obtain the first hyperplier feature, dequantizes the first hyperplier feature obtained through parsing based on a third quantization step to obtain the second hyperplier feature, and inputs the second hyperplier feature into the hyperdecoder network to obtain the first plier feature.

[0146] Optionally, since the image information represented by the second and third image features is essentially equivalent based on the above explanation, the encoder may input the second image feature into the context network to obtain the context feature of the second image feature, where the context feature of the second image feature is the context feature of the third image feature.

[0147] Note that the contextual features of the third image feature include the contextual features of each feature point in multiple feature points, and the first probability distribution parameter includes the probability distribution parameters of multiple feature points. In other words, the encoder may determine the contextual features of each feature point and the probability distribution parameters of each feature point in parallel.

[0148] In an alternative implementation, the encoder inputs a first image feature into a context network to obtain context features of the first image feature, determines a first plier feature based on a first hyperplier feature, quantizes the first plier feature based on a second quantization step to obtain a second plier feature, and inputs the second plier feature and context features into a probability distribution estimation network to obtain the first probability distribution parameter. Alternatively, the encoder may extract context features from an inversely quantized image feature, then obtain a second plier feature by applying a quantization operation to the first plier feature, and determine the first probability distribution parameter based on the second plier feature and context features. The second quantization step may be the same as or different from the first quantization step.

[0149] In this implementation, the implementation in which the encoder determines the first plier feature is consistent with the relevant process in the previously described implementation, and it should be noted that the details will not be described again here. In addition, in the implementation in which the probability distribution is determined based on the context feature, the hyperdecoder network is used to determine the plier feature based on the hyperplier feature, and the probability distribution estimation network is used to determine the probability distribution parameters based on the plier feature and the context feature. Both the hyperdecoder network and the probability distribution estimation network in this embodiment of the present application are acquired through pre-training. The network structure, training method, etc., of the hyperdecoder network and the probability distribution estimation network are not limited to the embodiment of the present application. For example, the hyperdecoder network and the probability distribution estimation network, respectively, may be a convolutional neural network, a recurrent neural network, a fully connected network, etc.

[0150] Optionally, the probability distribution estimation network in this embodiment of the present application is modeled using a Gaussian model (e.g., a Gaussian single model (GSM) or a Gaussian mixture model (GMM)). Specifically, it is assumed that the feature values ​​of each feature point in an unquantized image feature (e.g., a second or third image feature) are fitted to the Gaussian single model or Gaussian mixture model, and that the first probability distribution parameters obtained by the probability distribution estimation network include a mean μ and a standard deviation σ. Optionally, the probability distribution estimation network may alternatively use a Laplace distribution model. Correspondingly, the first probability distribution parameters include a position parameter λ and a scale parameter b. The probability distribution estimation network may alternatively use a logistic distribution model. Correspondingly, the first probability distribution parameters include a mean μ and a scale parameter S. A Gaussian model is used as an example. The probability distribution function corresponding to the probability distribution of any feature point in the first probability distribution parameter is given by equation (1) below, where x is the second feature value of the feature point:

number

[0151] Step 405: Based on the first quantization step, the first probability distribution parameter is quantized to obtain the second probability distribution parameter.

[0152] After obtaining the first probability distribution parameter of the unquantized image feature, the encoder quantizes the first probability distribution parameter based on the first quantization step to obtain a second probability distribution parameter, where the second probability distribution parameter represents the probability distribution of the quantized image feature (i.e., the first image feature). Based on the first quantization step, the probability distribution parameter of each feature point in the first probability distribution parameter is quantized to obtain the probability distribution parameter of the corresponding feature point in the second probability distribution parameter.

[0153] For example, a Gaussian model is used as an example. The first quantization step of the feature point with coordinates (k, j, l) is q i (k, j, l), and the probability distribution parameters of the feature point in the first probability distribution parameter are μ(k, j, l) and σ(k, j, l). μ(k, j, l) and σ(k, j, l) are quantized based on the quantization step q i (k, j, l) to obtain the probability distribution parameters μ S (k, j, l) and σ S (k, j, l) of the feature point in the second probability distribution parameter. When uniform quantization is performed, μ S = μ / q, σ S = σ / q. The probability distribution function corresponding to the probability distribution parameter of any feature point in the second probability distribution parameter is shown in the following formula (2). Here, x is the first feature value of the feature point:

Number

[0154] Here, the principle of step 405 is explained. It is assumed that the quantization operation is uniform quantization. A Gaussian model is used as an example. The probability distribution function of the variable x is shown in the above formula (1). In this case, the probability P1 of the variable x in the interval [a2*q, a1*q] is shown in the following formula (3). Here, q is the quantization step.

Number

[0155] The variable x is quantized based on the quantization step q, and the quantized variable x' = x / q is obtained. In this case, the probability P1 of the variable x in the interval [a2*q, a1*q] is equal to the probability P2 of the variable x' in the interval [a2, a1]. Based on this, the probability distribution function of the variable x' is assumed to be g(x). In this case,

number

number

[0156] Similar to the Laplace distribution model and the logistic model, the second probability distribution parameter may be obtained from the first probability distribution parameter by scaling (i.e., quantizing) the parameter of the corresponding model.

[0157] Step 406: Encode the first image features into a bitstream based on the second probability distribution parameter.

[0158] In this embodiment of the present application, after obtaining a second probability distribution parameter, the encoder encodes the first image features into a bitstream based on the second probability distribution parameter. The encoder encodes the image features of each feature point in the first image features into a bitstream based on the probability distribution parameter of each feature point in the second probability distribution parameter. For example, the encoder encodes the first image features into a bitstream through entropy encoding.

[0159] The above describes the encoding method provided in the embodiments of the present application. The above content will be explained again below with reference to Figures 6 to 10.

[0160] Figure 6 is a flowchart of an encoding and decoding method according to one embodiment of the present invention. Refer to Figure 6. In the encoding process, the image to be encoded is input to an encoder (Enc) network to obtain a quantized feature map y (i.e., a second image feature). The encoder network is an image feature extraction network. Based on the quantization step q1 (i.e., the first quantization step), quantization (Q) is performed on each feature element in the feature map y to obtain the feature map y s This is obtained. Feature map y s Rounding (R) is performed on each feature element within the feature map ^y s [y with ^] s For convenience, this is sometimes written as such; the same applies below) (i.e., the first image feature) is obtained. In addition, the feature map y is input to a hyperencoder (HyEnc) network to obtain a hyperplier feature z. Optionally, the hyperplier feature z is quantized based on quantization step q2 to obtain a hyperplier feature ^z (i.e., the first hyperplier feature). Quantization step q2 may be the same as or different from quantization step q1. The hyperplier feature ^z is encoded into a bitstream via entropy encoding (AE2). The hyperplier feature ^z is then obtained by parsing the bitstream via entropy decoding (AD2). Optionally, inverse quantization (IQ) is performed on the hyperplier feature ^z based on quantization step q2 to obtain a hyperplier feature z. The hyperplier feature z is input to a probability distribution estimation network to obtain probability distribution parameters μ and σ (i.e., the first probability distribution parameters). The probability distribution parameters μ and σ are quantized based on the quantization step q1, and the feature map ^y sThe probability distribution parameter μ of each feature element within S and σ S (i.e., the second probability distribution parameter) is obtained. Feature map ^y s The probability distribution parameter μ of each feature element within S and σ S Based on this, feature map ^y s This is encoded into the bitstream via entropy encoding (AE1).

[0161] Figure 7 is a flowchart of another encoding and decoding method according to one embodiment of the present invention. Refer to Figure 7. In the encoding process, the image to be encoded is input to an encoder (Enc) network to obtain a feature map y to be quantized. The encoder network is an image feature extraction network. Based on the quantization step q1, quantization (Q) is performed on each feature element in the feature map y to obtain the feature map y s This is obtained. Feature map y s Rounding (R) is performed on each feature element within the feature map ^y s The feature map y is then input into a hyperencoder (HyEnc) network to obtain the hyperplier feature z. Optionally, the hyperplier feature ^z (i.e., the first hyperplier feature) is obtained. The hyperplier feature ^z is encoded into a bitstream via entropy encoding (AE2). The hyperplier feature ^z is then obtained by parsing the bitstream via entropy decoding (AD2). Optionally, inverse quantization (IQ) is performed on the hyperplier feature ^z based on quantization step q2 to obtain the hyperplier feature z. The hyperplier feature z is then input into a hyperdecoder (HyDec) network (i.e., a probability distribution estimation network) to obtain the pryor feature φ (i.e., the first pryor feature). Also, the feature map ^y sThe data is dequantized based on the first quantization step q1 to obtain the feature map ^y (i.e., the third image feature). The feature map ^y is input to the context (Ctx) network to obtain the context features of each feature point in the feature map ^y. The context features and plier features φ of each feature point are input to the probability distribution estimation network to obtain the probability distribution parameters μ and σ (i.e., the first probability distribution parameters) of each feature element in the feature map y. The probability distribution parameters μ and σ are quantized based on the quantization step q1 to obtain the feature map ^y s The probability distribution parameter μ of each feature element within S and σ S (i.e., the second probability distribution parameter) is obtained. Feature map ^y s The probability distribution parameter μ of each feature element within S and σ S Based on this, feature map ^y s This is encoded into a bitstream via entropy encoding (AE1).

[0162] Figure 8 is a flowchart of yet another encoding and decoding method according to one embodiment of the present application. The difference between Figure 8 and Figure 6 is that in the encoding process, the feature map ^y s The first step is to dequantize the data based on the quantization step q1 to obtain the feature map ^y. The feature map ^y is then input into the hyperencoder network to obtain the hyperprior feature z.

[0163] Figure 9 is a flowchart of yet another encoding and decoding method according to one embodiment of the present application. The difference between Figure 9 and Figure 7 is that in the encoding process, the feature map ^y s The first step is to dequantize the data based on the quantization step q1 to obtain the feature map ^y. The feature map ^y is then input into the hyperencoder network to obtain the hyperprior feature z.

[0164] Figure 10 is a flowchart of yet another encoding and decoding method according to one embodiment of the present application. The difference between Figure 10 and Figures 7 and 9 is that in the encoding process, the feature map ^y s The input is then passed to the hyperencoder network to obtain the hyperprior feature ^z. Furthermore, after the prior feature φ is obtained via the hyperdecoder network, a quantization operation is applied, specifically, the prior feature φ is quantized based on the quantization step q3 (i.e., the second quantization step) to obtain the prior feature φ s The following is obtained. Quantization step q3 is the same as or different from quantization step q1. Context features and pryor features φ of each feature point. s This is input into the probability distribution estimation network to obtain the first probability distribution parameters μ and σ.

[0165] In conclusion, in this embodiment of the present application, in order to obtain probability distribution parameters for quantized image features, in the encoding process, a first probability distribution parameter is determined based on the hyperplier features of the unquantized image features via a probability distribution estimation network. Here, the first probability distribution parameter represents the probability distribution of the unquantized image features. Then, based on a first quantization step (i.e., a quantization step for quantizing the image features), the first probability distribution parameter is quantized to obtain a second probability distribution parameter that represents the probability distribution of the quantized image features. In this solution, it is found that only the probability distribution estimation network for determining the probability distribution parameters of the unquantized image features needs to be trained. Even in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and unaffected by the quantization step. In other words, the input numerical range of the probability distribution estimation network does not change with the bitrate. Therefore, training the probability distribution estimation network is not difficult, the network training is stable, and a probability distribution estimation network with good performance can be trained. This helps to improve encoding and decoding performance.

[0166] Figure 11 is a flowchart of another encoding method according to one embodiment of the present invention. This method is applied to the encoder side. Note that the probability distribution estimation network in the embodiment of Figure 4 is assumed to be called the first probability distribution estimation network. In this case, the difference between the embodiment of Figure 11 and the embodiment of Figure 4 is that in the encoding method shown in Figure 11, the second probability distribution parameters are obtained directly through the second probability distribution estimation network, specifically, the second probability distribution estimation network directly outputs the probability distribution parameters of the quantized image features. The network parameters of the second probability distribution estimation network are obtained based on the network parameters of the first probability distribution estimation network and the first quantization step. Thus, only the first probability distribution estimation network needs to be trained. Refer to Figure 11. This method includes the following steps.

[0167] Step 1101: Determine the first and second image features of the image to be encoded. The first image feature is obtained by quantizing the second image feature based on the first quantization step.

[0168] In this embodiment of the present application, the encoder inputs the image to be encoded into the image feature extraction network in order to obtain a second image feature of the image. The encoder quantizes the second image feature based on the first quantization step to obtain the first image feature. The specific implementation process is the same as the specific implementation process of step 401 in the embodiment of Figure 4. For details, please refer to the relevant description in step 401. Details are not described again here.

[0169] Step 1102: Determine the first hyperplier feature of the second image feature.

[0170] Optionally, in one implementation, the encoder inputs a second image feature into the hyperencoder network to obtain a first hyperprior feature. In another implementation, the encoder dequantizes the first image feature based on the first quantization step to obtain a third image feature of the image, inputs the third image feature into the hyperencoder network to obtain the first hyperprior feature. The specific implementation process is the same as the specific implementation process of step 402 in the embodiment of Figure 4. For details, please refer to the relevant description of step 402. Details will not be explained again here.

[0171] Step 1103: Encode the first hyperplier feature into a bitstream.

[0172] In this embodiment of the present application, the encoder encodes the first hyperplier feature into a bitstream, thereby allowing the decoder to decode based on the first hyperplier feature. Optionally, the encoder encodes the first hyperplier feature into a bitstream through entropy encoding. The specific implementation process is the same as the specific implementation process of step 403 in the embodiment of Figure 4. For details, please refer to the relevant description in step 403. Details are not described again here.

[0173] Step 1104: Determine the second probability distribution parameters based on the first hyperplier features via the second probability distribution estimation network. Here, the network parameters of the second probability distribution estimation network are obtained based on the network parameters of the first probability distribution estimation network and the first quantization step, and the first probability distribution estimation network is used to determine the probability distribution of the unquantized image features.

[0174] Optionally, to align with the decoder's decoding process, the encoder parses the bitstream to obtain a first hyperplier feature, and based on the first hyperplier feature obtained through parsing, determines a second probability distribution parameter via a second probability distribution estimation network.

[0175] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets. Based on this, the weights and offsets of the last convolutional layer in the second probability distribution estimation network are obtained based on the weights and offsets of the last convolutional layer in the first probability distribution estimation network and a first quantization step. Optionally, the second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by the first quantization step. Alternatively, in some embodiments, the second probability distribution estimation network is obtained by adjusting the network parameters of the last layer in the first probability distribution estimation network in a binary leftward or rightward displacement manner such that the adjusted network parameters are equal to the network parameters obtained before adjustment multiplied by the first quantization step.

[0176] For example, the last layer in the first probability distribution estimation network is a convolutional layer, and the weights w*q1 and offset b*q1 of the last layer in the second probability distribution estimation network are obtained by multiplying both the weights w and offset b of the convolutional layer by the first quantization step q1. Note that the network layers in the second probability distribution estimation network, excluding the last layer, are the same as those in the first probability distribution estimation network. In other words, the difference between the second probability distribution estimation network and the first probability distribution estimation network lies in the network parameters of the last layer. Thus, the first probability distribution estimation network may be obtained through training based on unquantized image features. After training for the first probability distribution estimation network is complete, the second probability distribution estimation network may be obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by the first quantization step.

[0177] Similar to the first hyperplier feature in step 404 in the embodiment of Figure 4, in this embodiment of the present application, the first hyperplier feature may be a quantized hyperplier feature or an unquantized hyperplier feature. Based on this, in an implementation where the first hyperplier feature is a quantized hyperplier feature, the encoder side dequantizes the first hyperplier feature based on the third quantization step to obtain a second hyperplier feature, and inputs the second hyperplier feature into the second probability distribution estimation network to obtain the second probability distribution parameter. If the first hyperplier feature is an unquantized hyperplier feature, the encoder side inputs the first hyperplier feature into the second probability distribution estimation network to obtain the second probability distribution parameter.

[0178] In addition to the aforementioned implementation for determining the second probability distribution parameter, the encoder may, alternatively, determine the second probability distribution parameter based on contextual features to improve its accuracy. This will be explained below.

[0179] In one implementation, the encoder inputs a third image feature of the image into a context network to obtain the context features of the third image feature. The third image feature is obtained by dequantizing the first image feature based on the first quantization step. The encoder determines the first plier feature based on the first hyperplier feature, and inputs the first plier feature and the context features into a second probability distribution estimation network to obtain the second probability distribution parameters. That is, the encoder extracts the context features from the dequantized image features and determines the second probability distribution parameters based on the context features and the first plier feature.

[0180] In an alternative implementation, the encoder inputs a first image feature into a context network to obtain context features of the first image feature. The encoder determines a first plier feature based on the first hyperplier feature and obtains a second plier feature by quantizing the first plier feature based on a second quantization step. The encoder inputs the second plier feature and context features into a second probability distribution estimation network to obtain second probability distribution parameters. That is, the encoder extracts context features from the quantized image feature, then obtains a second plier feature by applying a quantization operation to the first plier feature, and determines second probability distribution parameters based on the second plier feature and context features. The second quantization step is either the same as or different from the first quantization step.

[0181] Note that the specific implementation process for step 1104 is the same as the specific implementation process for step 404 in the embodiment shown in Figure 4. For details, please refer to the relevant explanation in step 404. Details will not be explained again here.

[0182] Step 1105: Encode the first image feature into a bitstream based on the second probability distribution parameter.

[0183] In this embodiment of the present application, the second probability distribution parameter includes probability distribution parameters for a plurality of feature points, and the encoder encodes the image features of each feature point in the first image feature into a bitstream based on the probability distribution parameter of each feature point in the second probability distribution parameter. For example, the encoder encodes the second image feature into a bitstream through entropy encoding.

[0184] The probability distribution estimation network in the encoding and decoding procedures shown in Figures 6 to 10 will be referred to as the first probability distribution estimation network. In this case, when the first probability distribution estimation network in Figures 6 to 10 is replaced by the second probability distribution estimation network, after the quantization operation on the probability distribution parameters is removed, a flowchart of the encoding and decoding method consistent with that in the embodiment of Figure 11 is obtained.

[0185] In conclusion, in the encoding process of this embodiment of the present application, the hyperplier features of the unquantized image features are also determined, but the second probability distribution parameters are subsequently obtained directly via a second probability distribution estimation network. The second probability distribution estimation network is obtained by processing the network parameters within the first probability distribution estimation network based on the first quantization step. This solution shows that only the first probability distribution estimation network (for determining the probability distribution parameters of the unquantized image features) needs to be trained. Even in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and unaffected by the quantization step. In other words, the input numerical range of the first probability distribution estimation network does not change with the bitrate. Therefore, training the first probability distribution estimation network is not difficult, the network training is stable, and a first probability distribution estimation network with good performance can be trained. This helps to improve encoding and decoding performance.

[0186] The decoding methods provided in the embodiments of the present application are described below. Note that the decoding method shown in Figure 12 corresponds to the encoding method shown in Figure 4, and the decoding method shown in Figure 13 corresponds to the decoding method shown in Figure 11.

[0187] Figure 12 is a flowchart of a decoding method according to one embodiment of the present invention. This method is applied to the decoder side. Refer to Figure 12. This method includes the following steps.

[0188] Step 1201: Parse the bitstream to obtain the first hyperplier feature of the image to be decoded.

[0189] In this embodiment of the present application, the decoder first parses the bitstream to obtain a first hyperplier feature of the image to be decoded. Optionally, the decoder parses the bitstream to obtain a first hyperplier feature through entropy decoding. For example, the decoder parses the bitstream through entropy decoding to obtain a first hyperplier feature based on specified probability distribution parameters. The specified probability distribution parameters are probability distribution parameters determined in advance via a probability distribution estimation network. The network structure, training method, etc., of the probability distribution estimation network are not limited to this embodiment. Note that the first hyperplier feature includes hyperplier features of multiple feature points. The first hyperplier feature obtained by the decoder through parsing is consistent with the first hyperplier feature determined by the encoder. In other words, the first hyperplier feature acquired by the decoder is the first hyperplier feature of the second image feature or the first hyperplier feature of the third image feature as described in the embodiment of Figure 4. The second image feature is an unquantized image feature, and the third image feature is an inversely quantized image feature.

[0190] Step 1202: Determine the first probability distribution parameter based on the first hyperplier feature via the probability distribution estimation network. The first probability distribution parameter represents the probability distribution of the unquantized image features of the image.

[0191] In this embodiment of the present application, the first hyperplier feature includes hyperplier features of a plurality of feature points, and the first probability distribution parameter includes probability distribution parameters of a plurality of feature points. The decoder determines the probability distribution parameter of each feature point in the first probability distribution parameter based on the hyperplier features of each feature point in the first hyperplier feature via a probability distribution estimation network.

[0192] Note that in implementations where encoding and decoding are not performed via a context network, the decoder can decode multiple feature points in parallel. In implementations where encoding and decoding are performed via a context network, the decoder cannot decode multiple feature points simultaneously. For example, the decoder may decode multiple feature points sequentially; the decoder may decode the feature points of a channel sequentially; the decoder may decode multiple groups of feature points sequentially, where the number of feature points in each group may differ; or the decoder may decode multiple feature points in a different order.

[0193] In addition, as explained above, the first hyperplier feature can be either a quantized or an unquantized hyperplier feature. Based on this, in implementations where encoding and decoding are not performed via a context network and the first hyperplier feature is a quantized hyperplier feature, the decoder dequantizes the first hyperplier feature based on a third quantization step to obtain a second hyperplier feature, and inputs the second hyperplier feature into the probability distribution estimation network to obtain the first probability distribution parameter. In implementations where encoding and decoding are not performed via a context network and the first hyperplier feature is an unquantized hyperplier feature, the decoder inputs the first hyperplier feature into the probability distribution estimation network to obtain the first probability distribution parameter.

[0194] In implementations that perform encoding and decoding via a context network, the first feature point is assumed to be one of several feature points. In this case, the decoder performs the following operations on the first feature point to determine its probability distribution parameters: determine the context features of the first feature point based on the image features of the decoded feature point in the first image feature; determine the first plier features of the first feature point based on the hyperplier features of the first feature point; and determine the probability distribution parameters of the first feature point via a probability distribution estimation network based on the first plier features and the context features of the first feature point, i.e., determine the probability distribution parameters of the first feature point in the first probability distribution parameters.

[0195] In the implementation where the first hyperplier feature is a quantized hyperplier feature, the decoder side, based on the third quantization step, dequantizes the first hyperplier feature obtained through parsing to obtain a second hyperplier feature, inputs the second hyperplier feature into the hyperdecoder network to obtain the first plier feature for each of the multiple feature points. For the first feature point, the decoder side dequantizes the hyperplier feature of the first feature point in the first hyperplier feature to obtain the hyperplier feature of the first feature point in the second hyperplier feature, inputs the hyperplier feature of the first feature point in the second hyperplier feature into the hyperdecoder network to obtain the first plier feature of the first feature point. In the implementation where the first hyperplier feature is a quantized hyperplier feature, the decoder side inputs the first hyperplier feature obtained through parsing into the hyperdecoder network to obtain the first plier feature for each of the multiple feature points. For the first feature point, the decoder inputs the plier feature of the first feature point in the first plier feature into the hyperdecoder network to obtain the first plier feature of the first feature point.

[0196] Optionally, the implementation process for the decoder to determine the context features of the first feature point based on the image features of the decoded feature point in the first image feature is as follows: The decoder determines the surrounding feature points of the first feature point from the decoded feature point, and dequantizes the image features of the surrounding feature points in the first image feature based on the first quantization step to obtain the surrounding features of the first feature point. Then, the decoder inputs the surrounding features of the first feature point into the context network to obtain the context features of the first feature point. Correspondingly, the implementation process for the decoder to determine the probability distribution parameter of the first feature point within the first probability distribution parameter based on the first plier feature and the context features of the first feature point via a probability distribution estimation network is as follows: The decoder inputs the first plier feature and the context features of the first feature point into the probability distribution estimation network to obtain the probability distribution parameter of the first feature point within the first probability distribution parameter. In other words, the decoder extracts contextual features from the inversely quantized image features and determines a first probability distribution parameter based on the contextual features and the first prior features. The peripheral feature points of the first feature point include one or more feature points in the neighborhood of the first feature point.

[0197] Optionally, an alternative implementation process in which the decoder determines the context features of the first feature point based on the image features of the decoded feature point in the first image feature is as follows: The decoder determines the surrounding feature points of the first feature point from the decoded feature point, inputs the image features of the surrounding feature points in the first image feature into the context network to obtain the context features of the first feature point. Correspondingly, an implementation process in which the decoder determines the probability distribution parameter of the first feature point within the first probability distribution parameter based on the first plier feature and the context features of the first feature point via a probability distribution estimation network is as follows: The decoder quantizes the first plier feature of the first feature point based on a second quantization step to obtain the second plier feature of the first feature point, inputs the second plier feature and the context features of the first feature point into the probability distribution estimation network to obtain the probability distribution parameter of the first feature point within the first probability distribution parameter. In other words, the decoder extracts context features from the quantized image features, then obtains second prior features by applying a quantization operation to the first prior features, and determines the first probability distribution parameters based on the second prior features and the context features. The second quantization step is either the same as or different from the first quantization step.

[0198] In implementations where encoding and decoding are performed via a context network, it should be noted that the implementation in which the decoder determines the first plier feature of the first feature point is the same as the relevant part in the embodiments described above. Details will not be explained again here. In addition, the image features of the decoded feature points in the first image feature are obtained by decoding the bitstream according to steps 1202 to 1204. Specifically, in implementations where encoding and decoding are performed via a context network, the decoder parses the bitstream according to steps 1202 to 1204 to sequentially obtain the image features of all feature points in the first image feature, and the decoder can decode at least one feature point each time. Furthermore, the first image feature is an image feature obtained by quantizing the second image feature of the image based on a first quantization step, the quantization operation is performed in the encoding process, and the second image feature is an image feature obtained in the encoding process.

[0199] Step 1203: Based on the first quantization step, the first probability distribution parameter is quantized to obtain the second probability distribution parameter.

[0200] The second probability distribution parameter is the probability distribution of multiple feature points. After obtaining the probability distribution parameter of the first feature point in the first probability distribution parameter through decoding, the decoder quantizes the probability distribution parameter of the first feature point in the first probability distribution parameter based on the first quantization step to obtain the probability distribution parameter of the first feature point in the second probability distribution parameter. Note that in implementations where encoding and decoding are not performed via a context network, the decoder may quantize the probability distribution parameters of multiple feature points in the first probability distribution parameter in parallel. In implementations where encoding and decoding are performed via a context network, each time the decoder obtains the probability distribution parameter of at least one feature point in the first probability distribution parameter, the decoder quantizes the probability distribution parameter of at least one feature point in the first probability distribution parameter.

[0201] Step 1204: Parse the bitstream based on the second probability distribution parameter to obtain the first image feature of the image.

[0202] After obtaining the probability distribution parameter of the first feature point within the second probability distribution parameter, the decoder parses the bitstream based on the probability distribution parameter of the first feature point within the second probability distribution parameter to obtain the image features of the first feature point within the first image feature. Optionally, the decoder parses the bitstream through entropy decoding to obtain the image features of each feature point in the first image feature. Note that in implementations where encoding and decoding are not performed via a context network, the decoder may parse multiple feature points in parallel to obtain the first image feature. In implementations where encoding and decoding are performed via a context network, each time the decoder obtains the probability distribution parameter of at least one feature point within the second probability distribution parameter, the decoder parses the bitstream to obtain the image features of at least one feature point within the first image feature, and obtains the first image features of the image until the image features of all feature points within the first image feature are obtained through parsing.

[0203] Step 1205: Reconstruct the image by dequantizing the first image features based on the first quantization step.

[0204] In this embodiment of the present application, after parsing the bitstream to obtain a first image feature, the decoder dequantizes the first image feature based on the first quantization step to obtain a third image feature of the image, and reconstructs the image based on the third image feature. Optionally, the decoder inputs the third image feature into a decoder network to reconstruct the image. The decoding process performed in the decoder network is the reverse process of feature extraction performed in the image feature extraction network. Note that the third image feature is consistent with the third image feature in the encoding process, and both of these third image features are dequantized image features.

[0205] Optionally, in some other embodiments, the decoder dequantizes the first image feature based on the fourth quantization step to obtain a fourth image feature of the image, and reconstructs the image based on the fourth image feature. The fourth quantization step may deviate from the first quantization step.

[0206] The decoding process described above will be explained again below with reference to Figures 6 to 10.

[0207] In the decoding process of the encoding and decoding method shown in Figure 6, the bitstream is parsed to obtain the hyperprior feature^z, which is then dequantized based on quantization step q2 to obtain the hyperprior feature z. The hyperprior feature z is then input into the probability distribution estimation network to obtain the probability distribution parameters μ and σ for each feature element in the feature map y. The probability distribution parameters μ and σ are quantized based on quantization step q1 to obtain the feature map y s The probability distribution parameter μ of each feature element in s and σ s The bitstream is obtained as a feature map ^y s The probability distribution parameter μ of each feature element in s and σ s Parsed based on, feature map ^y s This is obtained. Finally, the feature map ^y s The data is then input into a decoder (Dec) network, where the image is reconstructed.

[0208] In the decoding process of the encoding and decoding method shown in Figure 7, the bitstream is first parsed to obtain the hyperplier feature^z of each of the multiple feature points. The hyperplier feature^z of each feature point is then dequantized based on the quantization step q2 to obtain the hyperplier feature z of each feature point. Next, the hyperplier feature z of each feature point is input to the hyperdecoder network to obtain the plier feature φ of each feature point. Then, for the first feature point to be decoded, the surrounding feature points of the first feature point are determined from the decoded feature points. Feature map^y s The feature values ​​of the peripheral feature points in feature map ^y are dequantized based on the quantization step q1 to obtain the feature values ​​of the peripheral feature points in feature map ^y. That is, the peripheral features of the first feature point are obtained. The feature values ​​of the peripheral feature points in feature map ^y are input into the context network to obtain the context features of the first feature point. The context features and plier features of the first feature point are input into the probability distribution estimation network to obtain the probability distribution estimation parameters μ and σ of the first feature point. The probability distribution estimation parameters μ and σ of the first feature point are quantized based on the first quantization step to obtain the probability distribution estimation parameters μ s and σ s The bitstream is obtained. The probability distribution estimation parameter μ of the first feature point. s and σ s Parsed based on, feature map ^y s The image features of the first feature point in are obtained. Feature map ^y s When the image features of all feature points within are obtained by parsing the bitstream, the feature map ^y s Next, the feature map ^y s Each feature element within is dequantized based on the quantization step q1 to obtain the feature map ^y. Finally, the feature map ^y is input to the decoder network to reconstruct the image.

[0209] The decoding process of the encoding and decoding method shown in Figure 8 is the same as the decoding process of the encoding and decoding method shown in Figure 6. The decoding process of the encoding and decoding method shown in Figure 9 is the same as the decoding process of the encoding and decoding method shown in Figure 7. The difference between the decoding process of the encoding and decoding method shown in Figure 10 and the decoding processes in Figures 7 and 9 is that a quantization operation is applied after the pryor feature φ is obtained via the hyperdecoder network, specifically, the pryor feature φ is quantized based on the quantization step q3 and the pryor feature φ s The key point is that this yields context features and plier features φ of each feature point. s This is input into the probability distribution estimation network to obtain the first probability distribution parameters μ and σ.

[0210] In conclusion, in this embodiment of the present application, in order to obtain the probability distribution parameters of quantized image features, a first probability distribution parameter is determined via a probability distribution estimation network during the decoding process, where the first probability distribution parameter represents the probability distribution of the unquantized image features. Then, based on a first quantization step (i.e., a quantization step for quantizing the image features), the first probability distribution parameter is quantized to obtain a second probability distribution parameter representing the probability distribution of the quantized image features. In this solution, it is found that only the probability distribution estimation network for determining the probability distribution parameters of unquantized image features needs to be trained. Even in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and unaffected by the quantization step. In other words, the input numerical range of the probability distribution estimation network does not change with the bitrate. Therefore, training the probability distribution estimation network is not very difficult, network training is stable, and a probability distribution estimation network with good performance can be trained. This helps to improve encoding and decoding performance.

[0211] Figure 13 is a flowchart of another decoding method according to one embodiment of the present invention. This method is applied to the decoder side. Refer to Figure 13. This method includes the following steps.

[0212] Step 1301: Parse the bitstream to obtain the first hyperplier feature of the image to be decoded.

[0213] Please note that the specific implementation process for step 1301 is the same as the specific implementation process for step 1201 in the embodiment shown in Figure 12. For details, please refer to the relevant explanation in step 1201. Details will not be explained again here.

[0214] Step 1302: Determine the second probability distribution parameters based on the first hyperplier features via the second probability distribution estimation network. Here, the network parameters of the second probability distribution estimation network are obtained based on the network parameters of the first probability distribution estimation network and the first quantization step, and the first probability distribution estimation network is used to determine the probability distribution of the unquantized image features.

[0215] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets. Based on this, the weights and offsets of the last convolutional layer in the second probability distribution estimation network are obtained based on the weights and offsets of the last convolutional layer in the first probability distribution estimation network and a first quantization step. Optionally, the second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by the first quantization step. Alternatively, in some embodiments, the second probability distribution estimation network is obtained by adjusting the network parameters of the last layer in the first probability distribution estimation network in a binary leftward or rightward displacement manner such that the adjusted network parameters are equal to the network parameters obtained before adjustment multiplied by the first quantization step.

[0216] In this embodiment of the present application, the first hyperplier feature includes hyperplier features of a plurality of feature points, and the second probability distribution parameter includes probability distribution parameters of a plurality of feature points. The decoder determines the probability distribution parameter of each feature point in the second probability distribution parameter based on the hyperplier features of each feature point in the first hyperplier feature, via the second probability distribution estimation network.

[0217] Note that in implementations where encoding and decoding are not performed via a context network, the decoder can decode multiple feature points in parallel. In implementations where encoding and decoding are performed via a context network, the decoder cannot decode multiple feature points simultaneously.

[0218] In addition, as explained above, the first hyperplier feature can be either a quantized or unquantized hyperplier feature. Based on this, in implementations where encoding and decoding are not performed via a context network and the first hyperplier feature is a quantized hyperplier feature, the decoder dequantizes the first hyperplier feature based on a third quantization step to obtain a second hyperplier feature, and inputs the second hyperplier feature into a second probability distribution estimation network to obtain the second probability distribution parameter. In implementations where encoding and decoding are not performed via a context network and the first hyperplier feature is an unquantized hyperplier feature, the decoder inputs the first hyperplier feature into a second probability distribution estimation network to obtain the second probability distribution parameter.

[0219] In implementations that perform encoding and decoding via a context network, the first feature point is assumed to be one of several feature points. In this case, the decoder performs the following actions on the first feature point to determine the probability distribution parameter of the first feature point in the second probability distribution parameter: determine the context feature of the first feature point based on the image feature of the decoded feature point in the first image feature; determine the first plier feature of the first feature point based on the hyperplier feature of the first feature point in the first hyperplier feature; and determine the probability distribution parameter of the first feature point in the second probability distribution parameter based on the first plier feature and the context feature of the first feature point via the second probability distribution estimation network.

[0220] In an implementation where the first hyperplier feature is a quantized hyperplier feature, the decoder, based on the third quantization step, dequantizes the first hyperplier feature obtained through parsing to obtain a second hyperplier feature, and inputs the second hyperplier feature into the hyperdecoder network to obtain the first plier feature for each of the multiple feature points. In an implementation where the first hyperplier feature is a quantized hyperplier feature, the decoder inputs the first hyperplier feature obtained through parsing into the hyperdecoder network in order to obtain the first plier feature for each of the multiple feature points.

[0221] Optionally, the implementation process for the decoder to determine the context features of the first feature point based on the image features of the decoded feature point in the first image feature is as follows: The decoder determines the surrounding feature points of the first feature point from the decoded feature points, and dequantizes the image features of the surrounding feature points in the first image feature based on the first quantization step to obtain the surrounding features of the first feature point. Then, the decoder inputs the surrounding features of the first feature point into the context network to obtain the context features of the first feature point. Correspondingly, the implementation process for the decoder to determine the probability distribution parameter of the first feature point in the second probability distribution parameter based on the first plier feature and the context features of the first feature point via the second probability distribution estimation network is as follows: The decoder inputs the first plier feature and the context features of the first feature point into the second probability distribution estimation network to obtain the probability distribution parameter of the first feature point in the second probability distribution parameter. In other words, the decoder extracts contextual features from the inversely quantized image features and determines a second probability distribution parameter based on the contextual features and the first prior features. The peripheral feature points of the first feature point include one or more feature points in the neighborhood of the first feature point.

[0222] Optionally, an alternative implementation process in which the decoder determines the context features of the first feature point based on the image features of the decoded feature point in the first image feature is as follows: The decoder determines the surrounding feature points of the first feature point from the decoded feature point, inputs the image features of the surrounding feature points in the first image feature into the context network, and obtains the context features of the first feature point. Correspondingly, an implementation process in which the decoder determines the probability distribution parameter of the first feature point in the second probability distribution parameter based on the first plier feature and the context features of the first feature point via a second probability distribution estimation network is as follows: The decoder quantizes the first plier feature of the first feature point based on a second quantization step to obtain the second plier feature of the first feature point, inputs the second plier feature and the context features of the first feature point into the second probability distribution estimation network, and obtains the probability distribution parameter of the first feature point in the second probability distribution parameter. In other words, the decoder extracts context features from the quantized image features, then obtains second prior features by applying a quantization operation to the first prior features, and determines the first probability distribution parameters based on the second prior features and the context features. The second quantization step is either the same as or different from the first quantization step.

[0223] In implementations where encoding and decoding are performed via a context network, it should be noted that the implementation in which the decoder determines the first plier feature of the first feature point is the same as the relevant part in the embodiments described above. Details will not be explained again here. In addition, the image features of the decoded feature points in the first image feature are obtained by decoding the bitstream according to steps 1302 to 1304. Specifically, in implementations where encoding and decoding are performed via a context network, the decoder parses the bitstream according to steps 1302 to 1304 to sequentially obtain the image features of all feature points in the first image feature, and the decoder can decode at least one feature point each time. Furthermore, the first image feature is an image feature obtained by quantizing the second image feature of the image based on a first quantization step, the quantization operation is performed in the encoding process, and the second image feature is an image feature obtained in the encoding process.

[0224] Step 1303: Parse the bitstream based on the second probability distribution parameter to obtain the first image feature of the image.

[0225] After obtaining the probability distribution parameter of the first feature point within the second probability distribution parameter, the decoder parses the bitstream based on the probability distribution parameter of the first feature point within the second probability distribution parameter to obtain the image feature of the first feature point within the first image feature. Optionally, the decoder parses the bitstream through entropy decoding to obtain the image feature of each feature point in the first image feature. Note that in implementations where encoding and decoding are not performed via a context network, the decoder may parse multiple feature points in parallel to obtain the first image feature. In implementations where encoding and decoding are performed via a context network, each time the decoder obtains the probability distribution parameter of at least one feature point within the second probability distribution parameter, the decoder parses the bitstream to obtain the image feature of at least one feature point within the first image feature to obtain the first image feature of the image, until the image features of all feature points within the first image feature have been obtained through parsing.

[0226] Note that the first image feature acquired by the decoder is consistent with the first image feature acquired by the encoder, and the first image feature acquired by the encoder is an image feature obtained by quantizing the second image feature of the image based on the first quantization step.

[0227] Step 1304: Reconstruct the image by dequantizing the first image features based on the first quantization step.

[0228] In this embodiment of the present application, after parsing the bitstream to obtain a first image feature, the decoder dequantizes the first image feature based on the first quantization step to obtain a third image feature of the image, and reconstructs the image based on the third image feature. Optionally, the decoder inputs the third image feature into a decoder network to reconstruct the image. The decoding process performed in the decoder network is the reverse process of feature extraction performed in the image feature extraction network. Note that the third image feature is consistent with the third image feature in the encoding process, and both of these third image features are dequantized image features.

[0229] The probability distribution estimation network in the encoding and decoding procedures shown in Figures 6 to 10 will be referred to as the first probability distribution estimation network. In this case, the first probability distribution estimation network in Figures 6 to 10 is replaced by the second probability distribution estimation network, and after the quantization operation on the probability distribution parameters is removed, a flowchart of the encoding and decoding method consistent with that in the embodiment of Figure 13 is obtained.

[0230] In conclusion, in the decoding process of this embodiment of the present application, the second probability distribution parameter is obtained directly via a second probability distribution estimation network. The second probability distribution estimation network is obtained by processing the network parameters in the first probability distribution estimation network based on the first quantization step, and the first probability distribution estimation network is used to determine the probability distribution parameter of the unquantized image features. In this solution, it is found that only the first probability distribution estimation network (for determining the probability distribution parameter of the unquantized image features) needs to be obtained through training. Even in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and unaffected by the quantization step. In other words, the input numerical range of the first probability distribution estimation network does not change with the bitrate. Therefore, training the first probability distribution estimation network is not very difficult, the network training is stable, and a first probability distribution estimation network with good performance can be trained. This helps to improve encoding and decoding performance.

[0231] In addition, after the encoding and decoding methods provided in the embodiments of the present application have been validated through multiple experiments, it is concluded that this solution can achieve improved encoding and decoding performance for YUV format images with three components, namely Y, U, and V. From the aforementioned embodiment in Figure 4, it can be seen that, according to the encoding and decoding methods provided in this solution, a probability distribution estimation is performed once on unquantized image features to obtain a first probability distribution parameter, and then a second probability distribution parameter can be estimated at different bitrates (corresponding to different first quantization steps). It is not necessary to perform probability estimation for each bitrate. It can be seen that this solution reduces the complexity of the probability estimation operation and facilitates rate-distortion optimization (RDO) of feature maps. The probability estimation is unified for different bitrates, which facilitates training of the probability distribution estimation network.

[0232] Figure 14 shows the structure of an encoding device 1400 according to one embodiment of the present invention. The encoding device 1400 may be implemented as part of or as part of an encoder-side device by using software, hardware, or a combination thereof. The encoder-side device may be any of the encoder-side devices shown in Figures 1 to 3. Refer to Figure 14. The device 1400 includes a first decision module 1401, a second decision module 1402, a first encoding module 1403, a probability estimation module 1404, a quantization module 1405, and a second encoding module 1406.

[0233] The first decision module 1401 is configured to determine a first image feature and a second image feature of the image to be encoded, the first image feature being obtained by quantizing the second image feature based on a first quantization step.

[0234] The second decision module 1402 is configured to determine the first hyper-prior feature of the second image feature.

[0235] The first encoding module 1403 is configured to encode the first hyper-prior feature into a bitstream.

[0236] The probability estimation module 1404 is configured to determine the first probability distribution parameter based on the first hyper-prior feature via a probability distribution estimation network.

[0237] The quantization module 1405 is configured to quantize the first probability distribution parameter based on the first quantization step to obtain a second probability distribution parameter.

[0238] The second encoding module 1406 is configured to encode the first image feature into a bitstream based on the second probability distribution parameter.

[0239] Optionally, the second decision module 1402: includes a first hyper-encoder sub-module configured to input the second image feature into a hyper-encoder network to obtain the first hyper-prior feature.

[0240] Optionally, the second decision module 1402: includes an inverse quantization sub-module configured to inverse-quantize the first image feature based on the first quantization step to obtain a third image feature of the image; and a second hyper-encoder sub-module configured to input the third image feature into a hyper-encoder network to obtain the first hyper-prior feature. and.

[0241] Optionally, the probability estimation module 1404: A context submodule configured to input the aforementioned third image feature of the aforementioned image into a context network to obtain a context feature of the aforementioned third image feature, wherein the aforementioned third image feature is an image feature obtained by dequantizing the aforementioned first image feature based on the aforementioned first quantization step; A first determination submodule configured to determine a first plier feature based on the first hyperplier feature; A first probability estimation submodule configured to input the first plier feature and the context feature into the probability distribution estimation network to obtain the first probability distribution parameter, and Includes.

[0242] Optionally, the probability estimation module 1404 is: A context submodule configured to input the aforementioned first image feature into a context network to obtain the context features of the aforementioned first image feature; A second determination submodule configured to determine a first plier feature based on the first hyperplier feature; A quantization submodule configured to obtain a second plier feature by quantizing the first plier feature based on a second quantization step; A second probability estimation submodule is configured to input the second pryor feature and the context feature into the probability distribution estimation network to obtain the first probability distribution parameter, and Includes.

[0243] In this embodiment of the present application, in order to obtain probability distribution parameters of quantized image features, in the encoding process, a first probability distribution parameter is determined via a probability distribution estimation network based on the hyperplier features of the unquantized image features, and the first probability distribution parameter represents the probability distribution of the unquantized image features. Then, based on a first quantization step (i.e., a quantization step for quantizing the image features), the first probability distribution parameter is quantized to obtain a second probability distribution parameter that represents the probability distribution of the quantized image features. In this solution, it is found that only the probability distribution estimation network for determining the probability distribution parameters of the unquantized image features needs to be trained. Even in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and unaffected by the quantization step. In other words, the input numerical range of the probability distribution estimation network does not change with the bitrate. Therefore, training the probability distribution estimation network is not very difficult, the network training is stable, and a probability distribution estimation network with good performance can be trained. This helps to improve encoding and decoding performance.

[0244] It should be noted that during encoding by the encoding device provided in the above-described embodiments, the division into functional modules described above is used only as an illustrative example. In actual applications, the functions described above may be assigned to different functional modules and implemented on a case-by-case basis. Specifically, the internal structure of the device is divided into different functional modules to implement all or some of the functions described above. In addition, embodiments of the encoding device and encoding method provided in the above-described embodiments belong to the same concept. For details regarding the specific implementation process of the encoding device, please refer to the embodiments of the method. Details are again not described herein.

[0245] Figure 15 shows the structure of a decoding device 1500 according to one embodiment of the present invention. The encoding device 1500 may be implemented as part of or as part of a decoder-side device by using software, hardware, or a combination thereof. The decoding device may be any decoder-side shown in Figures 1 to 3. Refer to Figure 15. The device 1500 includes a first parse module 1501, a probability estimation module 1502, a quantization module 1503, a second parse module 1504, and a reconstruction module 1505.

[0246] The first parsing module 1501 is configured to parse the bitstream and obtain a first hyperplier feature of the image to be decoded.

[0247] The probability estimation module 1502 is configured to determine a first probability distribution parameter based on a first hyperplier feature via a probability distribution estimation network, where the first probability distribution parameter represents the probability distribution of the unquantized image features of the image.

[0248] The quantization module 1503 is configured to quantize a first probability distribution parameter based on a first quantization step to obtain a second probability distribution parameter.

[0249] The second parsing module 1504 is configured to parse the bitstream based on a second probability distribution parameter to obtain the first image features of the image.

[0250] The reconstruction module 1505 is configured to reconstruct the image by dequantizing the first image features based on the first quantization step.

[0251] Optionally, the first image feature is an image feature obtained by quantizing the second image feature of the image based on the first quantization step.

[0252] Optionally, the reconfiguration module 1505 is: An inverse quantization sub-module configured to inverse-quantize the first image feature based on the first quantization step to obtain a third image feature of the image; A reconstruction sub-module configured to reconstruct the image based on the third image feature and comprising.

[0253] Optionally, the first probability distribution parameter includes probability distribution parameters of a plurality of feature points, and the first hyper-prior feature includes hyper-prior features of a plurality of feature points. The probability estimation module 1502 includes a context sub-module, a first decision sub-module, and a probability estimation sub-module.

[0254] For a first feature point, the probability distribution parameter of the first feature point is determined via a context sub-module, a first decision sub-module, and a probability estimation sub-module, and the first feature point is any one of the plurality of feature points.

[0255] The context sub-module is configured to determine context features of the first feature point based on the image features of the decoded feature points in the first image feature.

[0256] The first decision sub-module is configured to determine first prior features of the first feature point based on the hyper-prior features of the first feature point.

[0257] The probability estimation sub-module is configured to determine the probability distribution parameter of the first feature point based on the first prior features of the first feature point and the context features of the first feature point via a probability distribution estimation network.

[0258] Optionally, the context sub-module: determines peripheral feature points of the first feature point from the decoded feature points; Based on the first quantization step, the image features of the peripheral feature points in the first image feature are dequantized to obtain the peripheral features of the first feature point; The surrounding features of the first feature point are input into the context network to obtain the context features of the first feature point. It is configured in such a way, Determining the probability distribution parameters of the first feature point via the probability distribution estimation network, based on the first plier feature of the first feature point and the context feature of the first feature point, is as follows: This includes inputting the first plier feature and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameters of the first feature point.

[0259] Optionally, the context submodule is: From the decoded feature points, the surrounding feature points of the first feature point are determined; The image features of the surrounding feature points in the first image feature are input into the context network to obtain the context features of the first feature point. It is configured in such a way, Determining the probability distribution parameters of the first feature point via the probability distribution estimation network, based on the first plier feature of the first feature point and the context feature of the first feature point, is as follows: Based on the second quantization step, the first plier feature of the first feature point is quantized to obtain the second plier feature of the first feature point; This includes inputting the second plier feature of the first feature point and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameters of the first feature point.

[0260] In this embodiment of the present application, in order to obtain the probability distribution parameters of the quantized image features, a first probability distribution parameter is determined via a probability distribution estimation network during the decoding process. Here, the first probability distribution parameter represents the probability distribution of the unquantized image features. Then, based on a first quantization step (i.e., a quantization step for quantizing the image features), the first probability distribution parameter is quantized to obtain a second probability distribution parameter that represents the probability distribution of the quantized image features. In this solution, it is found that only the probability distribution estimation network for determining the probability distribution parameters of the unquantized image features needs to be trained. Even in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and unaffected by the quantization step. Therefore, training the probability distribution estimation network based on the unquantized image features is not very difficult, the network training is stable, and a probability distribution estimation network with good performance can be trained. This helps to improve encoding and decoding performance.

[0261] It should be noted that the division into functional modules described above during decoding by the decoding device provided in the above embodiments is used only as an illustrative example. In actual applications, the above functions may be assigned to different functional modules and implemented on a case-by-case basis. Specifically, the internal structure of the device is divided into different functional modules to implement all or some of the above functions. In addition, embodiments of the decoding device and decoding method provided in the above embodiments belong to the same concept. For details regarding the specific implementation process of the decoding device, please refer to the embodiments of the method. Details are again not described herein.

[0262] Figure 16 shows the structure of an encoding device 1600 according to one embodiment of the present invention. The encoding device 1600 can be implemented as part of or as part of an encoder-side device by using software, hardware, or a combination thereof. The encoder-side device may be any of the encoder-side devices shown in Figures 1 to 3. Refer to Figure 16. The device 1600 includes a first decision module 1601, a second decision module 1602, a first encoding module 1603, a probability estimation module 1604, and a second encoding module 1605.

[0263] The first decision module 1601 is configured to determine a first image feature and a second image feature of the image to be encoded, the first image feature being obtained by quantizing the second image feature based on a first quantization step.

[0264] The second decision module 1602 is configured to determine the first hyperplier feature of the second image feature.

[0265] The first encoding module 1603 is configured to encode the first hyperplier feature into a bitstream.

[0266] The probability estimation module 1604 is configured to determine second probability distribution parameters based on first hyperplier features via a second probability distribution estimation network, the network parameters of the second probability distribution estimation network are obtained based on the network parameters of the first probability distribution estimation network and a first quantization step, and the first probability distribution estimation network is used to determine the probability distribution of unquantized image features.

[0267] The second encoding module 1605 is configured to encode the first image features into a bitstream based on a second probability distribution parameter.

[0268] Optionally, a second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by a first quantization step.

[0269] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets.

[0270] Optionally, the second decision module 1602 is: The system includes a first hyperencoder submodule configured to input the second image feature into a hyperencoder network to acquire the first hyperplier feature.

[0271] Optionally, the second decision module 1602 is: An inverse quantization submodule configured to inversely quantize the first image feature based on the first quantization step to obtain a third image feature of the image; A second hyperencoder submodule configured to input the third image feature into a hyperencoder network to acquire the first hyperplier feature, and Includes.

[0272] In the encoding process of this embodiment of the present application, the hyperplier features of the unquantized image features are also determined, but the second probability distribution parameters are subsequently obtained directly via a second probability distribution estimation network. The second probability distribution estimation network is obtained by processing the network parameters within the first probability distribution estimation network based on the first quantization step. In this solution, it is found that only the first probability distribution estimation network (for determining the probability distribution parameters of the unquantized image features) needs to be trained. Even in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and unaffected by the quantization step. Therefore, training the first probability distribution estimation network is not very difficult, the network training is stable, and a first probability distribution estimation network with good performance can be trained. This helps to improve encoding and decoding performance.

[0273] It should be noted that during encoding by the encoding device provided in the above-described embodiments, the division into functional modules described above is used only as an illustrative example. In actual applications, the functions described above may be assigned to different functional modules and implemented on a case-by-case basis. Specifically, the internal structure of the device is divided into different functional modules to implement all or some of the functions described above. In addition, embodiments of the encoding device and encoding method provided in the above-described embodiments belong to the same concept. For details regarding the specific implementation process of the encoding device, please refer to the embodiments of the method. Details are again not described herein.

[0274] Figure 17 shows the structure of a decoding device 1700 according to one embodiment of the present invention. The decoding device 1700 can be implemented as part of or as part of a decoder-side device by using software, hardware, or a combination thereof. The decoder-side device may be any of the decoder-sides shown in Figures 1 to 3. Refer to Figure 17. The device 1700 includes a first parse module 1701, a probability estimation module 1702, a second parse module 1703, and a reconstruction module 1704.

[0275] The first parsing module 1701 is configured to parse the bitstream and obtain a first hyperplier feature of the image to be decoded.

[0276] The probability estimation module 1702 is configured to determine second probability distribution parameters based on first hyperplier features via a second probability distribution estimation network, the network parameters of the second probability distribution estimation network are obtained based on the network parameters of the first probability distribution estimation network and a first quantization step, and the first probability distribution estimation network is used to determine the probability distribution of unquantized image features.

[0277] The second parsing module 1703 is configured to parse the bitstream based on a second probability distribution parameter to obtain the first image features of the image.

[0278] The reconstruction module 1704 is configured to reconstruct the image by dequantizing the first image features based on the first quantization step.

[0279] Optionally, a second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by a first quantization step.

[0280] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets.

[0281] Optionally, the first image feature is an image feature obtained by quantizing the second image feature of the image based on the first quantization step.

[0282] Optionally, the reconfiguration module 1704 is: An inverse quantization submodule configured to inversely quantize the first image feature based on the first quantization step to obtain a third image feature of the image; A reconstruction submodule configured to reconstruct the image based on the third image feature described above, Includes.

[0283] In the decoding process of this embodiment of the present application, the second probability distribution parameter is obtained directly via a second probability distribution estimation network. The second probability distribution estimation network is obtained by processing the network parameters in the first probability distribution estimation network based on the first quantization step, and the first probability distribution estimation network is used to determine the probability distribution parameter of the unquantized image features. In this solution, it is found that only the first probability distribution estimation network (for determining the probability distribution parameter of the unquantized image features) needs to be trained. Even in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and unaffected by the quantization step. In other words, the input numerical range of the first probability distribution estimation network does not change with the bitstream. Therefore, training the first probability distribution estimation network is not difficult, the network training is stable, and a first probability distribution estimation network with good performance can be trained. This helps to improve encoding and decoding performance.

[0284] It should be noted that the division into functional modules described above during decoding by the decoding device provided in the above embodiments is used only as an illustrative example. In actual applications, the above functions may be assigned to different functional modules and implemented on a case-by-case basis. Specifically, the internal structure of the device is divided into different functional modules to implement all or some of the above functions. In addition, embodiments of the decoding device and decoding method provided in the above embodiments belong to the same concept. For details regarding the specific implementation process of the decoding device, please refer to the embodiments of the method. Details are again not described herein.

[0285] Figure 18 is a block diagram of an encoding and decoding device 1800 according to one embodiment of the present application. The encoding and decoding device 1800 includes a processor 1801, a memory 1802, and a bus system 1803. The processor 1801 and the memory 1802 are connected through the bus system 1803. The memory 1802 is configured to store instructions. The processor 1801 is configured to execute instructions stored in the memory 1802 to perform the encoding or decoding method described in the embodiment of the present application. For the sake of avoiding repetition, further details are not described here again.

[0286] In this embodiment of the present application, the processor 1801 may be a central processing unit (CPU), or it may be another general-purpose processor, DSP, ASIC, FPGA, or another programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0287] Memory 1802 may include a ROM device or a RAM device. Any other suitable type of storage device may also be used as memory 1802. Memory 1802 may include code and data 18021 accessed by processor 1801 via bus 1803. Memory 1802 may further include an operating system 18023 and an application 18022. Application 18022 includes at least one program that allows processor 1801 to perform the encoding or decoding method described in embodiments of the present application. For example, application 18022 may include applications 1 to N and further include an encoding or decoding application (abbreviated as a coding application) for performing the encoding or decoding method described in embodiments of the present application.

[0288] In addition to the data bus, bus system 1803 may further include power buses, control buses, status signal buses, and so on. However, for clarity, the various types of buses in the diagram are shown as bus system 1803.

[0289] Optionally, the encoding and decoding device 1800 may further include one or more output devices, such as a display 1804. In one example, the display 1804 may be a touch-sensitive display that combines a display with a touch-sensitive unit capable of operating to sense touch input. The display 1804 may be connected to the processor 1801 via a bus 1803.

[0290] It should be noted that the encoding and decoding device 1800 may perform the encoding method or the decoding method according to the embodiments of the present invention.

[0291] Those skilled in the art will understand that the functions described with reference to the various exemplary logical blocks, modules, and algorithmic steps disclosed and described herein may be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, the functions described with reference to the exemplary logical blocks, modules, and steps may be stored as one or more instructions or codes in a computer-readable medium, or transmitted through a computer-readable medium, and executed by a hardware-based processing unit. The computer-readable medium may include computer-readable storage media corresponding to tangible media such as data storage media, or may include any communication media that facilitates the transmission of computer programs from one place to another (for example, according to a communication protocol). Thus, the computer-readable medium may generally correspond to (1) non-temporary tangible computer-readable storage media, or (2) communication media such as signals or carriers. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the technology described herein. A computer program product may include computer-readable media.

[0292] Such computer-readable storage media may include, but are not limited to, computer-readable storage media, RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other media that can be used to store program code required in the form of instructions or data structures and that can be accessed by a computer. In addition, any connection is appropriately called a computer-readable medium. For example, if instructions are transmitted from a website, server, or another remote source via coaxial cable, optical fiber, twisted pair, digital subscriber line (DSL), or wireless technology such as infrared, radio waves, or microwaves, then such coaxial cable, optical fiber, twisted pair, DSL, or wireless technology such as infrared, radio waves, or microwaves are included in the definition of a medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals, or other temporary media, and actually mean non-temporary tangible storage media. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, DVDs, and Blu-ray discs. A disk typically reproduces data magnetically, while a disc reproduces data optically through a laser. Combinations of the aforementioned items should also be included within the scope of computer-readable media.

[0293] Instructions may be executed by one or more processors, such as digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or equivalent integrated circuits or discrete logic circuits. Therefore, the term “processor” as used herein may refer to the aforementioned structures or any other structures applicable to implementations of the techniques described herein. Furthermore, in some respects, the functions described with reference to the exemplary logic blocks, modules, and steps described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a composite codec. In addition, the techniques may be fully implemented in one or more circuits or logic elements. For example, the various exemplary logic blocks, units, and modules within encoder 100 and decoder 200 may be understood as corresponding circuit devices or logic elements.

[0294] The technology in embodiments of this application may be implemented in a variety of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chipsets). Various components, modules, or units are described in embodiments of this application to highlight the functional aspects of apparatus configured to perform the disclosed technology, but they do not necessarily have to be implemented by different hardware units. In practice, as described above, the various units may be combined with appropriate software and / or firmware into a codec hardware unit, or they may be provided by interoperable hardware units (including one or more processors as described above).

[0295] In other words, all or part of the embodiments described above may be implemented using software, hardware, firmware, or any combination thereof. When software is used to implement an embodiment, all or part of the embodiment may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer instructions are loaded into a computer and executed, all or part of the procedures or functions according to the embodiments of the present application are generated. The computer may be a general-purpose computer, a dedicated computer, a computer network, or another programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted by wired means (e.g., coaxial cable, fiber optic cable, or digital subscriber line (DSL)) or wireless means (e.g., infrared, radio waves, or microwaves) from one website, computer, server, or data center to another website, computer, server, or data center. The computer-readable storage medium may be any available medium accessible by a computer, or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, or magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), semiconductor media (e.g., solid-state disks (SSDs)), etc. It should be noted that the computer-readable storage medium referred to in the embodiments of this application may be a non-volatile storage medium, i.e., a non-temporary storage medium.

[0296] It should be understood that “at least one” as used herein refers to one or more, and “multiple” refers to two or more. In the description of embodiments of this application, “ / ” means “or” unless otherwise specified. For example, A / B may represent A or B. In this application, “and / or” describes only the relationship between related objects and indicates that three relationships may exist. For example, A and / or B may represent the following three cases: A exists only, both A and B exist, or B exists only. In addition, in order to clearly describe the technical solutions in embodiments of this application, terms such as “first” and “second” are used in embodiments of this application to distinguish the same or similar items that provide essentially the same function or purpose. Those skilled in the art will understand that terms such as “first” and “second” do not limit the number or order of execution, and that terms such as “first” and “second” do not indicate a clear distinction.

[0297] Information (including, but not limited to, user device information and user personal information), data (including, but not limited to, data used for analysis, stored data, displayed data, etc.) and signals in embodiments of this application should be used under the authorization of the user or the full authorization of all parties, and it should be noted that the capture, use, and processing of the relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, images and videos in embodiments of this application are obtained under full authorization.

[0298] The foregoing description is merely an illustrative embodiment of the present application and is not intended to limit it. Any modification, equivalent substitution, or improvement made without departing from the spirit and principles of the present application should fall within the scope of protection.

Claims

1. An encoding method, wherein the encoding method is: The steps include determining the unquantized image features of the image to be encoded; The steps include determining the hyperplier features of the aforementioned unquantized image features; A step of determining a first probability distribution parameter based on the hyperplier feature via a probability distribution estimation network, wherein the first probability distribution parameter represents the probability distribution of the unquantized image feature; A step of quantizing the unquantized image features based on a first quantization step to obtain quantized image features, and a step of quantizing the first probability distribution parameter based on the first quantization step to obtain a second probability distribution parameter; The process includes the steps of encoding the hyperplier features into a bitstream based on a pre-configured third probability distribution parameter, and encoding the quantized image features into the bitstream based on a second probability distribution parameter, method.

2. Determining the hyperplier features of the aforementioned unquantized image features is: This includes inputting the aforementioned unquantized image features into a hyperencoder network to obtain the aforementioned hyperplier features, The method according to claim 1.

3. Determining the hyperplier features of the aforementioned unquantized image features is: Based on the first quantization step, the quantized image features are dequantized to obtain the dequantized image features; This includes inputting the inversely quantized image features into a hyperencoder network to obtain the hyperplier features, The method according to claim 1.

4. Determining the first probability distribution parameter based on the hyperplier features via the probability distribution estimation network is: The steps include: inputting the inversely quantized image features into a context network to obtain context features of the inversely quantized image features; The steps include: determining a first plier feature based on the hyperplier feature; The process includes the steps of inputting the first plier feature and the context feature into the probability distribution estimation network to obtain the first probability distribution parameter, The method according to claim 3.

5. Determining the first probability distribution parameter based on the hyperplier features via the probability distribution estimation network is: The quantized image features are input into a context network to obtain the context features of the quantized image features; Based on the hyperplier feature, the first plier feature is determined; Based on the second quantization step, the first plier feature is quantized to obtain the second plier feature; This includes inputting the second plier feature and the context feature into the probability distribution estimation network to obtain the first probability distribution parameter, The method according to any one of claims 1 to 2.

6. One or more processors; A computer-readable storage medium coupled to the processor and storing a program for execution by the processor An encoding device having, the program, when executed by the processor, enables the encoding device to perform the method according to any one of claims 1 to 5. Encoding device.

7. An encoding device having a processing circuit configured to perform the method described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program that, when executed on a computer or processor, enables the computer or processor to perform the method described in any one of claims 1 to 5.

9. A computer-readable storage medium having a bitstream obtained according to the method described in any one of claims 1 to 5.

Images