Encoding method and apparatus, decoding method and apparatus, device, storage medium, and computer program product

Abstract

This application discloses an encoding method and apparatus, a decoding method and apparatus, a device, a storage medium, and a computer program product, which belong to the field of encoding and decoding technology. When probability estimation is performed in the encoding and decoding process, the probability distribution of the unquantized image feature is estimated based on the hyper-prior feature of the unquantized image feature through a first probability distribution estimation network, and then the probability distribution of the quantized image feature is obtained through quantization. Alternatively, the probability distribution of the quantized image feature is directly estimated based on the hyper-prior feature of the unquantized image feature through a second probability distribution estimation network (a network obtained by performing a simple process on the network parameters of the first probability distribution estimation network). Even in a multi-bitrate scenario, the numerical range of the unquantized image feature is not affected by the quantization step and is stable. Therefore, it is not very difficult to train the first probability distribution estimation network based on the unquantized image feature, and a network with stable and good performance can be obtained through training. This helps to improve the encoding and decoding performance.

Description

[Technical Field]

[0001] Book The application relates to the field of encoding and decoding technologies, and in particular to encoding methods and apparatuses, decoding methods and apparatuses, devices, storage media, and computer program products. [Background technology]

[0002] Image compression technology can realize the effective transmission and storage of image information, and the types and data amounts of image information are increasing. current It plays an important role in the media era. Image compression technology includes image encoding and decoding. The encoding and decoding performance reflects the image quality and should be taken into consideration in image compression technology. Important element.

[0003] In the encoding process in the related art, an image feature y of an image is extracted through an image feature extraction network, and the image feature y is quantized based on a quantization step q to obtain an image feature ys. The image feature ys is input to a hyperencoder network to determine a hyperprior feature zs, and the hyperprior feature zs is encoded into a bitstream through entropy encoding. Entropy decoding is performed on the hyperprior feature zs in the bitstream to obtain a hyperprior feature zs', and a probability distribution parameter of the image feature ys is obtained through a probability distribution estimation network based on the hyperprior feature zs'. Based on the probability distribution parameter of the image feature ys, the image feature ys is encoded into a bitstream through entropy encoding. The decoding process is a step-by-step process of the encoding process. is symmetrical to Most of the image compression involves quantization operations. Through The quantization operation has a significant impact on the encoding and decoding performance.

[0004] However, the quantization operation in the encoding and decoding process needs to match the bit rate. In a multi-bit rate scenario, it is usually necessary to use different quantization steps in the encoding and decoding process to match different bit rates. However, different quantization steps result in large differences in the numerical range of the image feature ys obtained through quantization. In order to obtain a probability distribution estimation network for estimating the probability distribution parameters of the image feature ys at different bit rates through training, the probability distribution estimation network is trained by using image features ys with different numerical ranges. Acquired through However, the numerical range of the image feature ys varies greatly at different bit rates, and the probability distribution estimation network Regarding Training is difficult , non It is difficult to obtain a probability distribution estimation network that is stable and has good performance through training, which results in the encoding and decoding performance being affected. Summary of the Invention [Problem to be solved by the invention]

[0005] The present application provides an encoding method and apparatus, a decoding method and apparatus, a device, a storage medium, and a computer program product. Regarding This simplifies the training process, resulting in stable network training and a network with good performance. Acquired through This improves encoding and decoding performance. [Means for solving the problem]

[0006] According to a first aspect, the present application provides an encoding method, the method comprising: The method includes a step of determining a first image feature and a second image feature of an image to be encoded, wherein the first image feature is an image feature obtained by quantizing the second image feature based on a first quantization step; a step of determining a first hyperprior feature of the second image feature; a step of encoding the first hyperprior feature into a bitstream; a step of determining a first probability distribution parameter based on the first hyperprior feature via a probability distribution estimation network; a step of quantizing the first probability distribution parameter based on the first quantization step to obtain a second probability distribution parameter; and a step of encoding the first image feature into the bitstream based on the second probability distribution parameter.

[0007] It can be seen that when probability estimation is performed in the encoding process, the probability distribution of unquantized image features is estimated based on the hyperprior features of the unquantized image features through the probability distribution estimation network, and then the probability distribution of quantized image features is obtained through quantization. Even in multi-bitrate scenarios, the numerical range of unquantized image features is not affected by the quantization step and is stable. Therefore, by using unquantized image features, the probability distribution estimation network Regarding It is not very difficult to train, and a network with good performance is trained. Acquired through This helps to improve encoding and decoding performance.

[0008] Optionally, the step of determining the first hyperprior feature of the second image feature includes: inputting the second image feature into a hyperencoder network to obtain the first hyperprior feature, i.e., the encoder side inputs the unquantized image feature into a hyperencoder network to obtain the first hyperprior feature of the unquantized image feature.

[0009] Optionally, the step of determining the first hyperprior feature of the second image feature includes the steps of: dequantizing the first image feature based on the first quantization step to obtain a third image feature of the image; and inputting the third image feature into a hyperencoder network to obtain the first hyperprior feature, i.e., the encoder side inputs the dequantized image feature into the hyperencoder network, and the obtained first hyperprior feature of the dequantized image feature is regarded as the first hyperprior feature of the unquantized image feature.

[0010] Optionally, the step of determining the first probability distribution parameter based on the first hyperprior feature through a probability distribution estimation network includes the steps of: inputting a third image feature of the image into a context network to obtain a context feature of the third image feature, where the third image feature is an image feature obtained by dequantizing the first image feature based on the first quantization step; determining a first prior feature based on the first hyperprior feature; and inputting the first prior feature and the context feature into the probability distribution estimation network to obtain the first probability distribution parameter. That is, the encoder side extracts a context feature from the dequantized image feature and determines the first probability distribution parameter based on the context feature and the first prior feature. This helps to improve the accuracy of probability estimation.

[0011] Optionally, the step of determining the first probability distribution parameter through a probability distribution estimation network based on the first hyper-prior feature includes: inputting the first image feature into a context network to obtain a context feature for the first image feature; determining a first prior feature based on the first hyper-prior feature; quantizing the first prior feature based on a second quantization step to obtain a second prior feature; and inputting the second prior feature and the context feature into the probability distribution estimation network to obtain the first probability distribution parameter. That is, the encoder side may alternatively extract a context feature from the dequantized image feature, and then obtain a second prior feature by applying a quantization operation to the first prior feature, and determine the first probability distribution parameter based on the second prior feature and the context feature. This can also improve the accuracy of probability estimation to some extent.

[0012] Optionally, the first quantization step is obtained based on the bit rate of the image through a gain network, and the gain network is used to determine the quantization step corresponding to each of a plurality of types of bit rates. That is, the quantization step is obtained through network learning, and the quantization step can better match the bit rate, which is helpful in improving encoding and decoding performance.

[0013] According to a second aspect, there is provided a decoding method, the method comprising: The method includes parsing the bitstream to obtain first hyperprior features of the image to be decoded; determining first probability distribution parameters representing a probability distribution of unquantized image features of the image based on the first hyperprior features via a probability distribution estimation network; quantizing the first probability distribution parameters based on a first quantization step to obtain second probability distribution parameters; parsing the bitstream based on the second probability distribution parameters to obtain first image features of the image; and dequantizing the first image features based on the first quantization step to reconstruct the image.

[0014] It can be seen that when probability estimation is performed in the decoding process, the probability distribution of the unquantized image features is estimated based on the hyperprior features of the unquantized image features through a probability distribution estimation network, and then the probability distribution of the quantized image features is obtained through quantization. Even in multi-bitrate scenarios, the numerical range of the unquantized image features is not affected by the quantization step and is stable. Therefore, by using the unquantized image features, the probability distribution estimation network Regarding It is not very difficult to train, and a network with good performance is trained. Acquired through This helps to improve encoding and decoding performance.

[0015] Optionally, the first image feature is an image feature obtained by quantizing a second image feature of the image based on a first quantization step.

[0016] Optionally, the step of dequantizing the first image feature based on the first quantization step to reconstruct the image includes: dequantizing the first image feature based on the first quantization step to obtain a third image feature of the image, and reconstructing the image based on the third image feature.

[0017] Optionally, the first probability distribution parameter includes a probability distribution parameter of a plurality of feature points, and the first hyperprior feature includes a hyperprior feature of the plurality of feature points. The step of determining the first probability distribution parameter through a probability distribution estimation network based on the first hyperprior feature includes: Regarding The method includes performing the following operations, where the first feature point is any one of a plurality of feature points: determining a context feature of the first feature point based on the image feature of the decoded feature point in the first image feature; determining a first prior feature of the first feature point based on the hyper-prior feature of the first feature point; and determining a probability distribution parameter of the first feature point based on the first prior feature of the first feature point and the context feature of the first feature point via a probability distribution estimation network, i.e., determining a probability distribution parameter of the first feature point in the first probability distribution parameter. That is, the probability distribution is estimated based on the context feature. This helps to improve the accuracy of the probability estimation.

[0018] Optionally, the step of determining a context feature of the first feature point based on image features of the decoded feature point in the first image feature includes: determining peripheral feature points of the first feature point from the decoded feature point; dequantizing the image features of the peripheral feature points in the first image feature based on the first quantization step to obtain peripheral features of the first feature point; and inputting the peripheral features of the first feature point into a context network to obtain the context feature of the first feature point. The step of determining a probability distribution parameter of the first feature point via a probability distribution estimation network based on a first prior feature of the first feature point and a context feature of the first feature point includes inputting the first prior feature of the first feature point and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameter of the first feature point. That is, the decoder extracts a context feature from the dequantized image feature and determines the first probability distribution parameter based on the context feature and the first prior feature.

[0019] Optionally, the step of determining a context feature of the first feature point based on image features of the decoded feature point in the first image feature includes: determining peripheral feature points of the first feature point from the decoded feature point; and inputting the image features of the peripheral feature points in the first image feature into a context network to obtain the context feature of the first feature point. The step of determining a probability distribution parameter of the first feature point through the probability distribution estimation network based on a first prior feature of the first feature point and the context feature of the first feature point includes: quantizing the first prior feature of the first feature point based on a second quantization step to obtain a second prior feature of the first feature point; and inputting the second prior feature of the first feature point and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameter of the first feature point. That is, the decoder side extracts context features from the quantized image features, then obtains second prior features by applying a quantization operation to the first prior features, and determines first probability distribution parameters based on the second prior features and the context features.

[0020] According to a third aspect, there is provided an encoding method, the method comprising: The method includes the steps of: determining a first image feature and a second image feature of an image to be encoded, wherein the first image feature is an image feature obtained by quantizing the second image feature based on a first quantization step; determining a first hyperprior feature of the second image feature; encoding the first hyperprior feature into a bitstream; determining second probability distribution parameters based on the first hyperprior feature via a second probability distribution estimation network, wherein network parameters of the second probability distribution estimation network are obtained based on network parameters of the first probability distribution estimation network and the first quantization step, and the first probability distribution estimation network is used to determine a probability distribution of an unquantized image feature; and encoding the first image feature into the bitstream based on the second probability distribution parameters.

[0021] It can be seen that in the encoding process, the hyperprior features of the unquantized image features are also determined, but the second probability distribution parameters are subsequently obtained directly through a second probability distribution estimation network. The second probability distribution estimation network is obtained by processing the network parameters in the first probability distribution estimation network based on the first quantization step. In this solution, only the first probability distribution estimation network is trained. Acquired through Even in multi-bitrate scenarios, the numerical range of unquantized image features is stable and is not affected by the quantization step. Therefore, the first probability distribution estimation network Regarding It's not that hard to train , cheap The first probability distribution estimation network with good performance is trained. Acquired through This helps to improve encoding and decoding performance.

[0022] Optionally, the first probability distribution estimation network is the probability distribution estimation network of the first aspect or the second aspect.

[0023] Optionally, the second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by the first quantization step.

[0024] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets.

[0025] Optionally, the step of determining a first hyperprior feature of the second image feature includes inputting the second image feature into a hyperencoder network to obtain the first hyperprior feature, i.e., the encoder side inputs the unquantized image feature into the hyperencoder network to obtain the first hyperprior feature of the unquantized image feature.

[0026] Optionally, the step of determining a first hyperprior feature of the second image feature includes the steps of: dequantizing the first image feature based on the first quantization step to obtain a third image feature of the image; and inputting the third image feature into a hyperencoder network to obtain the first hyperprior feature, i.e., the encoder side inputs the dequantized image feature into the hyperencoder network, and the obtained first hyperprior feature of the dequantized image feature is regarded as the first hyperprior feature of the unquantized image feature.

[0027] Optionally, the step of determining the second probability distribution parameters based on the first hyperprior features via the second probability distribution estimation network includes the steps of: inputting a third image feature of the image into a context network to obtain a context feature of the third image feature, the third image feature being an image feature obtained by dequantizing the first image feature based on the first quantization step; determining a first prior feature based on the first hyperprior feature; and inputting the first prior feature and the context feature into the second probability distribution estimation network to obtain the second probability distribution parameters. That is, the encoder extracts a context feature from the dequantized image feature and determines the first probability distribution parameters based on the context feature and the first prior feature. This helps to improve the accuracy of the probability estimation.

[0028] Optionally, the step of determining the second probability distribution parameters based on the first hyper-prior features via the second probability distribution estimation network includes: inputting the first image features into a context network to obtain context features for the first image features; determining first prior features based on the first hyper-prior features; quantizing the first prior features based on a second quantization step to obtain second prior features; and inputting the second prior features and the context features into the second probability distribution estimation network to obtain second probability distribution parameters. That is, the encoder side may alternatively extract context features from the dequantized image features, and then apply a quantization operation to the first prior features to obtain second prior features, and determine the first probability distribution parameters based on the second prior features and the context features. This can also improve the accuracy of probability estimation to some extent.

[0029] According to a fourth aspect, there is provided a decoding method, the method comprising: The method includes the steps of: parsing the bitstream to obtain first hyperprior features of an image to be decoded; determining second probability distribution parameters based on the first hyperprior features via a second probability distribution estimation network, wherein network parameters of the second probability distribution estimation network are obtained based on network parameters of the first probability distribution estimation network and a first quantization step, and the first probability distribution estimation network is used to determine a probability distribution of unquantized image features; parsing the bitstream based on the second probability distribution parameters to obtain first image features of the image; and dequantizing the first image features based on the first quantization step to reconstruct the image.

[0030] In the decoding process, the second probability distribution parameters are directly obtained through the second probability distribution estimation network, which is obtained by processing the network parameters in the first probability distribution estimation network based on the first quantization step. In this solution, only the first probability distribution estimation network is trained. Acquired through Even in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and is not affected by the quantization step. In other words, the input numerical range of the first probability distribution estimation network does not change with the bitstream. Therefore, the first probability distribution estimation network Regarding It's not that hard to train , cheap The first probability distribution estimation network with good performance is trained. Acquired through This helps to improve encoding and decoding performance.

[0031] Optionally, the first probability distribution estimation network is the probability distribution estimation network of the first aspect or the second aspect.

[0032] Optionally, the second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by the first quantization step.

[0033] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets.

[0034] Optionally, the first image feature is an image feature obtained by quantizing a second image feature of the image based on a first quantization step.

[0035] Optionally, the step of inverse quantizing the first image feature based on the first quantization step to reconstruct the image includes: inverse quantizing the first image feature based on the first quantization step to obtain a third image feature of the image; and reconstructing the image based on the third image feature.

[0036] Optionally, the second probability distribution parameters include probability distribution parameters of a plurality of feature points, and the first hyperprior features include hyperprior features of a plurality of feature points. The step of determining the second probability distribution parameters based on the first hyperprior features via a second probability distribution estimation network includes: RegardingThe method includes performing the following operations, where the first feature point is any one of the plurality of feature points: determining a context feature of the first feature point based on the image feature of the decoded feature point in the first image feature; determining a first prior feature of the first feature point based on the hyper-prior feature of the first feature point; and determining a probability distribution parameter of the first feature point based on the first prior feature of the first feature point and the context feature of the first feature point via a second probability distribution estimation network, i.e., determining the probability distribution parameter of the first feature point in the second probability distribution parameter. That is, the probability distribution is estimated based on the context feature. This helps to improve the accuracy of the probability estimation.

[0037] Optionally, the step of determining a context feature of the first feature point based on the image feature of the decoded feature point in the first image feature includes: determining peripheral feature points of the first feature point from the decoded feature point; dequantizing the image feature of the peripheral feature point in the first image feature based on the first quantization step to obtain peripheral features of the first feature point; and inputting the peripheral feature of the first feature point into a context network to obtain the context feature of the first feature point. The step of determining a probability distribution parameter of the first feature point based on the first prior feature of the first feature point and the context feature of the first feature point via a second probability distribution estimation network includes: inputting the first prior feature of the first feature point and the context feature of the first feature point into the second probability distribution estimation network to obtain the probability distribution parameter of the first feature point. That is, the decoder side extracts a context feature from the dequantized image feature and determines a second probability distribution parameter based on the context feature and the first prior feature.

[0038] Optionally, the step of determining a context feature of the first feature point based on image features of the decoded feature point in the first image feature includes: determining peripheral feature points of the first feature point from the decoded feature point; and inputting the image features of the peripheral feature points in the first image feature into a context network to obtain a context feature of the first feature point. The step of determining a probability distribution parameter of the first feature point based on a first prior feature of the first feature point and a context feature of the first feature point via a second probability distribution estimation network includes: quantizing the first prior feature of the first feature point based on a second quantization step to obtain a second prior feature of the first feature point; and inputting the second prior feature of the first feature point and the context feature of the first feature point into the second probability distribution estimation network to obtain a probability distribution parameter of the first feature point. That is, the decoder side extracts context features from the quantized image features, then obtains second prior features by applying a quantization operation to the first prior features, and determines second probability distribution parameters based on the second prior features and the context features.

[0039] According to a fifth aspect, there is provided an encoding device having functionality for implementing the behavior of the encoding method of the first aspect, the encoding device including one or more modules configured to implement the encoding method provided in the first aspect.

[0040] That is, an encoding apparatus is provided, the apparatus comprising: a first determination module configured to determine a first image feature and a second image feature of an image to be encoded, the first image feature being an image feature obtained by quantizing the second image feature based on a first quantization step; a second determination module configured to determine a first hyperprior feature of the second image feature; a first encoding module configured to encode the first hyperprior feature into a bitstream; a probability estimation module configured to determine, via a probability distribution estimation network, first probability distribution parameters based on the first hyperprior features; a quantization module configured to quantize the first probability distribution parameters based on the first quantization step to obtain second probability distribution parameters; a second encoding module configured to encode the first image feature into the bitstream based on the second probability distribution parameters; Includes.

[0041] Optionally, the second decision module: and a first hyperencoder sub-module configured to input the second image features into a hyperencoder network to obtain the first hyperprior features.

[0042] Optionally, the second decision module: an inverse quantization sub-module configured to inverse quantize the first image feature based on the first quantization step to obtain a third image feature of the image; a second hyperencoder sub-module configured to input the third image features into a hyperencoder network to obtain the first hyperprior features; Includes.

[0043] Optionally, the probability estimation module: a context sub-module configured to input the third image feature of the image into a context network to obtain a context feature of the third image feature, the third image feature being an image feature obtained by dequantizing the first image feature based on the first quantization step; a first determining sub-module configured to determine a first prior feature based on the first hyperprior feature; a first probability estimation sub-module configured to input the first prior features and the context features into the probability distribution estimation network to obtain the first probability distribution parameters; Includes.

[0044] Optionally, the probability estimation module: a context submodule configured to input the first image feature into a context network to obtain a context feature for the first image feature; a second determining sub-module configured to determine a first prior feature based on the first hyperprior feature; a quantization submodule configured to quantize the first prior feature based on a second quantization step to obtain a second prior feature; a second probability estimation sub-module configured to input the second prior features and the context features into the probability distribution estimation network to obtain the first probability distribution parameters; and Includes.

[0045] According to a sixth aspect, there is provided a decoding device, the decoding device having functionality for implementing the behavior of the decoding method of the second aspect, the decoding device including one or more modules configured to implement the decoding method provided in the second aspect.

[0046] That is, a decoding device is provided, the device comprising: a first parsing module configured to parse the bitstream to obtain first hyperprior features of an image to be decoded; a probability estimation module configured to determine a first probability distribution parameter based on the first hyperprior feature via a probability distribution estimation network, the first probability distribution parameter representing a probability distribution of unquantized image features of the image; a quantization module configured to quantize the first probability distribution parameters based on a first quantization step to obtain second probability distribution parameters; a second parsing module configured to parse the bitstream based on the second probability distribution parameters to obtain first image features of the image; a reconstruction module configured to dequantize the first image feature based on the first quantization step to reconstruct the image; and Includes.

[0047] Optionally, the first image feature is an image feature obtained by quantizing a second image feature of the image based on a first quantization step.

[0048] Optionally, the reconstruction module: an inverse quantization sub-module configured to inverse quantize the first image feature based on the first quantization step to obtain a third image feature of the image; a reconstruction submodule configured to reconstruct the image based on the third image feature; and Includes.

[0049] Optionally, the first probability distribution parameter includes a probability distribution parameter of a plurality of feature points, and the first hyperprior feature includes a hyperprior feature of a plurality of feature points. The probability estimation module includes a context submodule, a first determination submodule, and a probability estimation submodule.

[0050] For a first feature point, a probability distribution parameter of the first feature point is determined via a context submodule, a first determination submodule, and a probability estimation submodule, and the first feature point is any one of the plurality of feature points.

[0051] The context submodule is configured to determine a context feature of the first feature point based on image features of the decoded feature point in the first image feature.

[0052] The first determination sub-module is configured to determine a first prior feature of the first feature point based on the hyper-prior feature of the first feature point.

[0053] The probability estimation submodule is configured to determine, via a probability distribution estimation network, a probability distribution parameter of the first feature point based on the first prior feature of the first feature point and the context feature of the first feature point.

[0054] Optionally, the context submodule: determining neighboring feature points of the first feature point from the decoded feature points; Inversely quantizing the image features of the peripheral feature points in the first image feature based on the first quantization step to obtain peripheral features of the first feature points; configured to input the peripheral features of the first feature point into a context network to obtain the context feature of the first feature point; Determining the probability distribution parameters of the first feature points via the probability distribution estimation network based on the first prior feature of the first feature points and the context feature of the first feature points includes: inputting the first prior feature of the first feature point and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameter of the first feature point.

[0055] Optionally, the context submodule: determining neighboring feature points of the first feature point from the decoded feature points; The image features of the peripheral feature points in the first image feature are input to a context network to obtain the context feature of the first feature point; Determining the probability distribution parameters of the first feature points via the probability distribution estimation network based on the first prior feature of the first feature points and the context feature of the first feature points includes: quantizing the first prior feature of the first feature point based on a second quantization step to obtain a second prior feature of the first feature point; inputting the second prior feature of the first feature point and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameter of the first feature point.

[0056] According to a seventh aspect, there is provided an encoding device having a function for implementing the operations of the encoding method of the third aspect, the encoding device including one or more modules configured to implement the encoding method provided in the third aspect.

[0057] That is, an encoding apparatus is provided, the apparatus comprising: a first determination module configured to determine a first image feature and a second image feature of an image to be encoded, the first image feature being an image feature obtained by quantizing the second image feature based on a first quantization step; a second determination module configured to determine a first hyperprior feature of the second image feature; a first encoding module configured to encode the first hyperprior feature into a bitstream; a probability estimation module configured to determine second probability distribution parameters based on the first hyperprior features via a second probability distribution estimation network, wherein the network parameters of the second probability distribution estimation network are obtained based on the network parameters of the first probability distribution estimation network and the first quantization step, and the first probability distribution estimation network is used to determine a probability distribution of unquantized image features; a second encoding module configured to encode the first image feature into the bitstream based on the second probability distribution parameters; Includes.

[0058] Optionally, the second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by the first quantization step.

[0059] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets.

[0060] Optionally, the second decision module: and a first hyperencoder sub-module configured to input the second image features into a hyperencoder network to obtain the first hyperprior features.

[0061] Optionally, the second decision module: an inverse quantization sub-module configured to inverse quantize the first image feature based on the first quantization step to obtain a third image feature of the image; a second hyperencoder sub-module configured to input the third image features into a hyperencoder network to obtain the first hyperprior features; Includes.

[0062] According to an eighth aspect, there is provided a decoding device, the decoding device having a function of implementing the operations of the decoding method of the fourth aspect, the decoding device including one or more modules configured to implement the decoding method provided in the fourth aspect.

[0063] That is, a decoding device is provided, the device comprising: a first parsing module configured to parse the bitstream to obtain first hyperprior features of an image to be decoded; a probability estimation module configured to determine second probability distribution parameters based on the first hyperprior features via a second probability distribution estimation network, wherein the network parameters of the second probability distribution estimation network are obtained based on the network parameters of the first probability distribution estimation network and a first quantization step, and the first probability distribution estimation network is used to determine a probability distribution of unquantized image features; a second parsing module configured to parse the bitstream based on the second probability distribution parameters to obtain first image features of the image; a reconstruction module configured to dequantize the first image feature based on the first quantization step to reconstruct the image; and Includes.

[0064] Optionally, the second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by the first quantization step.

[0065] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets.

[0066] Optionally, the first image feature is an image feature obtained by quantizing a second image feature of the image based on a first quantization step.

[0067] Optionally, the reconstruction module: an inverse quantization sub-module configured to inverse quantize the first image feature based on the first quantization step to obtain a third image feature of the image; a reconstruction submodule configured to reconstruct the image based on the third image feature; and Includes.

[0068] According to a ninth aspect, there is provided an encoder-side device. The encoder-side device includes a processor and a memory. The memory is configured to store a program for executing the encoding method provided in the first aspect and / or the third aspect and to store data for implementing the encoding method provided in the first aspect and / or the third aspect. The processor is configured to execute the program stored in the memory. The encoder-side device may further include a communication bus, the communication bus being configured to establish a connection between the processor and the memory.

[0069] According to a tenth aspect, there is provided a decoder-side device. The decoder-side device includes a processor and a memory. The memory is configured to store a program for executing the decoding method provided in the second aspect and / or the fourth aspect, and to store data for implementing the decoding method provided in the second aspect and / or the fourth aspect. The processor is configured to execute the program stored in the memory. The decoder-side device may further include a communication bus, the communication bus being configured to establish a connection between the processor and the memory.

[0070] According to an eleventh aspect, there is provided a computer-readable storage medium storing instructions that, when executed on a computer, enable the computer to perform the encoding method according to the first or third aspect, or the decoding method according to the second or fourth aspect.

[0071] According to a twelfth aspect, there is provided a computer program product comprising instructions which, when executed on a computer, enable the computer to perform the encoding method according to the first or third aspect, or the decoding method according to the second or fourth aspect.

[0072] The technical effects obtained in the fifth to twelfth aspects are the same as those obtained through the corresponding technical means in the first to fourth aspects, and the details will not be described again in this specification.

[0073] The technical solution provided in this application can bring at least the following beneficial effects:

[0074] To obtain the probability distribution parameters of the quantized image features, in the encoding process of the solution, first probability distribution parameters are determined based on the hyperprior features of the unquantized image features through a probability distribution estimation network, where the first probability distribution parameters represent the probability distribution of the unquantized image features. Then, the first probability distribution parameters are quantized based on a first quantization step (i.e., a quantization step for quantizing the image features) to obtain second probability distribution parameters for representing the probability distribution of the quantized image features. In the encoding process of another solution, the hyperprior features of the unquantized image features are also determined, but the second probability distribution parameters are subsequently obtained directly through a second probability distribution estimation network. The second probability distribution estimation network is obtained by processing the network parameters in the first probability distribution estimation network based on the first quantization step, where the first probability distribution network parameters are the probability distribution estimation network in the first solution. The decoding process is performed after the encoding process. is symmetrical to It can be seen that in these two solutions, only the first probability distribution estimation network (for determining the probability distribution parameters of the unquantized image features) needs to be trained. Even in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and is not affected by the quantization step. In other words, the input numerical range of the first probability distribution estimation network does not change with the bit rate. Therefore, the first probability distribution estimation network Regarding It's not that hard to train , cheap The first probability distribution estimation network with good performance is trained. Acquired through This helps to improve encoding and decoding performance. [Brief explanation of the drawings]

[0075] [Figure 1] FIG. 1 is a diagram of an implementation environment according to an embodiment of the present application.

[0076] [Figure 2] FIG. 2 is a diagram of another implementation environment according to an embodiment of the present application.

[0077] [Figure 3] FIG. 1 is a diagram of yet another implementation environment according to an embodiment of the present application.

[0078] [Figure 4] 1 is a flowchart of an encoding method according to an embodiment of the present application;

[0079] [Figure 5] FIG. 1 is a diagram of the structure of an image feature extraction network according to an embodiment of the present application.

[0080] [Figure 6] 1 is a flowchart of an encoding and decoding method according to an embodiment of the present application;

[0081] [Figure 7] 4 is a flowchart of another encoding and decoding method according to an embodiment of the present application;

[0082] [Figure 8] 10 is a flowchart of yet another encoding and decoding method according to an embodiment of the present application;

[0083] [Figure 9] 10 is a flowchart of yet another encoding and decoding method according to an embodiment of the present application;

[0084] [Figure 10] 10 is a flowchart of yet another encoding and decoding method according to an embodiment of the present application;

[0085] [Figure 11] 4 is a flowchart of another encoding method according to an embodiment of the present application;

[0086] [Figure 12] 3 is a flowchart of a decoding method according to an embodiment of the present application;

[0087] [Figure 13] 4 is a flowchart of another decoding method according to an embodiment of the present application;

[0088] [Figure 14] 1 is a structural diagram of an encoding device according to an embodiment of the present application;

[0089] [Figure 15] 1 is a structural diagram of a decoding device according to an embodiment of the present application;

[0090] [Figure 16] FIG. 2 is a structural diagram of another encoding device according to an embodiment of the present application;

[0091] [Figure 17] FIG. 10 is a structural diagram of another decoding device according to an embodiment of the present application;

[0092] [Figure 18] 1 is a block diagram of an encoding and decoding device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described. DETAILED DESCRIPTION OF THE INVENTION

[0093] To make the objectives, technical solutions and advantages of the present application clearer, the following further describes the implementation of the present application in detail with reference to the accompanying drawings.

[0094] The system architecture and service scenarios described in the embodiments of the present application are intended to more clearly explain the technical solutions in the embodiments of the present application, and do not constitute limitations on the technical solutions provided in the embodiments of the present application. Those skilled in the art may recognize that with the evolution of system architecture and the emergence of new service scenarios, the technical solutions provided in the embodiments of the present application may also be applicable to similar technical problems.

[0095] Before the encoding and decoding methods provided in the embodiments of the present application are described in detail, the terminology and implementation environment in the embodiments of the present application will be first described.

[0096] For ease of understanding, the terms used in the embodiments of this application will be explained first.

[0097] Bit rate: In image compression, the bit rate is the code length required to encode a pixel. A higher bit rate indicates better image reconstruction quality.

[0098] Convolutional Neural Networks alConvolutional neural network (CNN): A convolutional neural network (CNN) is a feedforward neural network with a deep structure that includes convolution operations. It is one of the representative algorithms of deep learning. CNNs include convolutional layers and may also include activation layers (e.g., rectified linear units (ReLU) and parametric ReLUs (PReLUs)), pooling layers, batch normalization (BN) layers, fully connected layers, etc. Typical CNNs include LeNet, AlexNet, VGGNet, and ResNet. A basic CNN may include a backbone network and a head network. A more complex CNN may include a backbone network, a neck network, and a head network.

[0099] Feature map: A feature map is three-dimensional data output by the convolutional layer, activation layer, pooling layer, and batch normalization layer in a convolutional neural network. The three dimensions are called width, height, and channel, respectively. A feature map contains image features of multiple feature points.

[0100] Backbone network: A backbone network is the first part of a convolutional neural network and is used to extract feature maps at multiple scales from an input image. Generally, a backbone network contains convolutional layers, pooling layers, activation layers, etc., but does not contain a fully connected layer. In general, the feature maps output by layers in the backbone network that are close to the input image have high resolution (width and height) but a small number of channels. Typical backbone networks include, for example, VGG-16, ResNet-50, and ResNeXt-101.

[0101] Head network: The head network is the final part of a convolutional neural network, which is used to process the feature maps to obtain the prediction results output by the neural network. Common head networks include a fully connected layer, a softmax module, etc.

[0102] Neck network: A neck network is an intermediate part of a convolutional neural network, which is used to further integrate the feature maps generated by the head network to obtain new feature maps. A typical neck network is, for example, a fast region convolutional neural network (CNN). r R - The feature pyramid network (FPN) in CNN is

[0103] The implementation environment of the embodiment of the present application will be described below.

[0104] FIG. 1 is a diagram of an implementation environment according to an embodiment of the present application. Please refer to FIG. 1. This implementation environment includes an encoder side 101 and a decoder side 102. The encoder side 101 is used to compress an image according to an encoding method provided in an embodiment of the present application, and the decoder side 102 is used to decode an image according to a decoding method provided in an embodiment of the present application. Optionally, the encoder side 101 includes an encoder, which is configured to compress an image. The decoder side 102 includes a decoder, which is configured to decode an image. When the encoder side 101 and the decoder side 102 are located in the same device, the encoder side 101 communicates with the decoder side 102 through a device internal connection line or a network. When the encoder side 101 and the decoder side 102 are located in different devices, the encoder side 101 communicates with the decoder side 102 through an external connection line. throughor wireless network Through It communicates with a decoder side 102. The encoder side 101 is also called a source device, and the decoder side 102 is also called a destination device.

[0105] FIG. 2 is a diagram of another implementation environment according to an embodiment of the present application. Referring to FIG. 2, the implementation environment includes a source device 10, a destination device 20, a link 30, and a storage device 40. The source device 10 can generate encoded images. Therefore, the source device 10 is also referred to as an image encoding device or an encoder side. The destination device 20 can decode the encoded images generated by the source device 10. Therefore, the destination device 20 is also referred to as an image decoding device or a decoder side. The link 30 can receive the encoded images generated by the source device 10 and transmit the encoded images to the destination device 20. The storage device 40 can receive the encoded images generated by the source device 10 and store the encoded images. In this case, the destination device 20 can obtain the encoded images directly from the storage device 40. Alternatively, the storage device 40 can correspond to a file server or another intermediate storage device that can store the encoded images generated by the source device 10. In this case, the destination device 20 can transmit or download the encoded images stored in the storage device 40 in a streaming manner.

[0106] Each of source device 10 and destination device 20 may include one or more processors and memory coupled to the one or more processors. Memory may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, any other medium usable for storing necessary program code in the form of instructions or data structures and accessible to a computer, etc. For example, each of source device 10 and destination device 20 may include a mobile phone, a smartphone, a personal digital assistant (PDA), a wearable device, a pocket personal computer (PPC), a tablet computer, a smart head unit, a smart television, a smart speaker, a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a telephone handset such as a so-called "smart" phone, a television, a camera, a display device, a digital media player, a video game console, an in-vehicle computer, and similar devices.

[0107] The link 30 may include one or more media or devices capable of transmitting the encoded image from the source device 10 to the destination device 20. In one possible implementation, the link 30 may include one or more communication media capable of enabling the source device 10 to directly transmit the encoded image to the destination device 20 in real time. In this embodiment of the present application, the source device 10 may modulate the encoded image according to a communication standard, which may be a wireless communication protocol, and transmit the modulated image to the destination device 20. The one or more communication media may include wireless communication media and / or wired communication media. For example, the one or more communication media may include a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may be part of a packet-based network. The packet-based network may be a local area network, a wide area network, a global network (e.g., the Internet), etc. The one or more communication media may include a router, a switch, a base station, another device facilitating communication from the source device 10 to the destination device 20, etc. This is not particularly limiting in this embodiment of the present application.

[0108] In one possible implementation, storage device 40 may store received encoded images sent by source device 10, and destination device 20 may retrieve the encoded images directly from storage device 40. In this case, storage device 40 may include any one of several types of distributed or locally accessed data storage media. For example, the any one of several types of distributed or locally accessed data storage media may be a hard disk drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded images.

[0109] In one possible implementation, storage device 40 may correspond to a file server or another intermediate storage device capable of storing encoded images generated by source device 10, and destination device 20 may stream or download the images stored in storage device 40. The file server may be any type of server capable of storing encoded images and transmitting the encoded images to destination device 20. In one possible implementation, the file server may include a network server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, a local disk drive, etc. Destination device 20 may obtain the encoded images through any standard data connection (including an Internet connection). The standard data connection may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL) or cable modem), or a combination of a wireless channel and a wired connection suitable for obtaining encoded images stored on a file server. The transmission of the encoded images from the storage device 40 may be a streaming transmission, a download transmission, or a combination thereof.

[0110] The implementation environment shown in Fig. 2 is merely one possible implementation. Furthermore, the technology in the embodiment of the present application can be applied not only to the source device 10 capable of encoding an image and the destination device 20 capable of decoding the encoded image shown in Fig. 2, but also to other devices capable of encoding an image and other devices capable of decoding the encoded image. This is not particularly limited in the embodiment of the present application.

[0111] 2, source device 10 includes data source 120, encoder 100, and output interface 140. In some embodiments, output interface 140 may include a modulator / demodulator (modem) and / or a transmitter. A transmitter is also referred to as a sender. Data source 120 may include an image capture device (e.g., a camera), an archive containing previously captured images, a feed-in interface for receiving images from an image content provider, and / or a computer graphics system for generating images, or a combination of these image sources.

[0112] Data source 120 may transmit images to encoder 100, which may encode received images transmitted from data source 120 to obtain encoded images. The encoder may transmit the encoded images to an output interface. In some embodiments, source device 10 transmits the encoded images directly to destination device 20 through output interface 140. In another embodiment, the encoded images may alternatively be stored in storage device 40, with destination device 20 subsequently retrieving the encoded images for decoding and / or display.

[0113] In the implementation shown in FIG. 2 , destination device 20 includes input interface 240, decoder 200, and display device 220. In some embodiments, input interface 240 includes a receiver and / or a modem. Input interface 240 may receive encoded images over link 30 and / or from storage device 40 and then transmit the encoded images to decoder 200. Decoder 200 may decode the received encoded images to obtain decoded images. The decoder may transmit the decoded images to display device 220. Display device 220 may be integrated with destination device 20 or may be located external to destination device 20. Generally, display device 220 displays the decoded images. Display device 220 may be any one of several types of display devices. For example, display device 220 may be a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0114] 2, in some aspects, the encoder 100 and the decoder 200 may be integrated with the encoder and decoder, respectively, and may include a suitable multiplexer-demultiplexer (MUX-DEMUX) unit or other hardware and software for encoding both audio and video in a common data stream or separate data streams. In some embodiments, where applicable, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol or another protocol such as the user datagram protocol (UDP).

[0115] Each of the encoder 100 and decoder 200 includes the following circuits: one or more microprocessors, digital signal processors (DSPs), and or , DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), discrete logic, hardware, or any combination thereof. If the technology in the embodiments of the present application is partially implemented in software, the device may store instructions for the software in an appropriate non-volatile computer-readable storage medium and execute the instructions in hardware through one or more processors to implement the technology in the embodiments of the present application. Any one of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered as one or more processors. Each of the encoder 100 and the decoder 200 may be included in one or more encoders or decoders. Any of the encoders or decoders may be integrated as part of a combined encoder / decoder (codec) in a corresponding device.

[0116] In this embodiment of the present application, the encoder 100 may generally be referred to as "signaling" or "sending" some information to another device, e.g., the decoder 200. The terms "signaling" or "sending" may generally refer to the transmission of syntax elements and / or other data used to decode a compressed image. Such transmission may occur in real time or near real time. Alternatively, such communication may occur after a period of time, for example, when the syntax elements in the encoded bitstream were stored to a computer-readable storage medium during encoding. A decoding device may then retrieve the syntax elements any time after the syntax elements were stored to the medium.

[0117] FIG. 3 is a diagram of yet another implementation environment according to an embodiment of the present application. In this implementation environment, the encoding and decoding method provided in the embodiment of the present application is applied to a virtual reality streaming scenario. See FIG. 3. This implementation environment includes an encoder side and a decoder side. The encoder side includes a video acquisition and preprocessing module (also referred to as a preprocessing module), a video encoding module, and a transmitting module. The decoder side includes a receiving module, a bitstream decoding module, and a rendering and display module.

[0118] The collecting module on the encoder side collects video, which includes multiple frames of images to be encoded. The preprocessing module then preprocesses each frame of the image. The video encoding module then encodes each preprocessed frame of the image according to the encoding method provided in the embodiment of the present application to obtain a bitstream. The transmitting module transmits the bitstream to the decoder side via a transport network. The receiving module on the decoder side first receives the bitstream. The decoding module then decodes the bitstream according to the decoding method provided in the embodiment of the present application to obtain image information. The rendering and display module then renders and displays the image information. In addition, after obtaining the bitstream, the encoder side may alternatively store the bitstream.

[0119] It should be noted that the encoding and decoding methods provided in the embodiments of the present application may be applied to multiple scenarios, and the images encoded in various scenarios may all be images included in an image file or all images included in a video file. The encoded images may be images in RGB, YUV444, or YUV420 format, for example. It should be noted that based on the implementation environments shown in Figures 1, 2, and 3, any one of the following encoding methods may be performed by the encoder side, and any one of the following decoding methods may be performed by the decoder side.

[0120] The encoding method provided in the embodiment of the present application will be described below.

[0121] 4 is a flowchart of an encoding method according to an embodiment of the present application. The method is applied on the encoder side. As shown in FIG. 4, the method includes the following steps:

[0122] Step 401: Determine a first image feature and a second image feature of an image to be encoded. The first image feature is an image feature obtained by quantizing the second image feature based on a first quantization step.

[0123] In this embodiment of the present application, the encoder inputs the image to be encoded into an image feature extraction network to obtain second image features of the image, where the second image features are unquantized image features, and the encoder quantizes the second image features based on a first quantization step to obtain first image features, where the first image features are quantized image features.

[0124] It should be noted that each of the first image feature and the second image feature includes image features of multiple feature points, and the image feature of each feature point in the first image feature may be referred to as a first feature value of the corresponding feature point, and the image feature of each feature point in the second image feature may be referred to as a second feature value of the corresponding feature point.

[0125] Optionally, the image feature extraction network is a convolutional neural network, where the first image feature is represented by a first feature map, the second image feature is represented by a second feature map, and each of the first feature map and the second feature map has a plurality of feature points. It should be noted that the image feature extraction network in this embodiment of the present application is obtained through pre-training, and the network structure, training method, etc. of the image feature extraction network are not limited in this embodiment. For example, the image feature extraction network may be a fully connected network or the aforementioned convolutional neural network, and the convolution in the convolutional neural network may be 2D convolution or 3D convolution. In addition, the number of network layers included in the image feature extraction network and the number of nodes in each layer are not limited in this embodiment.

[0126] FIG. 5 is a diagram illustrating the structure of an image feature extraction network according to an embodiment of the present application. Please refer to FIG. 5. The image feature extraction network is a convolutional neural network, which includes four convolutional (Conv) layers and three interleaved and cascaded grasp detection network (GDN) layers. The size of the convolutional kernel in each convolutional layer is 5×5, the number of channels of the output feature map is M, and 2x downsampling is performed on the width and height in each convolutional layer. For example, for an input 16W×16H×3 image, the size of the feature map output by the convolutional neural network is W×H×M. Note that the structure of the convolutional neural network illustrated in FIG. 5 is not intended to limit the embodiment of the present application. For example, the size of the convolutional kernel, the number of channels of the feature map, the downsampling factor, the number of downsamplings, the number of convolutional layers, etc. may all be adjusted.

[0127] Optionally, in this embodiment of the present application, the first quantization step is obtained based on the bit rate of the image through a gain network, and the gain network is adapted to multiple types of bit rates. Separately The first quantization step is used to determine a corresponding quantization step. For example, the encoder side determines a first quality factor based on the bit rate of the image and inputs the first quality factor into the gain network to obtain the first quantization step. It should be noted that different bit rates correspond to different quality factors, and different quantization steps can be obtained through the gain network. Alternatively, a mapping relationship between the bit rate and the quantization step is stored in advance, and the corresponding quantization step is obtained as the first quantization step from the mapping relationship based on the bit rate of the image. Optionally, in some other embodiments, the first quantization step is a first quantization step corresponding to the first quality factor obtained from the mapping relationship between the quality factor and the quantization step after the first quality factor is determined based on the bit rate of the image to be encoded.

[0128] The quality factor may be replaced by a quantization parameter. The above-mentioned implementation process may have multiple quantization processing methods, for example, uniform quantization or scalar quantization. The scalar quantization may further have an offset. Specifically, after a bias processing based on the offset is performed on the data to be quantized (e.g., the second image feature), the scalar quantization is performed based on the quantization step. Optionally, in this embodiment of the present application, the quantization processing performed on the image feature includes quantization and rounding. For example, if the second image feature is represented by a feature map y, the numerical range of the second image feature is in the interval [0,100], the first quantization step is represented by q1, where q1 is 0.5, and the first image feature is represented by a feature map y, s In this case, the encoder quantizes the feature values ​​of each feature point in the feature map y to generate the feature map y s and obtain the feature map y s The feature value of each feature point in is rounded to obtain a feature map ys', i.e., the first image feature. The numerical range of the first image feature is in the interval [0, 50]. As an example, uniform quantization is used. The feature value obtained by quantizing an arbitrary feature value x based on a quantization step q is x' = x * q.

[0129] Optionally, the first quantization steps for quantizing the image features of the feature points may be the same or different. For example, feature points in the same channel use the same first quantization step, or feature values ​​of different channels in the same spatial location use the same first quantization step. It is assumed that the size of the second image feature to be quantized is W×H×M. Under any quality factor i, the first quantization step of the feature point with coordinates (k,j,l) in the second image feature is q i (k,j,l), where q i(k,j,l) may be obtained through learning by a gain network or based on a stored mapping relationship, where k∈[1,W], j∈[1,H], and l∈[1,M]. It should be understood that different quantization parameters QP correspond to different quantization steps q, and there is a one-to-one correspondence between the quantization parameters QP and the quantization steps q. For example, in some standard solutions, the mapping relationship between the quantization parameters and the quantization steps is q=(2 1 / 6 ) QP-4 Of course, another function may alternatively be designed to represent the mapping relationship between QP and q.

[0130] Please note that the following quantization processing aspects are the same as those here. For the following quantization processing aspects, please refer to the aspects here. The details will not be described again below in the embodiments of the present application.

[0131] Step 402: Determine a first hyperprior feature of the second image feature.

[0132] In this embodiment of the present application, step 404 In To later obtain the probability distribution parameters (i.e., the first probability distribution parameters) of the unquantized image features, the encoder side determines a first hyperprior feature of the unquantized image features (e.g., a first hyperprior feature of the second image features) before step 404. There are several implementations in which the encoder side determines the first hyperprior feature of the second image features. Two of the several implementations are described below.

[0133] The first implementation, where the encoder determines the first hyperprior feature for the second image feature, is to input the second image feature into the hyperencoder network to obtain the first hyperprior feature. In other wordsIn the first embodiment, the encoder inputs the unquantized image feature into a hyper-encoder network to obtain a first hyper-prior feature of the unquantized image feature. A second implementation in which the encoder determines a first hyper-prior feature of a second image feature is to dequantize the first image feature based on the first quantization step to obtain a third image feature of the image; and input the third image feature into the hyper-encoder network to obtain the first hyper-prior feature. The first hyper-prior feature may also be considered as the first hyper-prior feature of the third image feature, or the first hyper-prior feature of the second image feature. The second image feature is the image feature obtained before quantization, and the third image feature is the image feature obtained through dequantization. Therefore, although the first image feature and the third image feature are numerically different, the image information represented by the first image feature and the third image feature is essentially equivalent.

[0134] Optionally, the hyper-encoder network outputs a first hyper-prior feature. Alternatively, the hyper-encoder network outputs a second hyper-prior feature. The encoder side quantizes the second hyper-prior feature based on a third quantization step to obtain a first hyper-prior feature, where the first hyper-prior feature is a quantized hyper-prior feature. The third quantization step may be the same as or different from the first quantization step. That is, a quantization operation may also be performed on the hyper-prior feature to compress the hyper-prior feature. Optionally, the hyper-prior feature may also be referred to as side information, which may be understood as further extracting features from the image features.

[0135] It should be noted that each of the first hyperprior feature and the second hyperprior feature includes hyperprior features of multiple feature points. For example, in step 404, the image feature of each feature point in the first image feature is input to a hyperencoder network to obtain hyperprior features of the feature points in the first hyperprior feature. In addition, the hyperencoder network in this embodiment of the present application is obtained through pre-training. The network structure, training method, etc. of the hyperencoder network are not limited in this embodiment of the present application. For example, the hyperencoder network may be a convolutional neural network or a fully connected network. Optionally, the hyperencoder network in this specification may also be referred to as a hyperprior network.

[0136] Step 403: Encode the first hyperprior feature into a bitstream.

[0137] In this embodiment of the present application, the encoder side encodes the first hyperprior feature into the bitstream, so that the decoder side subsequently performs decoding based on the first hyperprior feature.

[0138] Optionally, the encoder side encodes the first hyperprior feature into the bitstream through entropy encoding. For example, the encoder side encodes the first hyperprior feature into the bitstream through entropy encoding based on a specified probability distribution parameter. The specified probability distribution parameter is a probability distribution parameter determined in advance through a probability distribution estimation network. The network structure, training method, etc. of the probability distribution estimation network are not limited in the embodiments of the present application.

[0139] Step 404: Determine a first probability distribution parameter based on the first hyperprior feature via a probability distribution estimation network.

[0140] In this embodiment of the present application, a probability distribution estimation network is used to determine a probability distribution parameter of the unquantized image feature. Based on this, the encoder side determines a first probability distribution parameter based on the first hyperprior feature through the probability distribution estimation network, where the first probability distribution parameter represents the probability distribution of the unquantized image feature (e.g., the second image feature or the third image feature). It should be noted that the probability distribution parameter in this specification may be any parameter for representing the probability distribution of the image feature, such as the mean and variance (or standard deviation) of a Gaussian distribution, the location parameter and scale parameter of a Laplace distribution, or the mean and scale parameter of a logistic distribution, or another model parameter for another example.

[0141] Optionally, in alignment with the decoding process of the decoder side, the encoder side parses the bitstream to obtain first hyperprior features, and determines first probability distribution parameters through a probability distribution estimation network based on the first hyperprior features obtained through parsing.

[0142] It can be seen from the above description that the first hyperprior feature may be a quantized hyperprior feature or an unquantized hyperprior feature. Based on this, in an implementation in which the first hyperprior feature is a quantized hyperprior feature, the encoder side dequantizes the first hyperprior feature based on the third quantization step to obtain a second hyperprior feature, and inputs the second hyperprior feature into a probability distribution estimation network to obtain the first probability distribution parameter. In an implementation in which the first hyperprior feature is an unquantized hyperprior feature, the encoder side inputs the first hyperprior feature into a probability distribution estimation network to obtain the first probability distribution parameter. The probability distribution estimation network may be considered a hyperdecoder network, and the hyperdecoder network is used to determine the probability distribution parameter based on the hyperprior feature.

[0143] In addition to the above-described implementation of determining the first probability distribution parameter, the encoder side may alternatively determine the first probability distribution parameter based on context features to improve the accuracy of the first probability distribution parameter, as described below.

[0144] Optionally, in one implementation, the encoder side inputs a third image feature of the image to a context network to obtain a context feature of the third image feature. The third image feature is an image feature obtained by dequantizing the first image feature based on the first quantization step. The encoder side determines a first prior feature based on the first hyper-prior feature, and inputs the first prior feature and the context feature to a probability distribution estimation network to obtain a first probability distribution parameter. That is, the encoder side extracts a context feature from the dequantized image feature and determines the first probability distribution parameter based on the context feature and the first prior feature.

[0145] In an implementation where the first hyperprior feature is a quantized hyperprior feature, the encoder side parses the bitstream to obtain the first hyperprior feature, dequantizes the first hyperprior feature obtained through parsing based on a third quantization step to obtain a second hyperprior feature, and inputs the second hyperprior feature into a hyperdecoder network to obtain the first prior feature.In an implementation where the first hyperprior feature is a quantized hyperprior feature, the encoder side parses the bitstream to obtain the first hyperprior feature, and inputs the first hyperprior feature obtained through parsing into a hyperdecoder network to obtain the first prior feature.

[0146] Optionally, based on the foregoing description, since the image information represented by the second image feature and the third image feature is basically equivalent, the encoder side may input the second image feature into a context network to obtain the context feature of the second image feature, where the context feature of the second image feature is the context feature of the third image feature.

[0147] It should be noted that the context feature of the third image feature includes the context feature of each feature point in the plurality of feature points, and the first probability distribution parameter includes the probability distribution parameter of each feature point in the plurality of feature points, i.e., the encoder side may determine the context feature of each feature point in the plurality of feature points and the probability distribution parameter of each feature point in parallel.

[0148] In another implementation, the encoder inputs a first image feature into a context network to obtain a context feature for the first image feature, determines a first prior feature based on the first hyperprior feature, quantizes the first prior feature based on a second quantization step to obtain a second prior feature, and inputs the second prior feature and the context feature into a probability distribution estimation network to obtain a first probability distribution parameter. Alternatively, the encoder may extract a context feature from the dequantized image feature, then obtain a second prior feature by applying a quantization operation to the first prior feature, and determine a first probability distribution parameter based on the second prior feature and the context feature. The second quantization step may be the same as or different from the first quantization step.

[0149] Note that in this implementation, the implementation in which the encoder determines the first prior feature is consistent with the related process in the previous implementation, and the details will not be described again here. Additionally, in the implementation in which a probability distribution is determined based on a context feature, a hyperdecoder network is used to determine the prior feature based on the hyperprior feature, and a probability distribution estimation network is used to determine the probability distribution parameters based on the prior feature and the context feature. Both the hyperdecoder network and the probability distribution estimation network in this embodiment of the present application are obtained through pre-training. The network structures, training methods, etc. of the hyperdecoder network and the probability distribution estimation network are not limited in this embodiment of the present application. For example, each of the hyperdecoder network and the probability distribution estimation network may be a convolutional neural network, a recurrent neural network, a fully connected network, etc.

[0150] Optionally, the probability distribution estimation network in this embodiment of the present application is modeled by using a Gaussian model (e.g., a Gaussian single model (GSM) or a Gaussian mixture model (GMM)). Specifically, the feature value of each feature point in the unquantized image feature (e.g., the second image feature or the third image feature) is fitted to a Gaussian single model or a Gaussian mixture model, and a probability distribution estimation network is generated. distribution recommendation Regular The first probability distribution parameters obtained by the network are assumed to include a mean value μ and a standard deviation σ. Optionally, the probability distribution estimation network may alternatively use a Laplace distribution model. Correspondingly, the first probability distribution parameters include a location parameter λ and a scale parameter b. Alternatively, the probability distribution estimation network may alternatively use a logistic distribution model. Correspondingly, the first probability distribution parameters include a mean value μ and a scale parameter S. A Gaussian model is used as an example. The probability distribution function corresponding to the probability distribution of an arbitrary feature point in the first probability distribution parameters is shown in the following equation (1): where x is the second feature value of the feature point:

number

[0151] Step 405: Quantize the first probability distribution parameters based on a first quantization step to obtain second probability distribution parameters.

[0152] After obtaining the first probability distribution parameters of the unquantized image features, the encoder side quantizes the first probability distribution parameters based on a first quantization step to obtain second probability distribution parameters, where the second probability distribution parameters represent the probability distribution of the quantized image features (i.e., the first image features). Based on the first quantization step, the probability distribution parameters of each feature point in the first probability distribution parameters are quantized to obtain the probability distribution parameters of the corresponding feature points in the second probability distribution parameters.

[0153] Ga The Woos model is used as an example. The first quantization step of a feature point with coordinates (k,j,l) is q i (k,j,l), and the probability distribution parameters of the feature points in the first probability distribution parameter are μ(k,j,l) and σ(k,j,l). μ(k,j,l) and σ(k,j,l) are the quantization step q i (k, j, l) and quantized to obtain the probability distribution parameter μ of the feature point in the second probability distribution parameter. S (k,j,l) and σ S (k,j,l) is obtained. If uniform quantization is performed, μ S = μ / q, σ S =σ / q. The probability distribution function corresponding to the probability distribution parameter of an arbitrary feature point in the second probability distribution parameter is shown in the following formula (2), where x is the first feature value of the feature point:

number

[0154] Here, we will explain the principle of step 405. The quantization operation is assumed to be uniform quantization. A Gaussian model is used as an example. The probability distribution function of variable x is shown in the above equation (1). In this case, the probability P1 of variable x in the interval [a2*q, a1*q] is shown in the following equation (3), where q is the quantization step.

number

[0155] The variable x is quantized based on the quantization step q to obtain the quantized variable x'=x / q. In this case, the probability P1 of the variable x in the interval [a2*q, a1*q] is equal to the probability P2 of the variable x' in the interval [a2, a1]. Based on this, the probability distribution function of the variable x' is assumed to be g(x). In this case,

number

number

[0156] Laplace distribution model and logistic distribution Similar to the model, the second probability distribution parameters may be obtained from the first probability distribution parameters by scaling (ie, quantizing) the parameters of the corresponding model.

[0157] Step 406: Encode the first image feature into a bitstream based on the second probability distribution parameters.

[0158] In this embodiment of the present application, after obtaining the second probability distribution parameters, the encoder encodes the first image feature into a bitstream based on the second probability distribution parameters. The encoder encodes the image feature of each feature point in the first image feature into a bitstream based on the probability distribution parameters of each feature point in the second probability distribution parameters. For example, the encoder encodes the first image feature into a bitstream through entropy encoding.

[0159] The above describes the encoding method provided in the embodiment of the present application. Hereinafter, the above content will be described again with reference to FIGS.

[0160] FIG. 6 is a flowchart of an encoding and decoding method according to an embodiment of the present application. Please refer to FIG. 6. In the encoding process, an image to be encoded is input to an encoder (Enc) network to obtain a feature map y to be quantized (i.e., a second image feature). The encoder network is an image feature extraction network. Based on a quantization step q1 (i.e., a first quantization step), quantization (Q) is performed on each feature element in the feature map y to obtain the feature map y. s The feature map y s Rounding (R) is performed on each feature element in the feature map ^y s [^y s (for convenience, it may be written as such; similarly below) to obtain a first image feature. Additionally, the feature map y is input to a hyperencoder (HyEnc) network to obtain a hyperprior feature z. Optionally, the hyperprior feature z is quantized based on a quantization step q2 to obtain a hyperprior feature ^z (i.e., a first hyperprior feature). The quantization step q2 may be the same as or different from the quantization step q1. The hyperprior feature ^z is encoded into a bitstream through entropy encoding (AE2). Then, the hyperprior feature ^z is obtained by parsing the bitstream through entropy decoding (AD2). Optionally, inverse quantization (IQ) is performed on the hyperprior feature ^z based on the quantization step q2 to obtain a first image feature z. The hyperprior feature z is input to a probability distribution estimation network to obtain probability distribution parameters μ and σ (i.e., a first probability distribution parameter). The probability distribution parameters μ and σ are quantized based on the quantization step q1 to generate the feature map ^ys The probability distribution parameter μ of each feature element in S and σ S (i.e., the second probability distribution parameter) is obtained. s The probability distribution parameter μ of each feature element in S and σ S Based on the feature map ^y s is encoded into the bitstream through entropy encoding (AE1).

[0161] FIG. 7 is a flowchart of another encoding and decoding method according to an embodiment of the present application. Please refer to FIG. 7. In the encoding process, an image to be encoded is input to an encoder (Enc) network to obtain a feature map y to be quantized. The encoder network is an image feature extraction network. Quantization (Q) is performed on each feature element in the feature map y based on a quantization step q1 to obtain the feature map y. s The feature map y s Rounding (R) is performed on each feature element in the feature map ^y s , is obtained. The feature map y is then input to a hyperencoder (HyEnc) network to obtain a hyperprior feature z. Optionally, a hyperprior feature ^z (i.e., a first hyperprior feature) is obtained. The hyperprior feature ^z is encoded into a bitstream through entropy encoding (AE2). Then, the hyperprior feature ^z is obtained by parsing the bitstream through entropy decoding (AD2). Optionally, an inverse quantization (IQ) is performed on the hyperprior feature ^z based on a quantization step q2 to obtain a hyperprior feature z. The hyperprior feature z is input to a hyperdecoder (HyDec) network (i.e., a probability distribution estimation network) to obtain a prior feature φ (i.e., a first prior feature). The feature map ^y sis inverse-quantized based on the first quantization step q1 to obtain a feature map ^y (i.e., a third image feature). The feature map ^y is input to a context (Ctx) network to obtain a context feature for each feature point in the feature map ^y. The context feature and prior feature φ of each feature point are input to a probability distribution estimation network to obtain probability distribution parameters μ and σ (i.e., first probability distribution parameters) for each feature element in the feature map y. The probability distribution parameters μ and σ are quantized based on the quantization step q1 to obtain a feature map ^y s The probability distribution parameter μ of each feature element in S and σ S (i.e., the second probability distribution parameter) is obtained. s The probability distribution parameter μ of each feature element in S and σ S Based on the feature map ^y s is encoded into a bitstream through entropy encoding (AE1).

[0162] 8 is a flowchart of yet another encoding and decoding method according to an embodiment of the present application. The difference between FIG. 8 and FIG. 6 is that in the encoding process, the feature map ^y s is dequantized based on the quantization step q1 to obtain the feature map ^y. The feature map ^y is input to the hyperencoder network to obtain the hyperprior feature z.

[0163] 9 is a flowchart of yet another encoding and decoding method according to an embodiment of the present application. The difference between FIG. 9 and FIG. 7 is that in the encoding process, the feature map ^y s is dequantized based on the quantization step q1 to obtain the feature map ^y. The feature map ^y is input to the hyperencoder network to obtain the hyperprior feature z.

[0164] 10 is a flowchart of yet another encoding and decoding method according to an embodiment of the present application. The difference between FIG. 10 and FIG. 7 and FIG. 9 is that in the encoding process, the feature map ^y s is input to the hyperencoder network to obtain the hyperprior feature ^z. Furthermore, after the prior feature φ is obtained through the hyperdecoder network, a quantization operation is applied. Specifically, the prior feature φ is quantized based on the quantization step q3 (i.e., the second quantization step) to obtain the prior feature φ s The quantization step q3 can be the same as or different from the quantization step q1. The context feature and prior feature φ of each feature point are s is input to a probability distribution estimation network to obtain first probability distribution parameters μ and σ.

[0165] In conclusion, this application fruit In an embodiment, to obtain probability distribution parameters of quantized image features, in an encoding process, first probability distribution parameters are determined based on hyperprior features of unquantized image features via a probability distribution estimation network. Here, the first probability distribution parameters represent the probability distribution of the unquantized image features. Then, the first probability distribution parameters are quantized based on a first quantization step (i.e., a quantization step for quantizing the image features) to obtain second probability distribution parameters for representing the probability distribution of the quantized image features. In this solution, only the probability distribution estimation network for determining the probability distribution parameters of the unquantized image features is trained. Acquired through Even in multi-bitrate scenarios, the numerical range of unquantized image features is stable and is not affected by the quantization step. In other words, the input numerical range of the probability distribution estimation network does not change with the bit rate. Therefore, the probability distribution estimation network Regarding It's not that hard to train , cheap The probability distribution estimation network with good performance is trained. Acquired through This helps to improve encoding and decoding performance.

[0166] FIG. 11 is a flowchart of another encoding method according to an embodiment of the present application. This method is applied on the encoder side. Note that the probability distribution estimation network in the embodiment of FIG. 4 is assumed to be called the first probability distribution estimation network. In this case, the difference between the embodiment of FIG. 11 and the embodiment of FIG. 4 is that in the encoding method shown in FIG. 11, the second probability distribution parameters are directly obtained via the second probability distribution estimation network, specifically, the second probability distribution estimation network directly outputs the probability distribution parameters of the quantized image features. The network parameters of the second probability distribution estimation network are obtained based on the network parameters of the first probability distribution estimation network and the first quantization step. In this way, only the first probability distribution estimation network is trained. Acquired through Please refer to Figure 11. This method includes the following steps:

[0167] Step 1101: Determine a first image feature and a second image feature of the image to be encoded. The first image feature is an image feature obtained by quantizing the second image feature based on a first quantization step.

[0168] In this embodiment of the present application, the encoder inputs the image to be encoded into an image feature extraction network to obtain the second image feature of the image. The encoder quantizes the second image feature based on the first quantization step to obtain the first image feature. The specific implementation process is the same as the specific implementation process of step 401 in the embodiment of FIG. 4. For details, please refer to the relevant description in step 401. The details will not be described again here.

[0169] Step 1102: Determine a first hyperprior feature of the second image feature.

[0170] Optionally, in one implementation, the encoder side inputs the second image feature into the hyper-encoder network to obtain the first hyper-prior feature. In another implementation, the encoder side dequantizes the first image feature based on the first quantization step to obtain a third image feature of the image, and inputs the third image feature into the hyper-encoder network to obtain the first hyper-prior feature. The specific implementation process is the same as the specific implementation process of step 402 in the embodiment of FIG. 4. For details, please refer to the relevant description of step 402. The details will not be described again here.

[0171] Step 1103: Encode the first hyperprior feature into a bitstream.

[0172] In this embodiment of the present application, the encoder side encodes the first hyperprior feature into the bitstream, so that the decoder side then performs decoding based on the first hyperprior feature. Optionally, the encoder side encodes the first hyperprior feature into the bitstream through entropy encoding. The specific implementation process is the same as the specific implementation process of step 403 in the embodiment of FIG. 4. For details, please refer to the related description of step 403. The details will not be described again here.

[0173] Step 1104: Determine second probability distribution parameters based on the first hyperprior features via a second probability distribution estimation network, where the network parameters of the second probability distribution estimation network are obtained based on the network parameters of the first probability distribution estimation network and the first quantization step, and the first probability distribution estimation network is used to determine the probability distribution of the unquantized image features.

[0174] Optionally, in alignment with the decoding process of the decoder side, the encoder side parses the bitstream to obtain first hyperprior features, and determines second probability distribution parameters via a second probability distribution estimation network based on the first hyperprior features obtained through parsing.

[0175] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets. Based on this, the last layer in the second probability distribution estimation network is ( Convolutional Layer ) The weights and offsets of the first probability distribution estimation network are ( Convolutional Layer ) and the first quantization step. Optionally, the second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by the first quantization step. Alternatively, in some embodiments, the second probability distribution estimation network adjusts the network parameters of the last layer in the first probability distribution estimation network in a binary leftward or rightward displacement manner so that the adjusted network parameters are equal to the network parameters obtained before adjustment multiplied by the first quantization step. It is obtained by .

[0176] For example, the last layer in the first probability distribution estimation network is a convolutional layer, and both the weight w and offset b of the convolutional layer are multiplied by a first quantization step q1 to obtain the weight w*q1 and offset b*q1 of the last layer in the second probability distribution estimation network. Note that the network layers other than the last layer in the second probability distribution estimation network are the same as the network layers in the first probability distribution estimation network. In other words, the difference between the second probability distribution estimation network and the first probability distribution estimation network lies in the different network parameters of the last layer. In this way, the first probability distribution estimation network may be obtained through training based on unquantized image features. After training for the first probability distribution estimation network is completed, the second probability distribution estimation network may be obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by a first quantization step.

[0177] Similar to the first hyperprior feature in step 404 in the embodiment of FIG. 4 , in this embodiment of the present application, the first hyperprior feature may be a quantized hyperprior feature or an unquantized hyperprior feature. Accordingly, in an implementation where the first hyperprior feature is a quantized hyperprior feature, the encoder side dequantizes the first hyperprior feature based on the third quantization step to obtain a second hyperprior feature, and inputs the second hyperprior feature into a second probability distribution estimation network to obtain second probability distribution parameters. If the first hyperprior feature is an unquantized hyperprior feature, the encoder side inputs the first hyperprior feature into a second probability distribution estimation network to obtain second probability distribution parameters.

[0178] Determine the second probability distribution parameters variousIn addition to the implementation, the encoder side may alternatively determine the second probability distribution parameters based on context features to improve the accuracy of the second probability distribution parameters, as will be described below.

[0179] In one implementation, the encoder side inputs a third image feature of the image to a context network to obtain a context feature of the third image feature. The third image feature is an image feature obtained by dequantizing the first image feature based on the first quantization step. The encoder side determines a first prior feature based on the first hyper-prior feature, and inputs the first prior feature and the context feature to a second probability distribution estimation network to obtain a second probability distribution parameter. That is, the encoder side extracts a context feature from the dequantized image feature and determines the second probability distribution parameter based on the context feature and the first prior feature.

[0180] In another implementation, the encoder inputs a first image feature into a context network to obtain a context feature for the first image feature. The encoder determines a first prior feature based on the first hyper-prior feature and quantizes the first prior feature based on a second quantization step to obtain a second prior feature. The encoder inputs the second prior feature and the context feature into a second probability distribution estimation network to obtain a second probability distribution parameter. That is, the encoder extracts a context feature from the quantized image feature, then obtains a second prior feature by applying a quantization operation to the first prior feature, and determines a second probability distribution parameter based on the second prior feature and the context feature. The second quantization step may be the same as or different from the first quantization step.

[0181] It should be noted that the specific implementation process of step 1104 is similar to the specific implementation process of step 404 in the embodiment of Figure 4. For details, please refer to the related description in step 404. The details will not be described again here.

[0182] Step 1105: Encode the first image feature into a bitstream based on the second probability distribution parameters.

[0183] In this embodiment of the present application, the second probability distribution parameters include probability distribution parameters of multiple feature points, and the encoder encodes the image features of each feature point in the first image feature into a bitstream based on the probability distribution parameters of each feature point in the second probability distribution parameters, for example, by entropy encoding the second image feature into a bitstream.

[0184] The probability distribution estimation network in the encoding and decoding procedures shown in Figures 6 to 10 will be referred to as the first probability distribution estimation network. In this case, when the first probability distribution estimation network in Figures 6 to 10 is replaced with the second probability distribution estimation network, a flowchart of the encoding and decoding method consistent with that in the embodiment in Figure 11 is obtained after the quantization operation on the probability distribution parameters is removed.

[0185] In conclusion, in the encoding process in this embodiment of the present application, hyperprior features of unquantized image features are also determined, but the second probability distribution parameters are subsequently obtained directly via a second probability distribution estimation network. The second probability distribution estimation network is obtained by processing the network parameters in the first probability distribution estimation network based on the first quantization step. In this solution, only the first probability distribution estimation network (for determining the probability distribution parameters of unquantized image features) is trained. Acquired throughEven in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and is not affected by the quantization step. In other words, the input numerical range of the first probability distribution estimation network does not change with the bit rate. Therefore, the first probability distribution estimation network Regarding It's not that hard to train , cheap The first probability distribution estimation network with good performance is trained. Acquired through This helps to improve encoding and decoding performance.

[0186] The following describes the decoding method provided in the embodiment of the present application. The following decoding method shown in Figure 12 is consistent with the above encoding method shown in Figure 4, and the following decoding method shown in Figure 13 is consistent with the above encoding method shown in Figure 11. En Note that this is in accordance with the coding method.

[0187] 12 is a flowchart of a decoding method according to an embodiment of the present application. The method is applied on the decoder side. Please refer to FIG. 12. The method includes the following steps:

[0188] Step 1201: Parse the bitstream to obtain the first hyperprior feature of the image to be decoded.

[0189] In this embodiment of the present application, the decoder side first parses the bitstream to obtain a first hyperprior feature of the image to be decoded. Optionally, the decoder side parses the bitstream to obtain the first hyperprior feature through entropy decoding. For example, the decoder side parses the bitstream through entropy decoding based on a specified probability distribution parameter to obtain the first hyperprior feature. The specified probability distribution parameter is a probability distribution parameter determined in advance through a probability distribution estimation network. The network structure, training method, etc. of the probability distribution estimation network are not limited in this embodiment. It should be noted that the first hyperprior feature includes hyperprior features of multiple feature points. The first hyperprior feature obtained by the decoder side through parsing is consistent with the first hyperprior feature determined by the encoder side. In other words, the first hyperprior feature obtained by the decoder side is the first hyperprior feature of the second image feature or the first hyperprior feature of the third image feature described in the embodiment of Figure 4. The second image feature is an unquantized image feature, and the third image feature is a dequantized image feature.

[0190] Step 1202: Determine a first probability distribution parameter based on the first hyperprior feature via a probability distribution estimation network, where the first probability distribution parameter represents a probability distribution of unquantized image features of the image.

[0191] In this embodiment of the present application, the first hyperprior feature includes hyperprior features of multiple feature points, and the first probability distribution parameter includes probability distribution parameters of multiple feature points. The decoder side determines, through a probability distribution estimation network, the probability distribution parameter of each feature point in the first probability distribution parameter based on the hyperprior feature of each feature point in the first hyperprior feature.

[0192] Note that in implementations where encoding and decoding are not performed via a context network, the decoder side may decode multiple feature points in parallel. In implementations where encoding and decoding are performed via a context network, the decoder side cannot decode multiple feature points simultaneously. For example, the decoder side may sequentially decode multiple feature points; the decoder side may sequentially decode feature points for a channel; the decoder side may sequentially decode multiple groups of feature points, where the amount of feature points in every group may be different; or the decoder side may decode multiple feature points in a different order.

[0193] Additionally, it can be seen from the above description that the first hyperprior feature can be a quantized hyperprior feature or an unquantized hyperprior feature. Based on this, in an implementation where encoding and decoding are not performed through a context network and the first hyperprior feature is a quantized hyperprior feature, the decoder side dequantizes the first hyperprior feature based on the third quantization step to obtain a second hyperprior feature, and inputs the second hyperprior feature into a probability distribution estimation network to obtain the first probability distribution parameters. In an implementation where encoding and decoding are not performed through a context network and the first hyperprior feature is an unquantized hyperprior feature, the decoder side inputs the first hyperprior feature into a probability distribution estimation network to obtain the first probability distribution parameters.

[0194] Encoding and Decoding but Contextual network execution AndIn this implementation, the first feature point is assumed to be any one of the multiple feature points. In this case, the decoder performs the following operations on the first feature point to determine the probability distribution parameter of the first feature point: determine a context feature of the first feature point based on the image feature of the decoded feature point in the first image feature; determine a first prior feature of the first feature point based on the hyper-prior feature of the first feature point; and determine the probability distribution parameter of the first feature point based on the first prior feature of the first feature point and the context feature of the first feature point through a probability distribution estimation network, i.e., determine the probability distribution parameter of the first feature point in the first probability distribution parameter.

[0195] In the implementation where the first hyperprior feature is a quantized hyperprior feature, the decoder side dequantizes the first hyperprior feature obtained through parsing based on the third quantization step to obtain a second hyperprior feature, and inputs the second hyperprior feature into a hyperdecoder network to obtain a first prior feature for each feature point in the plurality of feature points. For the first feature point, the decoder side dequantizes the hyperprior feature of the first feature point in the first hyperprior feature to obtain a hyperprior feature for the first feature point in the second hyperprior feature, and inputs the hyperprior feature of the first feature point in the second hyperprior feature into the hyperdecoder network to obtain a first prior feature for the first feature point. In the implementation where the first hyperprior feature is a quantized hyperprior feature, the decoder side inputs the first hyperprior feature obtained through parsing into a hyperdecoder network to obtain a first prior feature for each feature point in the plurality of feature points. For the first feature point, the decoder side inputs the prior feature of the first feature point in the first prior feature into the hyper-decoder network to obtain the first prior feature of the first feature point.

[0196] Optionally, the decoder side may implement a process of determining a context feature of the first feature point based on the image feature of the decoded feature point in the first image feature as follows: The decoder side determines neighboring feature points of the first feature point from the decoded feature point, and dequantizes the image feature of the neighboring feature points in the first image feature based on a first quantization step to obtain the neighboring feature of the first feature point. Then, the decoder side inputs the neighboring feature of the first feature point into a context network to obtain the context feature of the first feature point. Correspondingly, the decoder side may implement a process of determining a probability distribution parameter of the first feature point within the first probability distribution parameter based on the first prior feature of the first feature point and the context feature of the first feature point via a probability distribution estimation network as follows: The decoder side inputs the first prior feature of the first feature point and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameter of the first feature point within the first probability distribution parameter. That is, the decoder side extracts context features from the dequantized image features, and determines first probability distribution parameters based on the context features and the first prior feature. The neighboring feature points of the first feature point include one or more feature points in the neighborhood of the first feature point.

[0197] Optionally, another implementation process in which the decoder side determines a context feature of the first feature point based on the image feature of the decoded feature point in the first image feature is as follows: the decoder side determines neighboring feature points of the first feature point from the decoded feature point, and inputs the image feature of the neighboring feature points in the first image feature into a context network to obtain the context feature of the first feature point. Correspondingly, an implementation process in which the decoder side determines a probability distribution parameter of the first feature point within the first probability distribution parameter based on the first prior feature of the first feature point and the context feature of the first feature point via a probability distribution estimation network is as follows: the decoder side quantizes the first prior feature of the first feature point based on a second quantization step to obtain a second prior feature of the first feature point, and inputs the second prior feature of the first feature point and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameter of the first feature point within the first probability distribution parameter. That is, the decoder extracts the context features from the quantized image features, then obtains the second prior features by applying a quantization operation to the first prior features, and determines the first probability distribution parameters based on the second prior features and the context features, where the second quantization step is the same as or different from the first quantization step.

[0198] It should be noted that in the implementation where encoding and decoding are performed through a context network, the implementation where the decoder side determines the first prior feature of the first feature point is similar to the related content in the above-mentioned embodiment. The details will not be described again here. In addition, the image feature of the decoded feature point in the first image feature is obtained by decoding the bitstream according to step 1202 to the subsequent step 1204. Specifically, but Contextual network execution AndIn this implementation, the decoder side parses the bitstream according to steps 1202 to 1204 to sequentially obtain image features of all feature points in the first image feature, and the decoder side can decode at least one feature point each time. Also, the first image feature is an image feature obtained by quantizing the second image feature of the image based on a first quantization step, the quantization operation is performed in an encoding process, and the second image feature is an image feature obtained in the encoding process.

[0199] Step 1203: Quantize the first probability distribution parameters based on a first quantization step to obtain second probability distribution parameters.

[0200] The second probability distribution parameter is a probability distribution of multiple feature points. After obtaining the probability distribution parameter of the first feature point in the first probability distribution parameter through decoding, the decoder side quantizes the probability distribution parameter of the first feature point in the first probability distribution parameter based on a first quantization step to obtain the probability distribution parameter of the first feature point in the second probability distribution parameter. Note that in an implementation where encoding and decoding are not performed via a context network, the decoder side may quantize the probability distribution parameters of multiple feature points in the first probability distribution parameter in parallel. Encoding and Decoding but Contextual network execution And In this implementation, each time the decoder side obtains the probability distribution parameters of at least one feature point within the first probability distribution parameters, the decoder side quantizes the probability distribution parameters of the at least one feature point within the first probability distribution parameters.

[0201] Step 1204: Parse the bitstream based on the second probability distribution parameter to obtain a first image feature of the image.

[0202] After obtaining the probability distribution parameters of the first feature point within the second probability distribution parameters, the decoder side parses the bitstream based on the probability distribution parameters of the first feature point within the second probability distribution parameters to obtain the image feature of the first feature point within the first image feature. Optionally, the decoder side parses the bitstream through entropy decoding to obtain the image feature of each feature point in the first image feature. Note that in an implementation where encoding and decoding are not performed via a context network, the decoder side may parse multiple feature points in parallel to obtain the first image feature. Encoding and Decoding but Contextual network execution And In this implementation, each time the decoder side obtains the probability distribution parameters of at least one feature point in the second probability distribution parameters, the decoder side parses the bitstream to obtain the image features of at least one feature point in the first image feature, and obtains the first image feature of the image until the image features of all feature points in the first image feature are obtained through parsing.

[0203] Step 1205: Inverse quantize the first image feature based on the first quantization step to reconstruct an image.

[0204] In this embodiment of the present application, after parsing the bitstream to obtain the first image feature, the decoder side dequantizes the first image feature based on the first quantization step to obtain the third image feature of the image, and reconstructs the image based on the third image feature. Optionally, the decoder side inputs the third image feature into a decoder network to reconstruct the image. The decoding process performed in the decoder network is the inverse process of the feature extraction performed in the image feature extraction network. It should be noted that the third image feature is consistent with the third image feature in the encoding process, and both of these third image features are dequantized image features.

[0205] Optionally, in some other embodiments, the decoder side dequantizes the first image feature based on a fourth quantization step to obtain a fourth image feature of the image, and reconstructs the image based on the fourth image feature. The fourth quantization step is It is different from Good too.

[0206] The above-mentioned decoding process will be explained again below with reference to FIGS.

[0207] In the decoding process of the encoding and decoding method shown in Figure 6, the bitstream is parsed to obtain the hyperprior features ^z, which are then dequantized based on a quantization step q2 to obtain the hyperprior features z. Then, the hyperprior features z are input to a probability distribution estimation network to obtain the probability distribution parameters μ and σ of each feature element in the feature map y. The probability distribution parameters μ and σ are quantized based on a quantization step q1 to obtain the feature map ^y. s The probability distribution parameter μ of each feature element in s and σ s The bitstream is then converted into the feature map ^y s The probability distribution parameter μ of each feature element in s and σ s parsed based on the feature map ^y s Finally, we obtain the feature map ^y s is input to a decoder (Dec) network to reconstruct the image.

[0208] In the decoding process of the encoding and decoding method shown in Figure 7, the bitstream is , PaThe hyperprior feature ^z of each feature point is dequantized based on the quantization step q2 to obtain the hyperprior feature z of each feature point. The hyperprior feature z of each feature point is then input to a hyperdecoder network to obtain the prior feature φ of each feature point. Then, for a first feature point to be decoded, the neighboring feature points of the first feature point are determined from the decoded feature points. s The feature values ​​of the surrounding feature points in the feature map ^y are inversely quantized based on the quantization step q1 to obtain the feature values ​​of the surrounding feature points in the feature map ^y. That is, the surrounding feature of the first feature point is obtained. The feature values ​​of the surrounding feature points in the feature map ^y are input to a context network to obtain the context feature of the first feature point. The context feature of the first feature point and the prior feature of the first feature point are input to a probability distribution estimation network to obtain the probability distribution estimation parameters μ and σ of the first feature point. The probability distribution estimation parameters μ and σ of the first feature point are quantized based on the first quantization step to obtain the probability distribution estimation parameter μ of the first feature point. s and σ s The bit stream is then processed to obtain the probability distribution estimation parameter μ s and σ s The feature map ^y is parsed based on s The image feature of the first feature point in the feature map ^y s When the image features of all feature points in are obtained by parsing the bitstream, the feature map ^y s Then, the feature map ^y s Each feature element in is dequantized based on the quantization step q1 to obtain the feature map ^y. Finally, the feature map ^y is input to the decoder network to reconstruct the image.

[0209] The decoding process of the encoding and decoding method shown in Figure 8 is the same as the decoding process of the encoding and decoding method shown in Figure 6. The decoding process of the encoding and decoding method shown in Figure 9 is the same as the decoding process of the encoding and decoding method shown in Figure 7. The difference between the decoding process of the encoding and decoding method shown in Figure 10 and the decoding processes in Figures 7 and 9 is that a quantization operation is applied after the prior feature φ is obtained through the hyper-decoder network, specifically, the prior feature φ is quantized based on the quantization step q3 to obtain the prior feature φ s The advantage of this is that we can obtain the context feature and prior feature φ for each feature point. s is input to a probability distribution estimation network to obtain first probability distribution parameters μ and σ.

[0210] In conclusion, in this embodiment of the present application, to obtain the probability distribution parameters of the quantized image features, in the decoding process, a first probability distribution parameter is determined through a probability distribution estimation network, where the first probability distribution parameter represents the probability distribution of the unquantized image features. Then, the first probability distribution parameter is quantized based on a first quantization step (i.e., a quantization step for quantizing the image features) to obtain a second probability distribution parameter for representing the probability distribution of the quantized image features. In this solution, only the probability distribution estimation network for determining the probability distribution parameters of the unquantized image features is trained. Acquired through Even in multi-bitrate scenarios, the numerical range of unquantized image features is stable and is not affected by the quantization step. In other words, the input numerical range of the probability distribution estimation network does not change with the bit rate. Therefore, the probability distribution estimation network Regarding It's not that hard to train , cheap The probability distribution estimation network with good performance is trained. Acquired throughThis helps to improve encoding and decoding performance.

[0211] 13 is a flowchart of another decoding method according to an embodiment of the present application. This method is applied on the decoder side. Please refer to FIG. 13. This method includes the following steps:

[0212] Step 1301: Parse the bitstream to obtain the first hyperprior feature of the image to be decoded.

[0213] It should be noted that the specific implementation process of step 1301 is the same as that of step 1201 in the embodiment of Figure 12. For details, please refer to the related description in step 1201. The details will not be described again here.

[0214] Step 1302: Determine second probability distribution parameters based on the first hyperprior feature via a second probability distribution estimation network, where the network parameters of the second probability distribution estimation network are obtained based on the network parameters of the first probability distribution estimation network and the first quantization step, and the first probability distribution estimation network is used to determine a probability distribution of the unquantized image feature.

[0215] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets. Based on this, the last layer in the second probability distribution estimation network ( Convolutional Layer ) The weights and offsets of the first probability distribution estimation network are ( Convolutional Layer )and the first quantization step. Optionally, the second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by the first quantization step. Alternatively, in some embodiments, the second probability distribution estimation network adjusts the network parameters of the last layer in the first probability distribution estimation network in a binary leftward or rightward displacement manner so that the adjusted network parameters are equal to the network parameters obtained before adjustment multiplied by the first quantization step. It is obtained by .

[0216] In this embodiment of the present application, the first hyperprior feature includes hyperprior features of multiple feature points, and the second probability distribution parameter includes probability distribution parameters of multiple feature points. The decoder side determines, through the second probability distribution estimation network, the probability distribution parameter of each feature point in the second probability distribution parameter based on the hyperprior feature of each feature point in the first hyperprior feature.

[0217] Note that in implementations where encoding and decoding are not performed via a context network, the decoder side may decode multiple feature points in parallel. but Contextual network execution And In this implementation, the decoder cannot decode multiple feature points simultaneously.

[0218] Additionally, it can be seen from the above description that the first hyperprior feature can be a quantized hyperprior feature or a non-quantized hyperprior feature. Based on this, in an implementation where the encoding and decoding are not performed through a context network and the first hyperprior feature is a quantized hyperprior feature, the decoder side dequantizes the first hyperprior feature based on the third quantization step to obtain a second hyperprior feature, and inputs the second hyperprior feature into a second probability distribution estimation network to obtain second probability distribution parameters. In an implementation where the encoding and decoding are not performed through a context network and the first hyperprior feature is a non-quantized hyperprior feature, the decoder side inputs the first hyperprior feature into a second probability distribution estimation network to obtain second probability distribution parameters.

[0219] Encoding and Decoding but Contextual network execution And In this implementation, the first feature point is assumed to be any one of the multiple feature points. In this case, the decoder performs the following operations on the first feature point to determine the probability distribution parameter of the first feature point in the second probability distribution parameter: determine a context feature of the first feature point based on the image feature of the decoded feature point in the first image feature; determine a first prior feature of the first feature point based on the hyperprior feature of the first feature point in the first hyperprior feature; and determine the probability distribution parameter of the first feature point in the second probability distribution parameter based on the first prior feature of the first feature point and the context feature of the first feature point via the second probability distribution estimation network.

[0220] In an implementation where the first hyperprior feature is a quantized hyperprior feature, the decoder side dequantizes the first hyperprior feature obtained through parsing based on the third quantization step to obtain a second hyperprior feature, and inputs the second hyperprior feature to a hyperdecoder network to obtain a first prior feature for each feature point in the plurality of feature points. In an implementation where the first hyperprior feature is a quantized hyperprior feature, the decoder side inputs the first hyperprior feature obtained through parsing to a hyperdecoder network to obtain a first prior feature for each feature point in the plurality of feature points.

[0221] Optionally, the decoder side may implement a process for determining a context feature of the first feature point based on the image feature of the decoded feature point in the first image feature as follows: the decoder side determines neighboring feature points of the first feature point from the decoded feature points, and dequantizes the image feature of the neighboring feature points in the first image feature based on the first quantization step to obtain the neighboring feature of the first feature point; the decoder side then inputs the neighboring feature of the first feature point into a context network to obtain the context feature of the first feature point; correspondingly, the decoder side may implement a process for determining a probability distribution parameter of the first feature point in a second probability distribution parameter based on the first prior feature of the first feature point and the context feature of the first feature point via a second probability distribution estimation network as follows: the decoder side inputs the first prior feature of the first feature point and the context feature of the first feature point into the second probability distribution estimation network to obtain the probability distribution parameter of the first feature point in the second probability distribution parameter. That is, the decoder side extracts context features from the dequantized image features, and determines second probability distribution parameters based on the context features and the first prior feature. The neighboring feature points of the first feature point include one or more feature points in the neighborhood of the first feature point.

[0222] Optionally, another implementation process in which the decoder side determines a context feature of the first feature point based on the image feature of the decoded feature point in the first image feature is as follows: the decoder side determines neighboring feature points of the first feature point from the decoded feature point, and inputs the image feature of the neighboring feature points in the first image feature into a context network to obtain the context feature of the first feature point. Correspondingly, an implementation process in which the decoder side determines a probability distribution parameter of the first feature point in a second probability distribution parameter based on the first prior feature of the first feature point and the context feature of the first feature point via a second probability distribution estimation network is as follows: the decoder side quantizes the first prior feature of the first feature point based on a second quantization step to obtain a second prior feature of the first feature point, and inputs the second prior feature of the first feature point and the context feature of the first feature point into the second probability distribution estimation network to obtain the probability distribution parameter of the first feature point in the second probability distribution parameter. That is, the decoder extracts the context features from the quantized image features, then obtains the second prior features by applying a quantization operation to the first prior features, and determines the first probability distribution parameters based on the second prior features and the context features, where the second quantization step is the same as or different from the first quantization step.

[0223] It should be noted that in the implementation where encoding and decoding are performed through a context network, the implementation where the decoder side determines the first prior feature of the first feature point is similar to the related content in the above-mentioned embodiment. The details will not be described again here. In addition, the image feature of the decoded feature point in the first image feature is obtained by decoding the bitstream according to step 1302 to the subsequent step 1304. Specifically, but Contextual network execution AndIn this implementation, the decoder side parses the bitstream according to steps 1302 to 1304 to sequentially obtain image features of all feature points in the first image feature, and the decoder side can decode at least one feature point each time. Also, the first image feature is an image feature obtained by quantizing the second image feature of the image based on a first quantization step, the quantization operation is performed in an encoding process, and the second image feature is an image feature obtained in the encoding process.

[0224] Step 1303: Parse the bitstream based on the second probability distribution parameter to obtain a first image feature of the image.

[0225] After obtaining the probability distribution parameters of the first feature point within the second probability distribution parameters, the decoder side parses the bitstream based on the probability distribution parameters of the first feature point within the second probability distribution parameters to obtain the image feature of the first feature point within the first image feature. Optionally, the decoder side parses the bitstream through entropy decoding to obtain the image feature of each feature point in the first image feature. Note that in an implementation where encoding and decoding are not performed via a context network, the decoder side may parse multiple feature points in parallel to obtain the first image feature. Encoding and Decoding but Contextual network execution And In this implementation, each time the decoder side obtains the probability distribution parameters of at least one feature point in the second probability distribution parameters, the decoder side parses the bitstream to obtain the image features of at least one feature point in the first image feature to obtain the first image feature of the image, until the image features of all feature points in the first image feature have been obtained through parsing.

[0226] It should be noted that the first image feature obtained by the decoder side is consistent with the first image feature obtained by the encoder side, and the first image feature obtained by the encoder side is an image feature obtained by quantizing the second image feature of the image based on the first quantization step.

[0227] Step 1304: Dequantize the first image feature based on the first quantization step to reconstruct an image.

[0228] In this embodiment of the present application, after parsing the bitstream to obtain the first image feature, the decoder side dequantizes the first image feature based on the first quantization step to obtain the third image feature of the image, and reconstructs the image based on the third image feature. Optionally, the decoder side inputs the third image feature into a decoder network to reconstruct the image. The decoding process performed in the decoder network is the inverse process of the feature extraction performed in the image feature extraction network. It should be noted that the third image feature is consistent with the third image feature in the encoding process, and both of these third image features are dequantized image features.

[0229] The probability distribution estimation network in the encoding and decoding procedures shown in Figures 6 to 10 will be referred to as the first probability distribution estimation network. In this case, after the first probability distribution estimation network in Figures 6 to 10 is replaced with the second probability distribution estimation network and the quantization operation on the probability distribution parameters is removed, a flowchart of the encoding and decoding method consistent with that in the embodiment in Figure 13 is obtained.

[0230] In conclusion, in the decoding process in this embodiment of the present application, the second probability distribution parameters are directly obtained through a second probability distribution estimation network. The second probability distribution estimation network is obtained by processing the network parameters in the first probability distribution estimation network based on the first quantization step, and the first probability distribution estimation network is used to determine the probability distribution parameters of the unquantized image features. In this solution, it can be seen that only the first probability distribution estimation network (for determining the probability distribution parameters of the unquantized image features) needs to be obtained through training. Even in a multi-bitrate scenario, the numerical range of the unquantized image features is stable and is not affected by the quantization step. In other words, the input numerical range of the first probability distribution estimation network does not change with the bit rate. Therefore, the first probability distribution estimation network Regarding It's not that hard to train , cheap The first probability distribution estimation network with good performance is trained. Acquired through This helps to improve encoding and decoding performance.

[0231] In addition, after the encoding and decoding method provided in the embodiment of the present application is verified through multiple experiments, it is concluded that this solution can achieve improved encoding and decoding performance for YUV format images for three components, namely Y, U, and V. From the above embodiment of FIG. 4, it can be seen that according to the encoding and decoding method provided in this solution, probability distribution estimation is performed once on unquantized image features to obtain first probability distribution parameters, and then second probability distribution parameters at different bit rates (corresponding to different first quantization steps) can be estimated. There is no need to perform probability estimation for each bit rate. This solution reduces the operational complexity of probability estimation and improves the rate-distortion optimization of feature maps. - distortion optimization izIt can be seen that the probability estimation is unified for different bit rates, which is the probability distribution estimation network. Regarding Makes training easier.

[0232] FIG. 14 is a diagram of the structure of an encoding apparatus 1400 according to an embodiment of the present application. The encoding apparatus 1400 may be implemented as a part of an encoder-side device or as the entire encoder-side device by using software, hardware, or a combination thereof. The encoder-side device may be any of the encoders shown in FIGS. 1 to 3. See FIG. 14. The apparatus 1400 includes a first determination module 1401, a second determination module 1402, a first encoding module 1403, a probability estimation module 1404, a quantization module 1405, and a second encoding module 1406.

[0233] The first determination module 1401 is configured to determine a first image feature and a second image feature of the image to be encoded, where the first image feature is an image feature obtained by quantizing the second image feature based on a first quantization step.

[0234] The second determining module 1402 is configured to determine a first hyperprior feature of the second image feature.

[0235] The first encoding module 1403 is configured to encode the first hyperprior feature into a bitstream.

[0236] The probability estimation module 1404 is configured to determine a first probability distribution parameter based on the first hyperprior feature via a probability distribution estimation network.

[0237] The quantization module 1405 is configured to quantize the first probability distribution parameters based on a first quantization step to obtain second probability distribution parameters.

[0238] The second encoding module 1406 is configured to encode the first image feature into a bitstream based on the second probability distribution parameters.

[0239] Optionally, the second determination module 1402: and a first hyperencoder sub-module configured to input the second image features into a hyperencoder network to obtain the first hyperprior features.

[0240] Optionally, the second determination module 1402: an inverse quantization sub-module configured to inverse quantize the first image feature based on the first quantization step to obtain a third image feature of the image; a second hyperencoder sub-module configured to input the third image features into a hyperencoder network to obtain the first hyperprior features; Includes.

[0241] Optionally, the probability estimation module 1404: a context sub-module configured to input the third image feature of the image into a context network to obtain a context feature of the third image feature, the third image feature being an image feature obtained by dequantizing the first image feature based on the first quantization step; a first determining sub-module configured to determine a first prior feature based on the first hyperprior feature; a first probability estimation sub-module configured to input the first prior features and the context features into the probability distribution estimation network to obtain the first probability distribution parameters; Includes.

[0242] Optionally, the probability estimation module 1404: a context submodule configured to input the first image feature into a context network to obtain a context feature for the first image feature; a second determining sub-module configured to determine a first prior feature based on the first hyperprior feature; a quantization submodule configured to quantize the first prior feature based on a second quantization step to obtain a second prior feature; a second probability estimation sub-module configured to input the second prior features and the context features into the probability distribution estimation network to obtain the first probability distribution parameters; and Includes.

[0243] In this embodiment of the present application, to obtain the probability distribution parameters of the quantized image features, in the encoding process, a first probability distribution parameter is determined based on the hyperprior features of the unquantized image features through a probability distribution estimation network, where the first probability distribution parameter represents the probability distribution of the unquantized image features. Then, the first probability distribution parameter is quantized based on a first quantization step (i.e., a quantization step for quantizing the image features) to obtain a second probability distribution parameter for representing the probability distribution of the quantized image features. In this solution, only the probability distribution estimation network for determining the probability distribution parameters of the unquantized image features is trained. Acquired through Even in multi-bitrate scenarios, the numerical range of unquantized image features is stable and is not affected by the quantization step. In other words, the input numerical range of the probability distribution estimation network does not change with the bit rate. Therefore, the probability distribution estimation network Regarding It's not that hard to train , cheap The probability distribution estimation network with good performance is trained. Acquired through This helps to improve encoding and decoding performance.

[0244] It should be noted that during encoding by the encoding device provided in the above embodiments, the division into the above functional modules is only used as an example for explanation. In actual applications, the above functions may be assigned to different functional modules and implemented based on requirements. Specifically, the internal structure of the device is divided into different functional modules to realize all or part of the above functions. In addition, the encoding device provided in the above embodiments and the encoding method embodiments belong to the same concept. For details regarding the specific implementation process of the encoding device, please refer to the method embodiments. The details will not be described again in this specification.

[0245] FIG. 15 is a diagram of the structure of a decoding device 1500 according to an embodiment of the present application. De The code apparatus 1500 may be implemented as part of a decoder-side device or as an entire decoder-side device using software, hardware, or a combination thereof. Decoder side The device may be any decoder side shown in Figures 1 to 3. See Figure 15. The device 1500 includes a first parsing module 1501, a probability estimation module 1502, a quantization module 1503, a second parsing module 1504, and a reconstruction module 1505.

[0246] The first parsing module 1501 is configured to parse the bitstream to obtain first hyperprior features of the image to be decoded.

[0247] The probability estimation module 1502 is configured to determine a first probability distribution parameter based on the first hyperprior feature via a probability distribution estimation network, where the first probability distribution parameter represents a probability distribution of unquantized image features of the image.

[0248] The quantization module 1503 is configured to quantize the first probability distribution parameters based on a first quantization step to obtain second probability distribution parameters.

[0249] The second parsing module 1504 is configured to parse the bitstream based on the second probability distribution parameters to obtain a first image feature of the image.

[0250] The reconstruction module 1505 is configured to dequantize the first image feature based on the first quantization step to reconstruct the image.

[0251] Optionally, the first image feature is an image feature obtained by quantizing a second image feature of the image based on a first quantization step.

[0252] Optionally, the reconstruction module 1505: an inverse quantization sub-module configured to inverse quantize the first image feature based on the first quantization step to obtain a third image feature of the image; a reconstruction submodule configured to reconstruct the image based on the third image feature; and Includes.

[0253] Optionally, the first probability distribution parameter includes a probability distribution parameter of a plurality of feature points, and the first hyperprior feature includes a hyperprior feature of a plurality of feature points. The probability estimation module 1502 includes a context submodule, a first determination submodule, and a probability estimation submodule.

[0254] For a first feature point, a probability distribution parameter of the first feature point is determined via a context submodule, a first determination submodule, and a probability estimation submodule, and the first feature point is any one of the plurality of feature points.

[0255] The context submodule is configured to determine a context feature of the first feature point based on image features of the decoded feature point in the first image feature.

[0256] The first determination sub-module is configured to determine a first prior feature of the first feature point based on the hyper-prior feature of the first feature point.

[0257] The probability estimation submodule is configured to determine, via a probability distribution estimation network, a probability distribution parameter of the first feature point based on the first prior feature of the first feature point and the context feature of the first feature point.

[0258] Optionally, the context submodule: determining neighboring feature points of the first feature point from the decoded feature points; Inversely quantizing the image features of the peripheral feature points in the first image feature based on the first quantization step to obtain peripheral features of the first feature points; The peripheral features of the first feature point are input to a context network to obtain the context feature of the first feature point. It is structured as follows: Determining the probability distribution parameters of the first feature points via the probability distribution estimation network based on the first prior feature of the first feature points and the context feature of the first feature points includes: inputting the first prior feature of the first feature point and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameter of the first feature point.

[0259] Optionally, the context submodule: determining neighboring feature points of the first feature point from the decoded feature points; Inputting the image features of the peripheral feature points in the first image feature into a context network to obtain the context feature of the first feature point. It is structured as follows: Determining the probability distribution parameters of the first feature points via the probability distribution estimation network based on the first prior feature of the first feature points and the context feature of the first feature points includes: quantizing the first prior feature of the first feature point based on a second quantization step to obtain a second prior feature of the first feature point; inputting the second prior feature of the first feature point and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameter of the first feature point.

[0260] In this embodiment of the present application, to obtain the probability distribution parameters of the quantized image features, in the decoding process, a first probability distribution parameter is determined through a probability distribution estimation network. Here, the first probability distribution parameter represents the probability distribution of the unquantized image features. Then, the first probability distribution parameter is quantized based on a first quantization step (i.e., a quantization step for quantizing the image features) to obtain a second probability distribution parameter for representing the probability distribution of the quantized image features. In this solution, only the probability distribution estimation network for determining the probability distribution parameters of the unquantized image features is trained. Acquired through Even in multi-bitrate scenarios, the numerical range of unquantized image features is stable and is not affected by the quantization step. Therefore, we propose a probability distribution estimation network based on unquantized image features. Regarding It's not that hard to train , cheap The probability distribution estimation network with good performance is trained. Acquired through This helps to improve encoding and decoding performance.

[0261] It should be noted that during decoding by the decoding device provided in the above embodiments, the division into the above functional modules is only used as an example for explanation. In actual applications, the above functions may be assigned to different functional modules and implemented based on requirements. Specifically, the internal structure of the device is divided into different functional modules to realize all or part of the above functions. In addition, the decoding device provided in the above embodiments and the embodiment of the decoding method belong to the same concept. For details regarding the specific implementation process of the decoding device, please refer to the embodiment of the method. The details will not be described again in this specification.

[0262] FIG. 16 is a diagram of the structure of an encoding apparatus 1600 according to an embodiment of the present invention. The encoding apparatus 1600 may be implemented as a part of an encoder-side device or as the entire encoder-side device by using software, hardware, or a combination thereof. The encoder-side apparatus may be any of the encoders shown in FIGS. 1 to 3. See FIG. 16. The apparatus 1600 includes a first determination module 1601, a second determination module 1602, a first encoding module 1603, a probability estimation module 1604, and a second encoding module 1605.

[0263] The first determination module 1601 is configured to determine a first image feature and a second image feature of the image to be encoded, where the first image feature is an image feature obtained by quantizing the second image feature based on the first quantization step.

[0264] The second determining module 1602 is configured to determine a first hyperprior feature of the second image feature.

[0265] The first encoding module 1603 is configured to encode the first hyperprior feature into a bitstream.

[0266] The probability estimation module 1604 is configured to determine second probability distribution parameters based on the first hyperprior feature via a second probability distribution estimation network, the network parameters of the second probability distribution estimation network being obtained based on the network parameters of the first probability distribution estimation network and the first quantization step, and the first probability distribution estimation network being used to determine a probability distribution of the unquantized image feature.

[0267] The second encoding module 1605 is configured to encode the first image feature into a bitstream based on the second probability distribution parameters.

[0268] Optionally, the second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by the first quantization step.

[0269] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets.

[0270] Optionally, the second determination module 1602: and a first hyperencoder sub-module configured to input the second image features into a hyperencoder network to obtain the first hyperprior features.

[0271] Optionally, the second determination module 1602: an inverse quantization sub-module configured to inverse quantize the first image feature based on the first quantization step to obtain a third image feature of the image; a second hyperencoder sub-module configured to input the third image features into a hyperencoder network to obtain the first hyperprior features; Includes.

[0272] In the encoding process in this embodiment of the present application, the hyperprior features of the unquantized image features are also determined, but the second probability distribution parameters are subsequently obtained directly via a second probability distribution estimation network. The second probability distribution estimation network is obtained by processing the network parameters in the first probability distribution estimation network based on the first quantization step. In this solution, only the first probability distribution estimation network (for determining the probability distribution parameters of the unquantized image features) is trained. Acquired through Even in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and is not affected by the quantization step. Therefore, the first probability distribution estimation network Regarding It's not that hard to train , cheap The first probability distribution estimation network with good performance is trained. Acquired through This helps to improve encoding and decoding performance.

[0273] It should be noted that during encoding by the encoding device provided in the above embodiments, the division into the above functional modules is only used as an example for explanation. In actual applications, the above functions may be assigned to different functional modules and implemented based on requirements. Specifically, the internal structure of the device is divided into different functional modules to realize all or part of the above functions. In addition, the encoding device provided in the above embodiments and the encoding method embodiments belong to the same concept. For details regarding the specific implementation process of the encoding device, please refer to the method embodiments. The details will not be described again in this specification.

[0274] FIG. 17 is a diagram of the structure of a decoding apparatus 1700 according to an embodiment of the present invention. The decoding apparatus 1700 may be implemented as part of a decoder-side device or as the entire decoder-side device by using software, hardware, or a combination thereof. The decoder-side apparatus may be any of the decoders shown in FIGS. 1 to 3. See FIG. 17. The apparatus 1700 includes a first parsing module 1701, a probability estimation module 1702, a second parsing module 1703, and a reconstruction module 1704.

[0275] The first parsing module 1701 is configured to parse the bitstream to obtain first hyperprior features of the image to be decoded.

[0276] The probability estimation module 1702 is configured to determine second probability distribution parameters based on the first hyperprior feature via a second probability distribution estimation network, the network parameters of the second probability distribution estimation network being obtained based on the network parameters of the first probability distribution estimation network and the first quantization step, and the first probability distribution estimation network being used to determine a probability distribution of the unquantized image feature.

[0277] The second parsing module 1703 is configured to parse the bitstream based on the second probability distribution parameters to obtain a first image feature of the image.

[0278] The reconstruction module 1704 is configured to dequantize the first image feature based on the first quantization step to reconstruct the image.

[0279] Optionally, the second probability distribution estimation network is obtained by multiplying the network parameters of the last layer in the first probability distribution estimation network by the first quantization step.

[0280] Optionally, the last layer in the first probability distribution estimation network is a convolutional layer, and the network parameters of the convolutional layer include weights and offsets.

[0281] Optionally, the first image feature is an image feature obtained by quantizing a second image feature of the image based on a first quantization step.

[0282] Optionally, the reconstruction module 1704: an inverse quantization sub-module configured to inverse quantize the first image feature based on the first quantization step to obtain a third image feature of the image; a reconstruction submodule configured to reconstruct the image based on the third image feature; and Includes.

[0283] In the decoding process of this embodiment of the present application, the second probability distribution parameters are directly obtained via a second probability distribution estimation network. The second probability distribution estimation network is obtained by processing the network parameters in the first probability distribution estimation network based on the first quantization step, and the first probability distribution estimation network is used to determine the probability distribution parameters of the unquantized image features. In this solution, only the first probability distribution estimation network (for determining the probability distribution parameters of the unquantized image features) is trained. Acquired through Even in multi-bitrate scenarios, the numerical range of the unquantized image features is stable and is not affected by the quantization step. In other words, the input numerical range of the first probability distribution estimation network does not change with the bitstream. Therefore, the first probability distribution estimation network Regarding It's not that hard to train , cheap A first probability distribution estimation network with good performance can be trained, which helps to improve encoding and decoding performance.

[0284] It should be noted that during decoding by the decoding device provided in the above embodiments, the division into the above functional modules is only used as an example for explanation. In actual applications, the above functions may be assigned to different functional modules and implemented based on requirements. Specifically, the internal structure of the device is divided into different functional modules to realize all or part of the above functions. In addition, the decoding device provided in the above embodiments and the embodiment of the decoding method belong to the same concept. For details regarding the specific implementation process of the decoding device, please refer to the embodiment of the method. The details will not be described again in this specification.

[0285] 18 is a block diagram of an encoding and decoding device 1800 according to an embodiment of the present application. The encoding and decoding device 1800 includes a processor 1801, a memory 1802, and a bus system 1803. The processor 1801 and the memory 1802 are connected through the bus system 1803. The memory 1802 is configured to store instructions. The processor 1801 is configured to execute the instructions stored in the memory 1802 to perform the encoding method or decoding method described in the embodiment of the present application. To avoid repetition, the details will not be described again here.

[0286] In this embodiment of the present application, processor 1801 may be a central processing unit (CPU), or processor 1801 may be another general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0287] The memory 1802 may include a ROM device or a RAM device. Any other suitable type of storage device may be used as the memory 1802. The memory 1802 may include code and data 18021 accessed by the processor 1801 via the bus 1803. The memory 1802 may further include an operating system 18023 and an application 18022. The application 18022 includes at least one program that allows the processor 1801 to execute the encoding or decoding method described in the embodiments of the present application. For example, the application 18022 may include applications 1 to N and may further include an encoding or decoding application (abbreviated as a coding application) for executing the encoding or decoding method described in the embodiments of the present application.

[0288] In addition to a data bus, the bus system 1803 may further include a power bus, a control bus, a status signal bus, etc. However, for clarity of explanation, various types of buses are illustrated in the figures as the bus system 1803.

[0289] Optionally, the encoding and decoding apparatus 1800 may further include one or more output devices, such as a display 1804. In one example, the display 1804 may be a touch-sensitive display that combines a display with a touch-sensitive unit operable to sense touch input. The display 1804 may be connected to the processor 1801 via a bus 1803.

[0290] It should be noted that the encoding and decoding device 1800 may perform the encoding method in the embodiments of the present application or the decoding method in the embodiments of the present application.

[0291] Those skilled in the art will appreciate that the functions described with reference to the various illustrative logical blocks, modules, and algorithm steps disclosed and described herein may be implemented by hardware, software, firmware, or any combination thereof. If implemented by software, the functions described with reference to the illustrative logical blocks, modules, and steps may be stored on or transmitted through a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media corresponding to tangible media, such as data storage media, or any communication medium that facilitates the transfer of a computer program from one place to another (e.g., according to a communication protocol). As such, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media or (2) communication media, such as a signal or carrier. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described herein. A computer program product may include computer-readable media.

[0292] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, flash memory, or any other medium that can be used to store necessary program code in the form of instructions or data structures and that can be accessed by a computer. Additionally, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or another remote source over coaxial cable, fiber optics, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio waves, or microwaves, the coaxial cable, fiber optics, twisted pair, DSL, or wireless technologies such as infrared, radio waves, or microwaves are included within the definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals, or other transitory media, and actually refer to non-transitory tangible storage media. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, DVDs, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically via lasers. Combinations of the above items should also be included within the scope of computer-readable media.

[0293] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or equivalent integrated circuits or discrete logic circuits. Accordingly, the term “processor,” as used herein, may refer to the foregoing structure or any other structure that may be applied to implementing the techniques described herein. Furthermore, in some aspects, the functionality described with reference to the exemplary logical blocks, modules, and steps described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or may be incorporated into a combined codec. In addition, the techniques may be implemented entirely in one or more circuits or logical elements. In one example, the various exemplary logical blocks, units, and modules in encoder 100 and decoder 200 may be understood as corresponding circuit devices or logical elements.

[0294] The techniques in the embodiments of the present application may be implemented in a variety of apparatuses or devices, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in the embodiments of the present application to highlight functional aspects of an apparatus configured to perform the disclosed techniques, but they are not necessarily implemented by different hardware units. In practice, as described above, the various units may be combined into a codec hardware unit along with appropriate software and / or firmware, or may be provided by interoperable hardware units (including one or more processors as described above).

[0295] In other words, all or part of the above-described embodiments may be implemented using software, hardware, firmware, or any combination thereof. When software is used to implement an embodiment, all or part of the embodiment may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer instructions are loaded into a computer and executed, the procedures or functions according to the embodiments of the present application are generated, in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or another programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optics, or digital subscriber line (DSL)) or wireless (e.g., infrared, radio waves, or microwave) methods. A computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that consolidates one or more available media. Available media may include magnetic media (e.g., floppy disks, hard disks, or magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), semiconductor media (e.g., solid-state disks, - It should be noted that the computer-readable storage medium mentioned in the embodiments of the present application may be a non-volatile storage medium, that is, a non-transitory storage medium.

[0296] It should be understood that "at least one" referred to in this specification refers to one or more, and "multiple" refers to two or more. In the description of the embodiments of the present application, " / " means "or" unless otherwise specified. For example, A / B can represent A or B. In this specification, "and / or" only describes the association relationship between associated objects and indicates that three relationships may exist. For example, A and / or B can represent the following three cases: only A exists, both A and B exist, and only B exists. In addition, to clearly describe the technical solutions in the embodiments of the present application, terms such as "first" and "second" are used in the embodiments of the present application to distinguish between identical or similar items that basically provide the same function or purpose. Those skilled in the art can understand that terms such as "first" and "second" do not limit the quantity or execution order, and terms such as "first" and "second" do not indicate a clear distinction.

[0297] It should be noted that information (including, but not limited to, user device information, user personal information, etc.), data (including, but not limited to, data used for analysis, stored data, displayed data, etc.), and signals in the embodiments of the present application are used with authorization by the user or full authorization by all parties, and the capture, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, images and videos in the embodiments of the present application are acquired with full authorization.

[0298] The foregoing description is merely an exemplary embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, or improvement made without departing from the spirit and principle of the present application should fall within the protection scope of the present application.

Claims

1. 1. An encoding method, the encoding method comprising: determining a first image feature and a second image feature of an image to be encoded, the first image feature being an image feature obtained by quantizing the second image feature based on a first quantization step; determining a first hyperprior feature of the second image feature; encoding the first hyperprior feature into a bitstream; determining first probability distribution parameters based on the first hyperprior features via a probability distribution estimation network; quantizing the first probability distribution parameters based on the first quantization step to obtain second probability distribution parameters; encoding the first image feature into the bitstream based on the second probability distribution parameters. method.

2. Determining a first hyperprior feature of the second image feature includes: inputting the second image features into a hyperencoder network to obtain the first hyperprior features; The method of claim 1.

3. Determining a first hyperprior feature of the second image feature includes: dequantizing the first image feature based on the first quantization step to obtain a third image feature of the image; inputting the third image feature into a hyperencoder network to obtain the first hyperprior feature; The method of claim 1.

4. Determining first probability distribution parameters based on the first hyperprior features via a probability distribution estimation network includes: inputting the third image feature of the image into a context network to obtain a context feature of the third image feature, the third image feature being an image feature obtained by dequantizing the first image feature based on the first quantization step; determining a first prior feature based on the first hyperprior feature; inputting the first prior feature and the context feature into the probability distribution estimation network to obtain the first probability distribution parameters; 3. The method according to claim 1 or 2.

5. Determining first probability distribution parameters based on the first hyperprior features via a probability distribution estimation network includes: inputting the first image feature into a context network to obtain a context feature for the first image feature; determining a first prior feature based on the first hyperprior feature; quantizing the first prior feature based on a second quantization step to obtain a second prior feature; inputting the second prior features and the context features into the probability distribution estimation network to obtain the first probability distribution parameters.

3. The method according to claim 1 or 2.

6. 1. A decoding method, the decoding method comprising: Parsing the bitstream to obtain a first hyperprior feature of the image to be decoded; determining, via a probability distribution estimation network, first probability distribution parameters representing a probability distribution of unquantized image features of the image based on the first hyperprior features; quantizing the first probability distribution parameters to obtain second probability distribution parameters based on a first quantization step; parsing the bitstream to obtain a first image feature of the image based on the second probability distribution parameter; and dequantizing the first image feature based on the first quantization step to reconstruct the image. method.

7. The method of claim 6 , wherein the first image feature is an image feature obtained by quantizing a second image feature of the image based on the first quantization step.

8. Dequantizing the first image feature based on the first quantization step to reconstruct the image includes: dequantizing the first image feature based on the first quantization step to obtain a third image feature of the image; reconstructing the image based on the third image feature.

8. The method according to claim 6 or 7.

9. the first probability distribution parameters include probability distribution parameters of a plurality of feature points, and the first hyperprior features include hyperprior features of the plurality of feature points; Determining first probability distribution parameters based on the first hyperprior feature via a probability distribution estimation network includes: performing the following operations with respect to a first feature point to determine a probability distribution parameter of the first feature point, the first feature point being any one of the plurality of feature points, and the following operations: determining a context feature of the first feature point based on image features of the decoded feature point in the first image feature; determining a first prior feature of the first feature point based on a hyper-prior feature of the first feature point; determining the probability distribution parameters of the first feature points based on the first prior features of the first feature points and the context features of the first feature points via the probability distribution estimation network; 8. The method according to any one of claims 6 to 7.

10. Determining a context feature of a first feature point based on image features of the decoded feature point in the first image feature includes: determining neighboring feature points of the first feature point from the decoded feature points; Inversely quantizing the image features of the peripheral feature points in the first image feature based on the first quantization step to obtain peripheral features of the first feature points; inputting the peripheral features of the first feature point into a context network to obtain the context feature of the first feature point; Determining the probability distribution parameter of the first feature point based on the first prior feature of the first feature point and the context feature of the first feature point via the probability distribution estimation network includes: inputting the first prior feature of the first feature point and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameter of the first feature point; 10. The method of claim 9.

11. Determining a context feature of a first feature point based on image features of the decoded feature point in the first image feature includes: determining neighboring feature points of the first feature point from the decoded feature points; inputting image features of the peripheral feature points of the first image feature into a context network to obtain the context feature of the first feature point; Determining the probability distribution parameter of the first feature point based on the first prior feature of the first feature point and the context feature of the first feature point via the probability distribution estimation network includes: quantizing the first prior feature of the first feature point based on a second quantization step to obtain a second prior feature of the first feature point; inputting the second prior feature of the first feature point and the context feature of the first feature point into the probability distribution estimation network to obtain the probability distribution parameters of the first feature point.

10. The method of claim 9.

12. 1. An encoding method, the encoding method comprising: determining a first image feature and a second image feature of an image to be encoded, the first image feature being an image feature obtained by quantizing the second image feature based on a first quantization step; determining a first hyperprior feature of the second image feature; encoding the first hyperprior feature into a bitstream; determining second probability distribution parameters based on the first hyperprior features via a second probability distribution estimation network, wherein the network parameters of the second probability distribution estimation network are obtained based on the network parameters of the first probability distribution estimation network and the first quantization step, and the first probability distribution estimation network is used to determine a probability distribution of unquantized image features; encoding the first image feature into the bitstream based on the second probability distribution parameters. method.

13. 1. A decoding method, the decoding method comprising: Parsing the bitstream to obtain a first hyperprior feature of the image to be decoded; determining second probability distribution parameters based on the first hyperprior features via a second probability distribution estimation network, wherein the network parameters of the second probability distribution estimation network are obtained based on the network parameters of the first probability distribution estimation network and a first quantization step, and the first probability distribution estimation network is used to determine a probability distribution of unquantized image features; parsing the bitstream based on the second probability distribution parameters to obtain a first image feature of the image; and dequantizing the first image feature based on the first quantization step to reconstruct the image. method.

14. 1. An encoding device, comprising: a first determination module configured to determine a first image feature and a second image feature of an image to be encoded, the first image feature being an image feature obtained by quantizing the second image feature based on a first quantization step; a second determination module configured to determine a first hyperprior feature of the second image feature; a first encoding module configured to encode the first hyperprior feature into a bitstream; a probability estimation module configured to determine, via a probability distribution estimation network, first probability distribution parameters based on the first hyperprior features; a quantization module configured to quantize the first probability distribution parameters based on the first quantization step to obtain second probability distribution parameters; a second encoding module configured to encode the first image feature into the bitstream based on the second probability distribution parameters; An apparatus having:

15. 1. A decoding device, the decoding device comprising: a first parsing module configured to parse the bitstream to obtain first hyperprior features of the image to be decoded; a probability estimation module configured to determine a first probability distribution parameter based on the first hyperprior feature via a probability distribution estimation network, the first probability distribution parameter representing a probability distribution of unquantized image features of the image; a quantization module configured to quantize the first probability distribution parameters based on a first quantization step to obtain second probability distribution parameters; a second parsing module configured to parse the bitstream based on the second probability distribution parameters to obtain first image features of the image; a reconstruction module configured to dequantize the first image feature based on the first quantization step to reconstruct the image; and An apparatus having:

16. 1. An encoding device, comprising: a first determination module configured to determine a first image feature and a second image feature of an image to be encoded, the first image feature being an image feature obtained by quantizing the second image feature based on a first quantization step; a second determination module configured to determine a first hyperprior feature of the second image feature; a first encoding module configured to encode the first hyperprior feature into a bitstream; a probability estimation module configured to determine second probability distribution parameters based on the first hyperprior features via a second probability distribution estimation network, wherein network parameters of the second probability distribution estimation network are obtained based on network parameters of the first probability distribution estimation network and the first quantization step, and the first probability distribution estimation network is used to determine a probability distribution of unquantized image features; a second encoding module configured to encode the first image feature into the bitstream based on the second probability distribution parameters; An apparatus having:

17. 1. A decoding device, the decoding device comprising: a first parsing module configured to parse the bitstream to obtain first hyperprior features of the image to be decoded; a probability estimation module configured to determine second probability distribution parameters based on the first hyperprior features via a second probability distribution estimation network, wherein the network parameters of the second probability distribution estimation network are obtained based on the network parameters of the first probability distribution estimation network and a first quantization step, and the first probability distribution estimation network is used to determine a probability distribution of unquantized image features; a second parsing module configured to parse the bitstream based on the second probability distribution parameters to obtain first image features of the image; a reconstruction module configured to dequantize the first image feature based on the first quantization step to reconstruct the image; and An apparatus having:

18. a decoder-side device, the decoder-side device having a memory and a processor; A device, wherein the memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory to perform the decoding method of any one of claims 6 to 7 or to perform the decoding method of claim 13.

19. A computer-readable storage medium storing a computer program that, when executed by a processor, causes the steps of the method of any one of claims 1 to 3 to be performed, the steps of the method of claim 12 to be performed, the steps of the method of any one of claims 6 to 7 to be performed, or the steps of the method of claim 13 to be performed.

20. A computer program which, when run by a processor, causes the steps of the method of any one of claims 6 to 7, the steps of the method of claim 13, the steps of the method of any one of claims 1 to 3, or the steps of the method of claim 12 to be performed.

Images