Image processing method and image processing device

Abstract

The embodiment of the present application discloses an image processing method and an image processing device, which are applicable to video image processing scenarios, and are used to improve image compression efficiency and reduce the storage cost of image coding information. The embodiment of the present application provides an image processing method, comprising: obtaining at least two transform coefficient blocks, each of the at least two transform coefficient blocks comprising: at least two transform coefficients, wherein each transform coefficient in a transform coefficient block corresponds to a different frequency point; determining the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order; encoding each transform coefficient corresponding to the one frequency point according to a preset coefficient scanning order to obtain image coding information.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method and an image processing device. Background Art

[0002] Currently, image storage and transmission in electronic devices have become essential processing methods, and image compression is becoming increasingly important in saving transmission bandwidth and storage devices.

[0003] There are many international standards for image compression, such as joint photographic experts group (JPEG), JPEG2000, H.264 intraframe coding, H.265 intraframe coding, etc. For example, the JPEG standard is widely used in hardware devices such as video surveillance cameras, and the generated image files can also be decoded by electronic devices for playback.

[0004] For example, in an image compression scheme based on JPEG, the encoder first transforms the image block, and the transformation method used includes discrete cosine transform (DCT). After the image block undergoes DCT, the low-frequency components of the image block are concentrated in the upper left corner of the image, and the high-frequency components are distributed in the lower right corner of the image. The coefficients in the first row and first column of the image block are direct current (DC) coefficients, that is, the average value of the image block, and the other 63 coefficients in the image block are alternating current (AC) coefficients. Then the DC coefficient and AC coefficient are quantized to obtain the quantized DC coefficient and AC coefficient. Next, the quantized DC coefficient and AC coefficient are compressed. After the image compression data output by the encoder is transmitted to the decoder, the decoder decodes, dequantizes, and inversely transforms the compressed image data to obtain the reconstructed image block.

[0005] The above-mentioned image compression method has the problem of low image compression efficiency. In addition, the compressed file generated by the above-mentioned image compression method consumes a large amount of storage resources of the decoding end, increasing the storage resource overhead of the decoding end. Summary of the invention

[0006] The embodiments of the present application provide an image processing method and an image processing device for improving image compression efficiency and reducing the storage cost of image coding information.

[0007] To solve the above technical problems, the present application provides the following technical solutions:

[0008] In a first aspect, an embodiment of the present application provides an image processing method, comprising: obtaining at least two transform coefficient blocks, each of the at least two transform coefficient blocks comprising: at least two transform coefficients, wherein each transform coefficient in a transform coefficient block corresponds to a different frequency point; determining the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order; encoding each transform coefficient corresponding to the one frequency point according to a preset coefficient scanning order to obtain image coding information. In an embodiment of the present application, a frequency point scanning order can be pre-set or configured, and a frequency point is determined based on the frequency point scanning order, and the frequency point can be any frequency point, each frequency point corresponds to a transform coefficient in each transform coefficient block in at least two transform coefficient blocks, that is, one frequency point corresponds to multiple transform coefficients, and the multiple transform coefficients are derived from different transform coefficient blocks, and a coefficient scanning order can be pre-set or configured, and each transform coefficient corresponding to a frequency point is encoded based on the coefficient scanning order, and finally image coding information can be obtained. In the embodiment of the present application, for the transform coefficients corresponding to the same frequency point in at least two transform coefficient blocks, image coding information can be obtained by only encoding them according to the coefficient scanning order. Therefore, encoding can be achieved for multiple transform coefficients corresponding to the same frequency point, thereby improving image compression efficiency. In the embodiment of the present application, the image coding information is obtained by encoding the transform coefficients corresponding to the same frequency point in at least two transform coefficient blocks according to the coefficient scanning order. The file size of the encoded image coding information is reduced, the storage cost of the image coding information is reduced, and storage resources are ultimately saved.

[0009] In a possible implementation, the determining the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order includes: obtaining at least two two-dimensional subband coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional subband coefficient array includes the transform coefficients corresponding to the same frequency point in the at least two transform coefficient blocks, and the transform coefficients included in different two-dimensional subband coefficient arrays correspond to different frequency points; determining a first two-dimensional subband coefficient array in the at least two two-dimensional subband coefficient arrays according to a preset frequency point scanning order; encoding each transform coefficient corresponding to the one frequency point according to a preset coefficient scanning order includes: determining a first transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order; and entropy encoding the first transform coefficient according to a context area corresponding to the first transform coefficient. In this solution, the encoding end can determine the first transform coefficient in the first two-dimensional sub-band coefficient array according to a preset coefficient scanning order. The first transform coefficient can be any transform coefficient in the first two-dimensional sub-band coefficient array. The encoding end can losslessly encode each transform coefficient in the first two-dimensional sub-band coefficient array, for example, the first transform coefficient can be entropy encoded according to the context area corresponding to the first transform coefficient. In the embodiment of the present application, by encoding the transform coefficients in the two-dimensional sub-band coefficient array, image encoding information can be obtained to achieve compression processing of the image.

[0010] In a possible implementation, the step of determining the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order includes: obtaining at least two two-dimensional subband coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional subband coefficient array includes the transform coefficients corresponding to the same frequency point in the at least two transform coefficient blocks, and the transform coefficients included in different two-dimensional subband coefficient arrays correspond to different frequency points; combining the at least two two-dimensional subband coefficient arrays according to a preset frequency point scanning order to obtain a three-dimensional subband coefficient array; determining the first two-dimensional subband coefficient array in the three-dimensional subband coefficient array according to a preset three-dimensional subband coefficient scanning order; encoding each transform coefficient corresponding to the one frequency point according to a preset coefficient scanning order includes: determining the first transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order; and entropy encoding the first transform coefficient according to the context area corresponding to the first transform coefficient. In this solution, the three-dimensional subband coefficient scanning order is the order in which the three-dimensional subband coefficient array is scanned. After the encoding end determines the first two-dimensional sub-band coefficient array, the encoding end can perform entropy encoding on each transform coefficient in the first two-dimensional sub-band coefficient array according to a preset coefficient scanning order. In the embodiment of the present application, by encoding the transform coefficients in the two-dimensional sub-band coefficient array, image encoding information can be obtained to achieve compression processing of the image.

[0011] In a possible implementation, the step of determining the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order includes: sorting the at least two transform coefficient blocks according to a preset frequency point scanning order to obtain at least two sorted transform coefficient blocks; obtaining multiple two-dimensional subband coefficient arrays corresponding to different frequency points from the at least two sorted transform coefficient blocks; combining the multiple two-dimensional subband coefficient arrays corresponding to different frequency points to obtain a three-dimensional subband coefficient array; determining a first two-dimensional subband coefficient array from the three-dimensional subband coefficient array according to a preset three-dimensional subband coefficient scanning order; encoding each transform coefficient corresponding to the one frequency point according to a preset coefficient scanning order includes: determining a first transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order; and entropy encoding the first transform coefficient according to a context area corresponding to the first transform coefficient. In this solution, after determining the first two-dimensional subband coefficient array, the encoding end can entropy encode each transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order. In the embodiment of the present application, by encoding the transform coefficients in the two-dimensional sub-band coefficient array, image encoding information can be obtained to achieve compression processing of the image.

[0012] In a possible implementation, the three-dimensional subband coefficient scanning order includes: a preset horizontal position index change order; or a preset vertical position index change order; or a preset frequency index change order. In this scheme, each transform coefficient in the three-dimensional subband coefficient array has a horizontal position index, a vertical position index and a frequency index. For example, each transform coefficient can be expressed as C (bx, by, fs), where bx is the horizontal position index, by is the vertical position index, and fs is the frequency index corresponding to the frequency point. The three-dimensional subband coefficient scanning order is the order in which the three-dimensional subband coefficient array is scanned. For example, the three-dimensional subband coefficient scanning order determines multiple two-dimensional subband coefficient arrays in the three-dimensional subband coefficient array from large to small (or small to large) in the order of change of the horizontal position index. After each two-dimensional subband coefficient array is determined, a transform coefficient in the two-dimensional subband coefficient array is determined according to the aforementioned coefficient scanning order, that is, the three-dimensional subband coefficient scanning order can be used to indicate that the two-dimensional subband coefficient arrays corresponding to different horizontal position indexes in the three-dimensional subband coefficient array are encoded in sequence.

[0013] In a possible implementation, the determining of the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order includes: determining a first two-dimensional subband coefficient array consisting of transform coefficients corresponding to the same frequency point in the at least two transform coefficient blocks according to a preset frequency point scanning order; encoding each transform coefficient corresponding to the one frequency point according to a preset coefficient scanning order includes: determining a first transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order; entropy encoding the first transform coefficient according to a context area corresponding to the first transform coefficient. In this scheme, the encoding end does not need to combine a three-dimensional subband coefficient array. After the encoding end determines the first two-dimensional subband coefficient array, the encoding end can encode each transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order. For example, the encoding end determines the first transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order, and entropy encodes the first transform coefficient according to the context area corresponding to the first transform coefficient. In the embodiment of the present application, by encoding the transform coefficients in the two-dimensional subband coefficient array, image encoding information can be obtained to achieve compression processing of the image.

[0014] In a possible implementation, the step of determining the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order includes: determining the transform coefficients in the at least two transform coefficient blocks included in a frequency sub-band according to a preset frequency point scanning order, and the transform coefficients in the at least two transform coefficient blocks included in the frequency sub-band correspond to the same frequency point; the step of encoding each transform coefficient corresponding to the frequency point according to the preset coefficient scanning order includes: determining the first transform coefficient included in the frequency sub-band according to the preset coefficient scanning order; and entropy encoding the first transform coefficient according to the context area corresponding to the first transform coefficient. In this scheme, the first transform coefficient can be any one of the multiple transform coefficients included in a frequency sub-band, and the first transform coefficient is entropy encoded according to the context area corresponding to the first transform coefficient. In the embodiment of the present application, by encoding the transform coefficients in the frequency sub-band, image encoding information can be obtained to achieve image compression processing.

[0015] In a possible implementation, the entropy encoding of the first transform coefficient according to the context area corresponding to the first transform coefficient includes: obtaining context information of the first transform coefficient from the context area corresponding to the first transform coefficient, wherein the context information of the first transform coefficient includes: the encoded transform coefficient in the context area corresponding to the first transform coefficient; estimating the first transform coefficient according to the context information of the first transform coefficient to obtain probability distribution information of the first transform coefficient; and arithmetically encoding the first transform coefficient according to the probability distribution information of the first transform coefficient. In this scheme, the encoder performs probability estimation on the first transform coefficient according to the context information of the first transform coefficient to obtain the probability distribution of the first transform coefficient, and then the encoder performs arithmetically encoding on the first transform coefficient according to the probability distribution of the first transform coefficient to obtain the image encoding information corresponding to the first transform coefficient.

[0016] In a possible implementation manner, the context area corresponding to the first transform coefficient includes at least one of the following: a spatial neighborhood of the first transform coefficient, a subband neighborhood of the first transform coefficient, and a three-dimensional neighborhood of the first transform coefficient.

[0017] In a possible implementation, the entropy encoding of the first transform coefficient according to the context area corresponding to the first transform coefficient includes: when the first transform coefficient is a plurality of transform coefficients, obtaining context information of the plurality of transform coefficients from the context area corresponding to the plurality of transform coefficients, wherein the context information of the plurality of transform coefficients includes: the encoded transform coefficients in the context area corresponding to the plurality of transform coefficients; estimating the plurality of transform coefficients in parallel according to the context information of the plurality of transform coefficients to obtain probability distribution information of the plurality of transform coefficients; and performing arithmetic encoding on the plurality of transform coefficients in parallel according to the probability distribution information of the plurality of transform coefficients. In this scheme, the encoder can obtain a plurality of transform coefficients, and the encoder performs entropy encoding on the plurality of transform coefficients in parallel according to the context information of the plurality of transform coefficients. By performing the entropy encoding in parallel, the image coding efficiency can be improved. As an example, when the encoder obtains the probability distribution of the coefficients, the encoder can also perform probability estimation on the probability distribution of the plurality of transform coefficients in parallel, that is, obtaining the probability distribution of the plurality of transform coefficients at one time, and then performing arithmetic encoding on the plurality of transform coefficients in parallel to improve the coding efficiency.

[0018] In a possible implementation, the context area corresponding to the multiple transform coefficients includes at least one of the following: a spatial neighborhood of the multiple transform coefficients, a subband neighborhood of the multiple transform coefficients, and a three-dimensional neighborhood of the multiple transform coefficients.

[0019] In a possible implementation, the determining the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order includes: obtaining at least two two-dimensional subband coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional subband coefficient array includes the transform coefficients corresponding to the same frequency point in the at least two transform coefficient blocks, and the transform coefficients included in different two-dimensional subband coefficient arrays correspond to different frequency points; determining at least two subband images corresponding to the at least two two-dimensional subband coefficient arrays according to a preset frequency point scanning order; encoding each transform coefficient corresponding to the one frequency point according to a preset coefficient scanning order includes: encoding the transform coefficients in the at least two subband images according to a preset coefficient scanning order. In this scheme, after the encoder obtains at least two two-dimensional sub-band coefficient arrays, the encoder adopts a method of lossless encoding of the sub-band image, using the at least two two-dimensional sub-band coefficient arrays as at least two sub-band images, and performing image lossless encoding on the transform coefficients in the at least two sub-band images according to a preset frequency point scanning order. The encoder can encode the two-dimensional sub-band coefficient array as a sub-band image during T-shirt lossless encoding, thereby simplifying the implementation process of lossless coding.

[0020] In a possible implementation, the obtaining of at least two transform coefficient blocks includes: obtaining transform coefficient coding information of at least two image blocks; performing entropy decoding on the transform coefficient coding information to obtain the at least two transform coefficient blocks. In this solution, the transform coefficient coding information may be transform coefficient coding information corresponding to at least two image blocks. For example, at least two image blocks may be image blocks in a JPEG file. The image processing device may use a Huffman decoding method to perform entropy decoding on the transform coefficient coding information of all image blocks in the JPEG file to obtain at least two transform coefficient blocks corresponding to at least two image blocks. In the embodiment of the present application, the transform coefficient coding information is entropy decoded to obtain at least two transform coefficient blocks, and the transform coefficient blocks are encoded to obtain image coding information, thereby realizing compression processing of the image.

[0021] In a possible implementation, the obtaining of at least two transform coefficient blocks includes: dividing an original image region into at least two image blocks, the original image region including: a pixel image region or a residual image region; transforming and quantizing the at least two image blocks to obtain the at least two transform coefficient blocks. In this solution, a transform coefficient block can be generated by transforming and quantizing the image block, so that the transform coefficient block can be encoded, image encoding information can be obtained, and image compression processing can be achieved.

[0022] In a second aspect, an embodiment of the present application further provides an image processing method, comprising: obtaining image coding information; determining a transform coefficient to be decoded corresponding to a frequency point in the image coding information according to a preset frequency point scanning order; decoding the transform coefficient to be decoded corresponding to the one frequency point according to a preset coefficient scanning order to obtain at least two transform coefficient blocks, wherein each transform coefficient block in the at least two transform coefficient blocks includes: at least two transform coefficients, and each transform coefficient in a transform coefficient block corresponds to a different frequency point. In an embodiment of the present application, a frequency point scanning order can be pre-set or configured, and a transform coefficient to be decoded corresponding to a frequency point in the image coding information is determined based on the frequency point scanning order, and the frequency point can be any frequency point, and a coefficient scanning order can be pre-set, and at least one transform coefficient to be decoded corresponding to a frequency point is decoded based on the coefficient scanning order to obtain at least two transform coefficient blocks, and each transform coefficient block in the at least two transform coefficient blocks includes: at least two transform coefficients, and each transform coefficient in a transform coefficient block corresponds to a different frequency point. In the embodiment of the present application, lossless decoding is performed on the transform coefficients to be decoded corresponding to the same frequency point in the image coding information. At least two transform coefficient blocks can be obtained by decoding according to the coefficient scanning order. Therefore, decoding can be achieved for multiple transform coefficients corresponding to the same frequency point. In the embodiment of the present application, the image coding information is obtained by encoding the transform coefficients corresponding to the same frequency point in at least two transform coefficient blocks according to the coefficient scanning order. The file size of the encoded image coding information is reduced, the storage cost of the image coding information is reduced, and storage resources are ultimately saved.

[0023] In a possible implementation, the step of determining the transform coefficient to be decoded corresponding to a frequency point in the image coding information according to a preset frequency point scanning order includes: determining a first two-dimensional subband coefficient array to be decoded in the image coding information according to a preset frequency point scanning order, the first two-dimensional subband coefficient array to be decoded including: a transform coefficient to be decoded corresponding to a frequency point in the image coding information; the step of decoding the transform coefficient to be decoded corresponding to the one frequency point according to a preset coefficient scanning order includes: determining the first transform coefficient to be decoded in the first two-dimensional subband coefficient array to be decoded according to a preset coefficient scanning order; and entropy decoding the first transform coefficient to be decoded according to a context area corresponding to the first transform coefficient to be decoded. In this solution, the decoding end adopts an image decoding method corresponding to the encoding end. If the encoding end executes the encoding method shown in steps C1 to C4, the image coding information received by the decoding end may include at least two two-dimensional subband coefficient arrays to be decoded. The frequency point scanning order is the scanning order of the two-dimensional subband coefficient array to be decoded. A first two-dimensional subband coefficient array to be decoded is obtained from at least two two-dimensional subband coefficient arrays to be decoded according to the preset frequency point scanning order. The first two-dimensional subband coefficient array to be decoded includes multiple transform coefficients to be decoded.

[0024] In a possible implementation, the determining the transform coefficient to be decoded corresponding to a frequency point in the image coding information according to a preset frequency point scanning order includes: determining a three-dimensional subband coefficient array to be decoded in the image coding information according to a preset frequency point scanning order, the three-dimensional subband coefficient array to be decoded includes: transform coefficients to be decoded corresponding to all frequency points in the image coding information respectively; determining a first two-dimensional subband coefficient array to be decoded in the three-dimensional subband coefficient array to be decoded according to a preset three-dimensional subband coefficient scanning order, the first two-dimensional subband coefficient array to be decoded includes: transform coefficients to be decoded corresponding to a frequency point in the image coding information; decoding the transform coefficient to be decoded corresponding to the one frequency point according to the preset coefficient scanning order includes: determining the first transform coefficient to be decoded in the first two-dimensional subband coefficient array to be decoded according to the preset coefficient scanning order; and entropy decoding the first transform coefficient to be decoded according to the context area corresponding to the first transform coefficient to be decoded. In this scheme, the decoding end adopts an image decoding method corresponding to the encoding end. When the encoding end adopts the method of generating a three-dimensional sub-band coefficient array, the decoding end performs lossless decoding on the second two-dimensional sub-band coefficient array according to the coefficient scanning order preset by the decoding end to obtain the at least two transform coefficient blocks.

[0025] In a possible implementation, the three-dimensional sub-band coefficient scanning order includes: a preset horizontal position index change order; or a preset vertical position index change order; or a preset frequency index change order.

[0026] In a possible implementation, the step of determining the transform coefficient to be decoded corresponding to a frequency point in the image coding information according to a preset frequency point scanning order includes: determining the transform coefficient to be decoded included in a frequency subband in the image coding information according to a preset frequency point scanning order, wherein the transform coefficient to be decoded included in the frequency subband corresponds to the same frequency point; the step of decoding the transform coefficient to be decoded corresponding to the frequency point according to a preset coefficient scanning order includes: determining the first transform coefficient to be decoded in the frequency subband according to the preset coefficient scanning order; and entropy decoding the first transform coefficient to be decoded according to the context area corresponding to the first transform coefficient to be decoded. In this solution, the decoding end obtains the image coding information corresponding to multiple frequency subbands, and the decoding end adopts a method of lossless decoding of the frequency subbands, using at least two two-dimensional subband coefficient arrays as at least two frequency subbands, and performing lossless decoding on the image coding information corresponding to the at least two frequency subbands according to a preset coefficient scanning order. The decoding end can decode the two-dimensional subband coefficient array as a frequency subband during lossless decoding, thereby simplifying the implementation process of lossless decoding.

[0027] In a possible implementation, entropy decoding the first transform coefficient to be decoded according to the context area corresponding to the first transform coefficient to be decoded includes: obtaining context information of the first transform coefficient to be decoded from the context area corresponding to the first transform coefficient to be decoded, wherein the context area corresponding to the first transform coefficient to be decoded includes: decoded transform coefficients in the context area corresponding to the first transform coefficient to be decoded; estimating the first transform coefficient to be decoded according to the context information of the first transform coefficient to obtain probability distribution information of the first transform coefficient to be decoded; arithmetically decoding the first transform coefficient to be decoded according to the probability distribution information of the first transform coefficient to obtain a decoded first transform coefficient, wherein the decoded first transform coefficient belongs to the at least two transform coefficient blocks. In this solution, after the decoding end obtains the context information of the first transform coefficient, the decoding end entropy decodes the first transform coefficient according to the context information of the first transform coefficient to obtain the decoded first transform coefficient. The entropy decoding performed by the decoding end may include probability estimation and arithmetic decoding. Specifically, the decoding end performs probability estimation on the first transform coefficient according to the context information of the first transform coefficient to obtain the probability distribution of the first transform coefficient, and then the decoding end performs arithmetically decoding on the first transform coefficient according to the probability distribution of the first transform coefficient to obtain image decoding information corresponding to the first transform coefficient.

[0028] In a possible implementation, the context area corresponding to the first transform coefficient to be decoded includes at least one of the following: a spatial neighborhood of the first transform coefficient to be decoded, a subband neighborhood of the first transform coefficient to be decoded, and a three-dimensional neighborhood of the first transform coefficient to be decoded.

[0029] In a possible implementation, entropy decoding the first transform coefficient to be decoded according to the context area corresponding to the first transform coefficient to be decoded includes: when the first transform coefficient to be decoded is a plurality of transform coefficients to be decoded, obtaining context information of the plurality of transform coefficients to be decoded from the context area corresponding to the plurality of transform coefficients to be decoded, wherein the plurality of context information to be decoded includes: decoded transform coefficients in the context area corresponding to the plurality of transform coefficients to be decoded; estimating the plurality of transform coefficients to be decoded in parallel according to the context information of the plurality of transform coefficients to be decoded to obtain probability distribution information of the first transform coefficient to be decoded; arithmetically decoding the plurality of transform coefficients to be decoded in parallel according to the probability distribution information of the plurality of transform coefficients to be decoded to obtain a decoded first transform coefficient, wherein the decoded first transform coefficient belongs to the at least two transform coefficient blocks. In this solution, the decoding end adopts an image decoding method corresponding to the encoding end, and the encoding end can perform encoding in parallel during encoding, so that the image decoding efficiency can be improved by parallel decoding at the decoding end.

[0030] In a possible implementation, the context area corresponding to the multiple transform coefficients to be decoded includes at least one of the following: a spatial neighborhood of the multiple transform coefficients to be decoded, a subband neighborhood of the multiple transform coefficients to be decoded, and a three-dimensional neighborhood of the multiple transform coefficients to be decoded.

[0031] In a possible implementation, the step of determining the transform coefficient to be decoded corresponding to a frequency point in the image coding information according to a preset frequency point scanning order includes: determining the first sub-band image in the image coding information according to a preset frequency point scanning order, the first sub-band image including: the transform coefficient to be decoded corresponding to a frequency point in the image coding information; the step of decoding the transform coefficient to be decoded corresponding to the one frequency point according to the preset coefficient scanning order includes: decoding the coefficient to be decoded of the first sub-band image according to the preset coefficient scanning order to obtain the at least two transform coefficient blocks. In this solution, the decoding end adopts a lossless decoding method for the sub-band image to obtain a plurality of transform coefficients, and then the decoding end adopts a lossless decoding method for the sub-band image, using at least two two-dimensional sub-band coefficient arrays as at least two sub-band images, and performing lossless decoding on the image coding information corresponding to the at least two sub-band images according to the preset coefficient scanning order. The decoding end can decode the two-dimensional sub-band coefficient array as a sub-band image during lossless decoding, thereby simplifying the implementation process of lossless decoding.

[0032] In a possible implementation, after obtaining at least two transform coefficient blocks, the method further includes: entropy encoding the at least two transform coefficient blocks to obtain transform coefficient coding information; generating at least two image blocks according to the transform coefficient coding information; restoring the original image area according to the at least two image blocks, and the original image area includes: a pixel image area or a residual image area. In this scheme, the decoding end adopts an image decoding method corresponding to the encoding end. If the encoding end obtains a transform coefficient block by entropy decoding the transform coefficient coding information and encodes the transform coefficient block, the decoding end can entropy encode the at least two transform coefficient blocks to obtain the transform coefficient coding information. In the embodiment of the present application, at least two image blocks are generated according to the transform coefficient coding information, and then the original image area is restored according to the at least two image blocks, so that the image decompression process can be completed.

[0033] In a possible implementation, after obtaining at least two transform coefficient blocks, the method further includes: dequantizing and inverse transforming the at least two transform coefficient blocks to obtain at least two image blocks; restoring the original image area according to the at least two image blocks, the original image area including: a pixel image area or a residual image area. In this solution, after generating at least two transform coefficient blocks, the decoder generates at least two image blocks according to the at least two transform coefficient blocks. The decoder can process the at least two transform coefficient blocks by inverse transform and inverse quantization, thereby generating at least two image blocks.

[0034] In a third aspect, an embodiment of the present application provides an image processing device, comprising: a transceiver module, used to obtain at least two transform coefficient blocks, each of the at least two transform coefficient blocks includes: at least two transform coefficients, wherein each transform coefficient in a transform coefficient block corresponds to a different frequency point; a processing module, used to determine the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order; and encode each transform coefficient corresponding to the one frequency point according to a preset coefficient scanning order to obtain image coding information. In an embodiment of the present application, a frequency point scanning order can be pre-set or configured, and a frequency point is determined based on the frequency point scanning order. The frequency point can be any frequency point, and each frequency point corresponds to a transform coefficient in each transform coefficient block in at least two transform coefficient blocks, that is, one frequency point corresponds to multiple transform coefficients, and the multiple transform coefficients come from different transform coefficient blocks. The coefficient scanning order can be pre-set or configured, and each transform coefficient corresponding to a frequency point is encoded based on the coefficient scanning order, and finally the image coding information can be obtained. In the embodiment of the present application, for the transform coefficients corresponding to the same frequency point in at least two transform coefficient blocks, image coding information can be obtained by only encoding them according to the coefficient scanning order. Therefore, encoding can be achieved for multiple transform coefficients corresponding to the same frequency point, thereby improving image compression efficiency. In the embodiment of the present application, the image coding information is obtained by encoding the transform coefficients corresponding to the same frequency point in at least two transform coefficient blocks according to the coefficient scanning order. The file size of the encoded image coding information is reduced, the storage cost of the image coding information is reduced, and storage resources are ultimately saved.

[0035] In one possible implementation, the processing module is used to obtain at least two two-dimensional subband coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional subband coefficient array includes transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks, and different two-dimensional subband coefficient arrays include transform coefficients corresponding to different frequency points; determine a first two-dimensional subband coefficient array in the at least two two-dimensional subband coefficient arrays according to a preset frequency point scanning order; determine a first transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order; and perform entropy encoding on the first transform coefficient according to a context area corresponding to the first transform coefficient.

[0036] In one possible implementation, the processing module is used to obtain at least two two-dimensional subband coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional subband coefficient array includes transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks, and different two-dimensional subband coefficient arrays include transform coefficients corresponding to different frequency points; the at least two two-dimensional subband coefficient arrays are combined according to a preset frequency point scanning order to obtain a three-dimensional subband coefficient array; a first two-dimensional subband coefficient array in the three-dimensional subband coefficient array is determined according to a preset three-dimensional subband coefficient scanning order; a first transform coefficient in the first two-dimensional subband coefficient array is determined according to a preset coefficient scanning order; and the first transform coefficient is entropy encoded according to a context area corresponding to the first transform coefficient.

[0037] In one possible implementation, the processing module is used to sort the at least two transform coefficient blocks according to a preset frequency point scanning order to obtain at least two sorted transform coefficient blocks; obtain multiple two-dimensional sub-band coefficient arrays corresponding to different frequency points from the at least two sorted transform coefficient blocks; combine the multiple two-dimensional sub-band coefficient arrays corresponding to different frequency points to obtain a three-dimensional sub-band coefficient array; determine a first two-dimensional sub-band coefficient array from the three-dimensional sub-band coefficient array according to a preset three-dimensional sub-band coefficient scanning order; determine a first transform coefficient in the first two-dimensional sub-band coefficient array according to a preset coefficient scanning order; and entropy encode the first transform coefficient according to a context area corresponding to the first transform coefficient.

[0038] In a possible implementation, the three-dimensional sub-band coefficient scanning order includes: a preset horizontal position index change order; or a preset vertical position index change order; or a preset frequency index change order.

[0039] In one possible implementation, the processing module is used to determine a first two-dimensional sub-band coefficient array consisting of transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks according to a preset frequency point scanning order; determine a first transform coefficient in the first two-dimensional sub-band coefficient array according to a preset coefficient scanning order; and perform entropy encoding on the first transform coefficient according to a context area corresponding to the first transform coefficient.

[0040] In a possible implementation, the processing module is used to determine the transform coefficients in the at least two transform coefficient blocks included in a frequency sub-band according to a preset frequency point scanning order, and the transform coefficients in the at least two transform coefficient blocks included in the one frequency sub-band correspond to the same frequency point; determine the first transform coefficient included in the one frequency sub-band according to a preset coefficient scanning order; and perform entropy encoding on the first transform coefficient according to a context area corresponding to the first transform coefficient.

[0041] In a possible implementation, the processing module is used to obtain context information of the first transform coefficient from the context area corresponding to the first transform coefficient, wherein the context information of the first transform coefficient includes: an encoded transform coefficient in the context area corresponding to the first transform coefficient; estimating the first transform coefficient according to the context information of the first transform coefficient to obtain probability distribution information of the first transform coefficient; and arithmetically encoding the first transform coefficient according to the probability distribution information of the first transform coefficient.

[0042] In a possible implementation manner, the context area corresponding to the first transform coefficient includes at least one of the following: a spatial neighborhood of the first transform coefficient, a subband neighborhood of the first transform coefficient, and a three-dimensional neighborhood of the first transform coefficient.

[0043] In one possible implementation, the processing module is used to obtain context information of the multiple transform coefficients from the context area corresponding to the multiple transform coefficients when the first transform coefficient is multiple transform coefficients, wherein the context information of the multiple transform coefficients includes: encoded transform coefficients in the context area corresponding to the multiple transform coefficients; estimating the multiple transform coefficients in parallel according to the context information of the multiple transform coefficients to obtain probability distribution information of the multiple transform coefficients; and arithmetically encoding the multiple transform coefficients in parallel according to the probability distribution information of the multiple transform coefficients.

[0044] In a possible implementation, the context area corresponding to the multiple transform coefficients includes at least one of the following: a spatial neighborhood of the multiple transform coefficients, a subband neighborhood of the multiple transform coefficients, and a three-dimensional neighborhood of the multiple transform coefficients.

[0045] In a possible implementation, the processing module is used to obtain at least two two-dimensional sub-band coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional sub-band coefficient array includes transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks, and different two-dimensional sub-band coefficient arrays include transform coefficients corresponding to different frequency points; determine at least two sub-band images corresponding to the at least two two-dimensional sub-band coefficient arrays according to a preset frequency point scanning order; and encode the transform coefficients in the at least two sub-band images according to a preset coefficient scanning order.

[0046] In a possible implementation, the transceiver module is used to obtain transform coefficient coding information of at least two image blocks; and perform entropy decoding on the transform coefficient coding information to obtain the at least two transform coefficient blocks.

[0047] In a possible implementation, the transceiver module is used to divide an original image area into at least two image blocks, where the original image area includes: a pixel image area or a residual image area; and transform and quantize the at least two image blocks to obtain the at least two transform coefficient blocks.

[0048] In the third aspect of the present application, the constituent modules of the image processing device may also execute the steps described in the aforementioned first aspect and various possible implementations, see the aforementioned description of the first aspect and various possible implementations for details.

[0049] In a fourth aspect, an embodiment of the present application also provides an image processing device, comprising: a transceiver module for acquiring image coding information; a processing module for determining a transform coefficient to be decoded corresponding to a frequency point in the image coding information according to a preset frequency point scanning order; decoding the transform coefficient to be decoded corresponding to the one frequency point according to a preset coefficient scanning order to obtain at least two transform coefficient blocks, wherein each transform coefficient block in the at least two transform coefficient blocks includes: at least two transform coefficients, and each transform coefficient in a transform coefficient block corresponds to a different frequency point.

[0050] In a possible implementation, the processing module is used to determine a first two-dimensional subband coefficient array to be decoded in the image coding information according to a preset frequency point scanning order, wherein the first two-dimensional subband coefficient array to be decoded includes: a transform coefficient to be decoded corresponding to a frequency point in the image coding information; determining the first transform coefficient to be decoded in the first two-dimensional subband coefficient array to be decoded according to a preset coefficient scanning order; and performing entropy decoding on the first transform coefficient to be decoded according to a context area corresponding to the first transform coefficient to be decoded.

[0051] In a possible implementation, the processing module is used to determine a three-dimensional subband coefficient array to be decoded in the image coding information according to a preset frequency point scanning order, the three-dimensional subband coefficient array to be decoded includes: transform coefficients to be decoded corresponding to all frequency points in the image coding information; determine a first two-dimensional subband coefficient array to be decoded in the three-dimensional subband coefficient array to be decoded according to a preset three-dimensional subband coefficient scanning order, the first two-dimensional subband coefficient array to be decoded includes: a transform coefficient to be decoded corresponding to a frequency point in the image coding information; determine a first transform coefficient to be decoded in the first two-dimensional subband coefficient array to be decoded according to a preset coefficient scanning order; and perform entropy decoding on the first transform coefficient to be decoded according to a context area corresponding to the first transform coefficient to be decoded.

[0052] In a possible implementation, the three-dimensional sub-band coefficient scanning order includes: a preset horizontal position index change order; or a preset vertical position index change order; or a preset frequency index change order.

[0053] In a possible implementation, the processing module is used to determine the transform coefficients to be decoded included in a frequency subband in the image coding information according to a preset frequency point scanning order, and the transform coefficients to be decoded included in the frequency subband correspond to the same frequency point; determine the first transform coefficient to be decoded in the frequency subband according to a preset coefficient scanning order; and perform entropy decoding on the first transform coefficient to be decoded according to a context area corresponding to the first transform coefficient to be decoded.

[0054] In a possible implementation, the processing module is used to obtain context information of the first transform coefficient to be decoded from the context area corresponding to the first transform coefficient to be decoded, wherein the context area corresponding to the first transform coefficient to be decoded includes: a decoded transform coefficient in the context area corresponding to the first transform coefficient to be decoded; estimating the first transform coefficient to be decoded according to the context information of the first transform coefficient to be decoded to obtain probability distribution information of the first transform coefficient to be decoded; arithmetically decoding the first transform coefficient to be decoded according to the probability distribution information of the first transform coefficient to be decoded to obtain a decoded first transform coefficient, and the decoded first transform coefficient belongs to the at least two transform coefficient blocks.

[0055] In a possible implementation, the context area corresponding to the first transform coefficient to be decoded includes at least one of the following: a spatial neighborhood of the first transform coefficient to be decoded, a subband neighborhood of the first transform coefficient to be decoded, and a three-dimensional neighborhood of the first transform coefficient to be decoded.

[0056] In one possible implementation, the processing module is used to obtain context information of the multiple transform coefficients to be decoded from the context area corresponding to the multiple transform coefficients to be decoded when the first transform coefficient to be decoded is a plurality of transform coefficients to be decoded, wherein the multiple context information to be decoded include: decoded transform coefficients in the context area corresponding to the multiple transform coefficients to be decoded; estimating the multiple transform coefficients to be decoded in parallel according to the context information of the multiple transform coefficients to be decoded to obtain probability distribution information of the first transform coefficient to be decoded; and arithmetically decoding the multiple transform coefficients to be decoded in parallel according to the probability distribution information of the multiple transform coefficients to be decoded to obtain a decoded first transform coefficient, and the decoded first transform coefficient belongs to the at least two transform coefficient blocks.

[0057] In a possible implementation, the context area corresponding to the multiple transform coefficients to be decoded includes at least one of the following: a spatial neighborhood of the multiple transform coefficients to be decoded, a subband neighborhood of the multiple transform coefficients to be decoded, and a three-dimensional neighborhood of the multiple transform coefficients to be decoded.

[0058] In a possible implementation, the processing module is used to determine the first sub-band image in the image coding information according to a preset frequency point scanning order, the first sub-band image including: the transform coefficient to be decoded corresponding to a frequency point in the image coding information; the coefficient to be decoded of the first sub-band image is decoded according to a preset coefficient scanning order to obtain the at least two transform coefficient blocks.

[0059] In a possible implementation, the processing module is further used to, after obtaining at least two transform coefficient blocks, perform entropy encoding on the at least two transform coefficient blocks to obtain transform coefficient encoding information; generate at least two image blocks according to the transform coefficient encoding information; and restore the original image area according to the at least two image blocks, wherein the original image area includes: a pixel image area or a residual image area.

[0060] In a possible implementation, the processing module is further used to, after obtaining at least two transform coefficient blocks, perform inverse quantization and inverse transformation on the at least two transform coefficient blocks to obtain at least two image blocks; and restore the original image area based on the at least two image blocks, wherein the original image area includes: a pixel image area or a residual image area.

[0061] In the fourth aspect of the present application, the constituent modules of the image processing device may also execute the steps described in the aforementioned second aspect and various possible implementations, see the aforementioned description of the second aspect and various possible implementations for details.

[0062] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enables the computer to execute the method described in the first aspect or the second aspect above.

[0063] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method described in the first aspect or the second aspect above.

[0064] In a seventh aspect, an embodiment of the present application provides an image processing device, which may include an image encoding device or an image decoding device or an entity such as a chip, and the image processing device includes: a processor. The image processing may also include a memory; the memory is used to store instructions; the processor is used to execute the instructions in the memory, so that the image processing device performs the method as described in any one of the first aspect or the second aspect.

[0065] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the method described in the first aspect or the second aspect above.

[0066] In a ninth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method described in the first or second aspect above.

[0067] In a tenth aspect, the present application provides a chip system, which includes a processor for supporting an image encoding device or an image decoding device to implement the functions involved in the first aspect or the second aspect, for example, sending or processing the data and / or information involved in the above method. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the image encoding device or the image decoding device. The chip system can be composed of a chip, or it can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A schematic diagram of the structure of an image processing system provided in an embodiment of the present application;

[0069] Figure 2 A schematic diagram of an image encoding device and an image decoding device provided in an embodiment of the present application being applied to a terminal device;

[0070] Figure 3 A schematic diagram of an interaction process between an image encoding device and an image decoding device provided in an embodiment of the present application;

[0071] Figure 4 A schematic diagram of an image encoding process provided in an embodiment of the present application;

[0072] Figure 5 A schematic diagram of obtaining a transform coefficient block provided in an embodiment of the present application;

[0073] Figure 6 A schematic diagram of obtaining a two-dimensional sub-band coefficient array provided in an embodiment of the present application;

[0074] Figure 7 A schematic diagram of a transform coefficient block provided in an embodiment of the present application;

[0075] Figure 8 A schematic diagram of an image decoding process provided in an embodiment of the present application;

[0076] Fig. 9 A schematic diagram of the correspondence between a two-dimensional subband coefficient array and a transform coefficient block provided in an embodiment of the present application;

[0077] Fig.10 A schematic diagram of an image encoding process provided in an embodiment of the present application;

[0078] Fig.11a A schematic diagram of a three-dimensional subband coefficient array provided in an embodiment of the present application;

[0079] Fig.11b A schematic diagram of a three-dimensional neighborhood of a transform coefficient provided in an embodiment of the present application;

[0080] Fig.12 A schematic diagram of an image decoding process provided in an embodiment of the present application;

[0081] Fig.13 A schematic diagram of an image encoding process provided in an embodiment of the present application;

[0082] Fig.14 A schematic diagram of an image encoding process provided in an embodiment of the present application;

[0083] Fig.15 A schematic diagram of an image encoding process provided in an embodiment of the present application;

[0084] Fig.16 A schematic diagram of an image decoding process provided in an embodiment of the present application;

[0085] Fig.17 A schematic diagram of the structure of an image encoding device provided in an embodiment of the present application;

[0086] Fig.18 A schematic diagram of the structure of an image decoding device provided in an embodiment of the present application;

[0087] Fig.19 A schematic diagram of the structure of another image encoding device provided in an embodiment of the present application;

[0088] Fig. 20 A schematic diagram of the composition structure of another image decoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0089] The embodiments of the present application provide an image processing method and an image processing device for improving image compression efficiency and reducing the storage cost of image coding information.

[0090] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0091] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0092] The technical solution of the embodiment of the present application can be applied to various image processing systems, such as Figure 1 As shown, it is a schematic diagram of the composition structure of the image processing system provided by the embodiment of the present application. The image processing system 100 may include: an image encoding device 101 and an image decoding device 102. Among them, the image encoding device 101 can be used to generate image coding information according to at least two transform coefficient blocks. For example, at least two transform coefficient blocks can be generated according to at least two image blocks corresponding to the original image area. Among them, the original image area can be the entire image or a rectangular image area, or an image area of ​​other shapes, which is not limited this time. For example, the original image area includes: a pixel image area or a residual image area. In addition, the image coding information can be an image coding code stream (also called a compressed code stream), and then the image coding information can be transmitted to the image decoding device 102 through an image transmission channel. The image decoding device 102 can receive the image coding information, and then the image decoding device 102 can decode the image coding information to obtain at least two transform coefficient blocks. For example, at least two transform coefficient blocks can also be used to restore the original image.

[0093] In the embodiments of the present application, the image encoding device can be applied to various terminal devices with image communication needs, wireless devices and core network devices with image transcoding needs, for example, the image encoding device can be the image encoder of the above terminal device or wireless device or core network device. Similarly, the image decoding device can be applied to various terminal devices with image communication needs, wireless devices and core network devices with image transcoding needs, for example, the image decoding device can be the image decoder of the above terminal device or wireless device or core network device.

[0094] For example, the image encoding device is specifically an image encoder, and the image decoding device is specifically an image decoder. The terminal device may include: an image encoder and a channel encoder, or the terminal device may include an image decoder and a channel decoder. Specifically, the channel encoder is used to channel encode the image signal, the channel decoder is used to channel decode the image signal, and the image encoder is used to encode the original image area, for example, the image encoder has a lossless encoding function. The image decoder is used to decode the image encoding information, for example, the image decoder has a lossless decoding function. For example, the lossless encoding methods that can be used include entropy encoding, Huffman encoding, etc. For example, the lossless decoding methods that can be used include entropy decoding, Huffman decoding, etc.

[0095] like Figure 2 As shown, it is a schematic diagram of the image encoding device and the image decoding device provided by the embodiment of the present application applied to a terminal device. The first terminal device 20 may include: an image encoder 201 and a channel encoder 202. The second terminal device 21 may include: an image decoder 211 and a channel decoder 212. The first terminal device 20 is connected to a wireless or wired first network communication device 22, and the first network communication device 22 and the wireless or wired second network communication device 23 are connected through a digital channel, and the second terminal device 21 is connected to the wireless or wired second network communication device 23. Among them, the above-mentioned wireless or wired network communication equipment can generally refer to signal transmission equipment, such as communication base stations, data exchange equipment, etc.

[0096] In image communication, the terminal device at the transmitting end performs image encoding on the collected image signal, and then performs channel encoding, and transmits it in a digital channel through a wireless network or core network. The terminal device at the receiving end performs channel decoding based on the received signal to obtain the image signal encoding stream, and then restores the image signal through image decoding, which is played back by the terminal device at the receiving end.

[0097] The embodiment of the present application provides encoding of the transform coefficient block corresponding to the original image area, improves the image encoding performance, ultimately saves storage resources, and further improves the image encoding efficiency. For example, the compressed file generated by the current entropy coding consumes a large amount of storage resources on the server side, and the embodiment of the present application can be applied to the efficient transcoding of the entropy coded image. The embodiment of the present application can also be applied to the server side to transcode the compressed file generated by the entropy coding, improve the image encoding performance, and ultimately save storage resources.

[0098] Next, based on the aforementioned system architecture, image encoding device, and image decoding device, the image processing method provided by the embodiment of the present application is described. Figure 3 As shown, it is a schematic diagram of an interaction process between the image encoding device and the image decoding device in an embodiment of the present application, wherein the following steps 301 to 303 can be performed by the image encoding device (hereinafter referred to as the encoding end), and the following steps 311 to 313 can be performed by the image decoding device (hereinafter referred to as the decoding end), which mainly includes the following processes:

[0099] 301. Obtain at least two transform coefficient blocks, each of the at least two transform coefficient blocks comprising: at least two transform coefficients, wherein each transform coefficient in a transform coefficient block corresponds to a different frequency point.

[0100] Among them, each of the at least two transform coefficient blocks includes: at least two transform coefficients, each transform coefficient in a transform coefficient block corresponds to a different frequency point, that is, for each transform coefficient block, all transform coefficients in the transform coefficient block have different frequency points, and each transform coefficient includes frequency point information. In addition to the frequency point, each transform coefficient can also include position information.

[0101] The frequency point of a transform coefficient can be represented by (fx, fy), where fx and fy represent the two-dimensional frequency coordinate values ​​corresponding to the frequency point. The position information of a transform coefficient can be represented by (bx, by), where bx and by represent the two-dimensional position coordinate values ​​of the transform coefficient. Each transform coefficient in the transform coefficient block can be represented by In addition, frequency indexes can be set for different frequency points, for example, the frequency index is fs.

[0102] In some embodiments of the present application, step 301 of acquiring at least two transform coefficient blocks includes:

[0103] A1. Divide an original image region into at least two image blocks, where the original image region includes: a pixel image region or a residual image region;

[0104] A2. Transform and quantize at least two image blocks to obtain at least two transform coefficient blocks.

[0105] Among them, the original image area (also referred to as the original image for short) can be the entire image or a rectangular image area, or an image area of ​​other shapes. The original image area is the image to be encoded (or called to be processed). The encoding end divides the image or image area to be processed into at least two image blocks of the same preset size. For example, the size of the sub-image block of the preset size can be 4×4, 8×8, 16×16, 32×32, 64×64, 128×128 and 256×256, etc. In the embodiment of the present application, the image block is obtained by dividing the original image, or the image block is transmitted to the encoding end by an external device of the encoding end. Among them, the image block can also be called a sub-image, or a sub-image block, etc.

[0106] It should be noted that, in the embodiment of the present application, the image data included in the image block may include a brightness component, a chrominance component, an R component, a G component, a B component, etc.

[0107] In an embodiment of the present application, after the encoder divides the original image area to be processed into image blocks of preset size, each image block may be transformed and quantized to obtain at least two transform coefficient blocks. For example, after the encoder obtains at least two image blocks, the encoder obtains at least two transform coefficient blocks based on the at least two image blocks, wherein each image block may generate a transform coefficient block, and the image block and the transform coefficient block are one-to-one corresponding. The transform coefficient block is a data block composed of multiple transform coefficients, and for example, the transform coefficient block may be obtained by performing a discrete cosine transform (DCT) on the image block. For example, in the DCT transform, f m It represents the transform coefficient corresponding to a certain frequency point (for example, frequency point m), k = 0, ..., n-1, k represents the position of the pixel point, x k represents the pixel value at position k,

[0108] In the embodiment of the present application, a transform coefficient block can be generated by transforming and quantizing an image block, so that the transform coefficient block can be encoded, image coding information can be obtained, and image compression processing can be achieved.

[0109] In some embodiments of the present application, step 301 of acquiring at least two transform coefficient blocks includes:

[0110] B1. Obtaining transform coefficient coding information of at least two image blocks;

[0111] B2. Perform entropy decoding on the transform coefficient coding information to obtain at least two transform coefficient blocks.

[0112] The transform coefficient coding information may be transform coefficient coding information corresponding to at least two image blocks. For example, the at least two image blocks may be image blocks in a JPEG file. The image processing device may use a Huffman decoding method to entropy decode the transform coefficient coding information of all image blocks in the JPEG file to obtain at least two transform coefficient blocks corresponding to the at least two image blocks. In the embodiment of the present application, the transform coefficient coding information is entropy decoded to obtain at least two transform coefficient blocks, and the transform coefficient blocks are encoded to obtain image coding information, thereby realizing compression processing of the image.

[0113] 302. Determine transform coefficients in at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order.

[0114] 303. Encode each transform coefficient corresponding to a frequency point according to a preset coefficient scanning order to obtain image coding information.

[0115] In an embodiment of the present application, the frequency point scanning order may be preset or pre-configured in the encoding end, and the frequency point scanning order refers to the order of scanning frequency points. Each of the at least two transform coefficient blocks includes: at least two transform coefficients, wherein each transform coefficient in a transform coefficient block corresponds to a different frequency point. Therefore, a frequency point scanning order may be set for all frequency points, and a frequency point is determined by the frequency point scanning order. Similarly, different frequency points may be determined in sequence by the frequency point scanning order. In step 302 and step 303, the encoding processing flow for one frequency point is taken as an example. The encoding processing flow for other frequency points may be implemented based on step 302 and step 303 and the detailed flow included in step 302 and step 303.

[0116] In an embodiment of the present application, at least two transform coefficient blocks may be N transform coefficient blocks, and the transform coefficient in at least two transform coefficient blocks corresponding to one frequency point may be a transform coefficient in each transform coefficient block in the N transform coefficient blocks corresponding to one frequency point. Then, one frequency point corresponds to N transform coefficients, and these N transform coefficients are respectively derived from each transform coefficient block in the N transform coefficient blocks. The value of N may be a positive integer, for example, N is an integer greater than or equal to 2.

[0117] In an embodiment of the present application, the coefficient scanning order may be preset or pre-configured in the encoding end. The coefficient scanning order refers to the scanning order of the transform coefficients. A transform coefficient corresponding to a frequency point is determined according to the preset coefficient scanning order, and then the determined transform coefficient is encoded. Based on the coefficient scanning order in step 303, other transform coefficients corresponding to a frequency point can be determined in sequence, and then the transform coefficients determined in sequence are encoded, and finally image encoding information can be obtained.

[0118] In the embodiment of the present application, after the encoding end obtains at least two transform coefficient blocks, multiple transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks can be encoded. For example, the lossless encoding methods that can be used include entropy coding, Huffman coding, etc. It is not limited that the entropy coding in the embodiment of the present application uses a deep network, and the structure of the deep network is not limited.

[0119] After all transform coefficients corresponding to a certain frequency point are encoded, the next frequency point is determined according to the preset frequency point scanning order, and then multiple transform coefficients corresponding to the next frequency point are encoded, and all frequency points are obtained in sequence according to the frequency point scanning order. When the transform coefficients corresponding to all frequency points are losslessly encoded, the encoder can generate image coding information. After the encoder generates the image coding information, the encoder can write the image coding information into the bitstream, and then send the bitstream to the decoder.

[0120] Among them, the frequency point scanning order is the order of selecting frequency points during encoding, and the frequency point scanning order is referred to as "preset order" or "preset order one" in subsequent embodiments. There are many ways to implement the frequency point scanning order, for example, it can be a processing order obtained by zig-zag scanning, and other scanning methods can also be used to obtain the processing order, such as row-by-row scanning, column-by-column scanning, diagonal scanning, horizontal reverse scanning and vertical reverse scanning. Among them, row-by-row scanning refers to scanning continuously from the first row of the image to the last row, and column-by-column scanning refers to scanning continuously from the first column of the image to the last column. Diagonal scanning refers to scanning continuously from the first row of the image to the last row in a diagonal manner. Horizontal reverse scanning refers to scanning continuously from the last row of the image to the first row in the horizontal direction, and vertical reverse scanning refers to scanning continuously from the last row of the image to the first row in the vertical direction. The coefficient scanning order is the order of selecting the transform coefficients during encoding, and the coefficient scanning order is referred to as "preset order" or "preset order two" in subsequent embodiments. There are many ways to implement the coefficient scanning order, for example, it can be a processing order obtained by zig-zag scanning, and other scanning methods can also be used to obtain the processing order, such as row-by-row scanning, column-by-column scanning, diagonal scanning, horizontal reverse scanning, and vertical reverse scanning.

[0121] The frequency point scanning order and the coefficient scanning order are the same image scanning order; or the frequency point scanning order and the coefficient scanning order are different image scanning orders, which is not limited here.

[0122] An example is given below. First, all transform coefficients corresponding to the frequency point (fx=0, fy=0) are obtained according to the frequency point scanning order. Next, a transform coefficient corresponding to the frequency point (fx=0, fy=0) is obtained according to the coefficient scanning order. The transform coefficient is encoded. Then, the next transform coefficient corresponding to the frequency point (fx=0, fy=0) is obtained according to the coefficient scanning order. The next transform coefficient is encoded until all transform coefficients corresponding to the frequency point (fx=0, fy=0) are encoded according to the coefficient scanning order. After encoding all transform coefficients corresponding to the frequency point (fx=0, fy=0), all transform coefficients corresponding to the frequency point (fx=0, fy=1) are obtained. All transform coefficients corresponding to the frequency point (fx=0, fy=1) are encoded. The transform coefficients corresponding to all frequency points are determined in sequence according to the frequency point scanning order for encoding. When the transform coefficients corresponding to all frequency points are encoded, the encoding end can output the image encoding information.

[0123] In some embodiments of the present application, step 302 determines the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order, including:

[0124] C1. Obtain at least two two-dimensional subband coefficient arrays from at least two transform coefficient blocks, wherein one two-dimensional subband coefficient array includes transform coefficients corresponding to the same frequency points in at least two transform coefficient blocks, and different two-dimensional subband coefficient arrays include transform coefficients corresponding to different frequency points.

[0125] C2. Determine a first two-dimensional sub-band coefficient array among at least two two-dimensional sub-band coefficient arrays according to a preset frequency point scanning order.

[0126] Step 303 encodes each transform coefficient corresponding to a frequency point according to a preset coefficient scanning order, including:

[0127] C3. determining a first transform coefficient in a first two-dimensional subband coefficient array according to a preset coefficient scanning order;

[0128] C4. Perform entropy encoding on the first transform coefficient according to a context area corresponding to the first transform coefficient.

[0129] Among them, the encoding end can arrange at least two transform coefficient blocks according to different frequency points, so as to obtain at least two two-dimensional sub-band coefficient arrays. The same two-dimensional sub-band coefficient array includes transform coefficients in the transform coefficient blocks corresponding to the same frequency points, and different two-dimensional sub-band coefficient arrays have different frequency points, that is, the encoding end can attribute the transform coefficients in at least two transform coefficient blocks corresponding to the same frequency point to one two-dimensional sub-band coefficient array, so as many frequency points as there are, the two-dimensional sub-band coefficient arrays corresponding to multiple frequency points can be obtained.

[0130] An example is given below to obtain the transform coefficient value of a specified frequency point in an 8x8 transform coefficient block in an image to form a two-dimensional subband coefficient array of the frequency point. After obtaining the two-dimensional subband coefficient array obtained by obtaining the transform coefficient values ​​of the frequency points in all 8x8 transform coefficient blocks in the image, similarly, obtain the two-dimensional subband coefficient arrays of the remaining frequency points. An 8x8 transform coefficient block has a total of 64 frequency points, so 64 two-dimensional subband coefficient arrays of the current encoded image can be obtained.

[0131] The encoding end combines the transform coefficients of at least two transform coefficient blocks according to different frequency points, and can obtain at least two two-dimensional sub-band coefficient arrays. According to the preset frequency point scanning order, the first two-dimensional sub-band coefficient array is obtained from the at least two two-dimensional sub-band coefficient arrays. The first two-dimensional sub-band coefficient array is a two-dimensional sub-band coefficient array in the at least two two-dimensional sub-band coefficient arrays. The first two-dimensional sub-band coefficient array is a two-dimensional sub-band coefficient array obtained according to the frequency point scanning order. The first two-dimensional sub-band coefficient array includes multiple transform coefficients. The encoding end can determine the first transform coefficient in the first two-dimensional sub-band coefficient array according to the preset coefficient scanning order. The first transform coefficient can be any one of the transform coefficients in the first two-dimensional sub-band coefficient array. The encoding end can perform lossless encoding on each transform coefficient in the first two-dimensional sub-band coefficient array, for example, the first transform coefficient can be entropy encoded according to the context area corresponding to the first transform coefficient. In the embodiment of the present application, by encoding the transform coefficients in the two-dimensional sub-band coefficient array, image encoding information can be obtained to achieve compression processing of the image.

[0132] The frequency point scanning order may be the order of scanning the two-dimensional subband coefficient array, and the coefficient scanning order is the scanning order of the transform coefficients in the two-dimensional subband coefficient array. The frequency point scanning order is referred to as "preset order one" in subsequent embodiments, and the coefficient scanning order is referred to as "preset order two" in subsequent embodiments.

[0133] In some embodiments of the present application, step 302 determines the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order, including:

[0134] D1. Obtain at least two two-dimensional subband coefficient arrays from at least two transform coefficient blocks, wherein one two-dimensional subband coefficient array includes transform coefficients corresponding to the same frequency points in at least two transform coefficient blocks, and different two-dimensional subband coefficient arrays include transform coefficients corresponding to different frequency points;

[0135] D2. Combining at least two two-dimensional sub-band coefficient arrays according to a preset frequency point scanning order to obtain a three-dimensional sub-band coefficient array;

[0136] D3. Determine a first two-dimensional sub-band coefficient array in the three-dimensional sub-band coefficient array according to a preset three-dimensional sub-band coefficient scanning order.

[0137] Step 303 encodes each transform coefficient corresponding to a frequency point according to a preset coefficient scanning order, including:

[0138] D4, determining a first transform coefficient in a first two-dimensional subband coefficient array according to a preset coefficient scanning order;

[0139] D5. Perform entropy encoding on the first transform coefficient according to a context area corresponding to the first transform coefficient.

[0140] Among them, the difference between the scenario of steps D1 to D5 and the scenario of the aforementioned steps C1 to C4 is that in the scenario of steps D1 to D5, a three-dimensional subband coefficient array needs to be generated, and the first transform coefficient in the first two-dimensional field coefficient array in the three-dimensional subband coefficient array is encoded, while in the scenario of steps C1 to C4, no three-dimensional subband coefficient array is generated.

[0141] Specifically, steps D1 to D5 are described below. After generating at least two two-dimensional sub-band coefficient arrays, the encoding end combines the at least two two-dimensional sub-band coefficient arrays according to a preset frequency point scanning order to obtain a three-dimensional sub-band coefficient array, that is, one frequency point corresponds to one two-dimensional sub-band coefficient array, and the two-dimensional sub-band coefficient arrays corresponding to all frequency points can be combined together according to the preset frequency point scanning order to obtain a three-dimensional sub-band coefficient array.

[0142] The three-dimensional subband coefficient array includes multiple two-dimensional subband coefficient arrays. The encoding end first determines a two-dimensional subband coefficient array from the three-dimensional subband coefficient array according to a preset three-dimensional subband coefficient scanning order, for example, determines a first two-dimensional subband coefficient array, and the first two-dimensional subband coefficient array is a two-dimensional subband coefficient array in the three-dimensional subband coefficient array. Among them, the three-dimensional subband coefficient scanning order is the order in which the three-dimensional subband coefficient array is scanned. After the encoding end determines the first two-dimensional subband coefficient array, the encoding end can perform entropy encoding on each transform coefficient in the first two-dimensional subband coefficient array according to the preset coefficient scanning order. In the embodiment of the present application, by encoding the transform coefficients in the two-dimensional subband coefficient array, image encoding information can be obtained to achieve compression processing of the image.

[0143] In some embodiments of the present application, step 302 determines the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order, and encodes each transform coefficient corresponding to a frequency point according to the preset coefficient scanning order, including:

[0144] E1. Sort at least two transform coefficient blocks according to a preset frequency point scanning order to obtain at least two sorted transform coefficient blocks;

[0145] E2. Obtain multiple two-dimensional sub-band coefficient arrays corresponding to different frequency points from the at least two sorted transform coefficient blocks;

[0146] E3. Combining multiple two-dimensional sub-band coefficient arrays corresponding to different frequency points to obtain a three-dimensional sub-band coefficient array;

[0147] E4. Determine a first two-dimensional sub-band coefficient array from the three-dimensional sub-band coefficient array according to a preset three-dimensional sub-band coefficient scanning order.

[0148] Step 303 encodes each transform coefficient corresponding to a frequency point according to a preset coefficient scanning order, including:

[0149] E5. Determine a first transform coefficient in a first two-dimensional subband coefficient array according to a preset coefficient scanning order;

[0150] E6. Perform entropy encoding on the first transform coefficient according to the context area corresponding to the first transform coefficient.

[0151] The difference between the scenario of steps E1 to E6 and the scenario of steps D1 to D5 is that the method of generating the three-dimensional subband coefficient array in the scenario of steps E1 to E6 is different from the method of generating the three-dimensional subband coefficient array in steps D1 to D5. That is, in the scenario of steps E1 to E6, at least two transform coefficient blocks are first sorted, and then multiple two-dimensional subband coefficient arrays corresponding to different frequency points are determined according to different frequency points, and the multiple two-dimensional subband coefficient arrays corresponding to different frequency points are combined to obtain the three-dimensional subband coefficient array.

[0152] Specifically, steps E1 to E6 are described below. After the encoder obtains multiple two-dimensional subband coefficient arrays, they are combined to obtain a three-dimensional subband coefficient array. The three-dimensional subband coefficient array includes multiple two-dimensional subband coefficient arrays. The encoder first determines a two-dimensional subband coefficient array from the three-dimensional subband coefficient array according to a preset three-dimensional subband coefficient scanning order, for example, determines a first two-dimensional subband coefficient array, and the first two-dimensional subband coefficient array is a two-dimensional subband coefficient array in the three-dimensional subband coefficient array. Among them, the three-dimensional subband coefficient scanning order is the order in which the three-dimensional subband coefficient array is scanned. After the encoder determines the first two-dimensional subband coefficient array, the encoder can perform entropy encoding on each transform coefficient in the first two-dimensional subband coefficient array according to the preset coefficient scanning order. In the embodiment of the present application, by encoding the transform coefficients in the two-dimensional subband coefficient array, image encoding information can be obtained to achieve compression processing of the image.

[0153] An example is given below. The encoding end sorts each transform coefficient in the transform coefficient block (hereinafter referred to as the coefficient block) according to the frequency point scanning order, and then obtains the coefficients of the same frequency point to combine them to obtain a three-dimensional sub-band coefficient array. For example, the transform coefficient block can be an 8x8 transform coefficient block. Arrange each transform coefficient in the 8x8 transform coefficient block according to the frequency point scanning order to obtain a sorted 8x8 transform coefficient block. Obtain the transform coefficient value of the specified frequency point in the 8x8 transform coefficient block in the image to form a two-dimensional sub-band coefficient array of the frequency point, and combine the two-dimensional sub-band coefficient arrays corresponding to multiple frequency points to obtain a three-dimensional sub-band coefficient array. After obtaining the three-dimensional sub-band coefficient array, the encoding end can also encode each transform coefficient in the three-dimensional sub-band coefficient array. For details, see the example of the encoding process in the subsequent embodiments.

[0154] Further, in some embodiments of the present application, the three-dimensional sub-band coefficient scanning order includes:

[0155] A preset order of changing the horizontal position index; or,

[0156] The change order of the preset vertical position index; or,

[0157] The preset frequency index changes in order.

[0158] Each transform coefficient in the three-dimensional subband coefficient array has a horizontal position index, a vertical position index and a frequency index. For example, each transform coefficient can be expressed as C(bx, by, fs), where bx is the horizontal position index, by is the vertical position index, and fs is the frequency index corresponding to the frequency point. The three-dimensional subband coefficient scanning order is the order in which the three-dimensional subband coefficient array is scanned. For example, the three-dimensional subband coefficient scanning order determines multiple two-dimensional subband coefficient arrays in the three-dimensional subband coefficient array from large to small (or small to large) in the order of change of the horizontal position index. After each two-dimensional subband coefficient array is determined, a transform coefficient in the two-dimensional subband coefficient array is determined according to the aforementioned coefficient scanning order, that is, the three-dimensional subband coefficient scanning order can be used to indicate that the two-dimensional subband coefficient arrays corresponding to different horizontal position indexes in the three-dimensional subband coefficient array are encoded in sequence.

[0159] For example, the three-dimensional subband coefficient scanning order determines multiple two-dimensional subband coefficient arrays in the three-dimensional subband coefficient array in the order of change of the vertical position index from large to small (or small to large). After each two-dimensional subband coefficient array is determined, a transform coefficient in the two-dimensional subband coefficient array is determined according to the aforementioned coefficient scanning order. That is, the three-dimensional subband coefficient scanning order can be used to indicate that the two-dimensional subband coefficient arrays corresponding to different vertical position indices in the three-dimensional subband coefficient array are encoded in sequence.

[0160] For another example, the three-dimensional subband coefficient scanning order determines multiple two-dimensional subband coefficient arrays in the three-dimensional subband coefficient array in the order of change of the frequency index from large to small (or small to large), and after each two-dimensional subband coefficient array is determined, a transform coefficient in the two-dimensional subband coefficient array is determined according to the aforementioned coefficient scanning order, that is, the three-dimensional subband coefficient scanning order can be used to indicate that the two-dimensional subband coefficient arrays corresponding to different frequency indices in the three-dimensional subband coefficient array are encoded in sequence. The specific implementation method of the three-dimensional subband coefficient scanning order can be determined according to the application scenario, and is not limited at this time.

[0161] There are many ways to implement the scanning order of the three-dimensional sub-band coefficients, and examples are as follows:

[0162] The three-dimensional subband coefficient array is represented as C(bx,by,fs), (bx,by) represents the horizontal position index and vertical position index of the transform coefficient block, bx=0,...,M-1, by=0,...,N-1, Wherein, M is the maximum number of image blocks in the horizontal direction with a single image block as a unit, N is the maximum number of image blocks in the vertical direction with a single image block as a unit, W represents the width value of the current image, H represents the height value of the current image, subW represents the width value of the image block in the current image, and subH represents the height value of the image block in the current image. (fx, fy) represents the frequency point of each transform coefficient in the transform coefficient block, and fs represents the frequency index corresponding to each frequency point, where fx = 0, .., subW-1, fy = 0, .., subH-1.

[0163] Method 1: When processing each transform coefficient in the three-dimensional subband coefficient array C(bx,by,fs), the two-dimensional coefficient subband arrays of fs=0 to subW×subH-1 layers are processed in turn. When processing the two-dimensional coefficient subband array of each layer, each transform coefficient in the two-dimensional coefficient subband array C(bx,by) is processed according to the aforementioned coefficient scanning order.

[0164] Method 2: When processing each transform coefficient in the three-dimensional subband coefficient array C(bx,by,fs), the two-dimensional coefficient subband arrays from bx=0 to M-1 layers are processed in turn. When processing the two-dimensional coefficient subband array of each layer, each transform coefficient in the two-dimensional coefficient subband array C(by,fs) is processed according to the aforementioned coefficient scanning order.

[0165] Method three: When processing each transform coefficient in the three-dimensional subband coefficient array C(bx,by,fs), the two-dimensional coefficient subband arrays of by=0 to N-1 layers are processed in turn. When processing the two-dimensional coefficient subband array of each layer, each transform coefficient in the two-dimensional coefficient subband array C(bx,fs) is processed according to a preset coefficient scanning order.

[0166] It should be noted that the preset coefficient scanning order may be a zig-zag scanning order, a row-by-row scanning order, a column-by-column scanning order, and the like.

[0167] In some embodiments of the present application, step 302 determines the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order, including:

[0168] F1. Determine, according to a preset frequency point scanning order, a first two-dimensional sub-band coefficient array consisting of transform coefficients corresponding to the same frequency points in at least two transform coefficient blocks.

[0169] Step 303 encodes each transform coefficient corresponding to a frequency point according to a preset coefficient scanning order, including:

[0170] F2. determining a first transform coefficient in a first two-dimensional subband coefficient array according to a preset coefficient scanning order;

[0171] F3. Perform entropy encoding on the first transform coefficient according to a context area corresponding to the first transform coefficient.

[0172] Specifically, after the encoding end obtains at least two transform coefficient blocks, the encoding end can also obtain a first two-dimensional sub-band coefficient array consisting of transform coefficients corresponding to the same frequency point from at least two transform coefficient blocks according to a preset frequency point scanning order. The encoding end does not need to combine a three-dimensional sub-band coefficient array. After the encoding end determines the first two-dimensional sub-band coefficient array, the encoding end can encode each transform coefficient in the first two-dimensional sub-band coefficient array respectively according to a preset coefficient scanning order. For example, the encoding end determines the first transform coefficient in the first two-dimensional sub-band coefficient array according to a preset coefficient scanning order, and entropy encodes the first transform coefficient according to the context area corresponding to the first transform coefficient. In the embodiment of the present application, by encoding the transform coefficients in the two-dimensional sub-band coefficient array, image coding information can be obtained to achieve compression processing of the image.

[0173] As an example, the encoding end first scans each transform coefficient in at least two transform coefficient blocks according to the frequency point scanning order, for example, determines a frequency point (the frequency index of the frequency point is fs) according to the frequency point scanning order, obtains a transform coefficient in a transform coefficient block corresponding to the frequency index fs specified in at least two transform coefficient blocks, and forms a two-dimensional subband coefficient array of a frequency point. After determining a two-dimensional subband coefficient array, each transform coefficient in the two-dimensional subband coefficient array can be processed according to a preset coefficient scanning order, for example, each transform coefficient in the two-dimensional subband coefficient array can be encoded by using entropy coding to obtain image coding information.

[0174] In some embodiments of the present application, step 302 determines the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order, including:

[0175] G1. Determine transform coefficients in at least two transform coefficient blocks included in a frequency sub-band according to a preset frequency point scanning order, wherein the transform coefficients in at least two transform coefficient blocks included in a frequency sub-band correspond to the same frequency point.

[0176] Step 303 encodes each transform coefficient corresponding to a frequency point according to a preset coefficient scanning order, including:

[0177] G2. Determine a first transform coefficient included in a frequency subband according to a preset coefficient scanning order;

[0178] G3. Perform entropy encoding on the first transform coefficient according to a context area corresponding to the first transform coefficient.

[0179] Among them, in the aforementioned application scenario, the encoding end needs to generate a two-dimensional subband coefficient array or a three-dimensional subband coefficient array. In step G1, the encoding end can also adopt a method of not generating a two-dimensional subband coefficient array or a three-dimensional subband coefficient array. The encoding end determines the transform coefficients in at least two transform coefficient blocks included in a frequency subband according to a preset frequency point scanning order. The transform coefficients in at least two transform coefficient blocks included in a frequency subband correspond to the same frequency point. The encoding end encodes the transform coefficients in at least two transform coefficient blocks corresponding to a frequency subband according to a preset coefficient scanning order. For example, the first transform coefficient included in a frequency subband is determined according to a preset coefficient scanning order. The first transform coefficient can be any one of the multiple transform coefficients included in a frequency subband. The first transform coefficient is entropy encoded according to the context area corresponding to the first transform coefficient. In the embodiment of the present application, by encoding the transform coefficients in the frequency subband, image encoding information can be obtained to achieve compression processing of the image.

[0180] In some embodiments of the present application, the specific process of entropy encoding the aforementioned first transform coefficient by the encoding end is described, and the encoding process of other transform coefficients in the first two-dimensional sub-band coefficient array or the encoding process of other transform coefficients included in a frequency sub-band can be implemented according to the entropy encoding process of the first transform coefficient. For example, the entropy encoding of the first transform coefficient according to the context area corresponding to the first transform coefficient in the aforementioned steps C4, D5, E6, F3, and G3 includes:

[0181] H1. Acquire context information of the first transform coefficient from a context area corresponding to the first transform coefficient, wherein the context information of the first transform coefficient includes: an encoded transform coefficient in the context area corresponding to the first transform coefficient.

[0182] Specifically, the context area corresponding to the first transform coefficient can be preset. For example, the context area corresponding to the first transform coefficient is determined according to the horizontal position index, vertical position index and frequency index of the current first transform coefficient. The encoded transform coefficient in the context area corresponding to the first transform coefficient is a transform coefficient that has been entropy encoded before the first transform coefficient is entropy encoded. Therefore, the context information of the first transform coefficient is obtained from the context area corresponding to the first transform coefficient.

[0183] In some embodiments of the present application, the context area corresponding to the first transform coefficient includes at least one of the following: a spatial neighborhood of the first transform coefficient, a subband neighborhood of the first transform coefficient, and a three-dimensional neighborhood of the first transform coefficient.

[0184] Among them, the context area corresponding to the first transform coefficient has multiple implementation methods, for example, it can include the spatial neighborhood of the first transform coefficient, or the subband neighborhood of the first transform coefficient, or the three-dimensional neighborhood of the first transform coefficient. Among them, the spatial neighborhood of the first transform coefficient refers to the adjacent area in the same horizontal space or vertical space as the first transform coefficient, the subband neighborhood of the first transform coefficient refers to the adjacent area in the same subband as the first transform coefficient, and the three-dimensional neighborhood refers to the adjacent area in the same three-dimensional space as the first transform coefficient. The spatial neighborhood of the first transform coefficient, the subband neighborhood of the first transform coefficient, or the three-dimensional neighborhood of the first transform coefficient can include the encoded transform coefficient, so that the encoding end can obtain the context information of the first transform coefficient.

[0185] H2. Estimate the first transform coefficient according to the context information of the first transform coefficient to obtain probability distribution information of the first transform coefficient.

[0186] H3. Perform arithmetic coding on the first transform coefficient according to the probability distribution information of the first transform coefficient.

[0187] In an embodiment of the present application, after the encoder obtains the context information of the first transform coefficient, the encoder performs entropy coding on the first transform coefficient according to the context information of the first transform coefficient to obtain image coding information corresponding to the first transform coefficient. The entropy coding process performed by the encoder may include probability estimation and arithmetic coding. Specifically, the encoder performs probability estimation on the first transform coefficient according to the context information of the first transform coefficient to obtain the probability distribution of the first transform coefficient, and then the encoder performs arithmetic coding on the first transform coefficient according to the probability distribution of the first transform coefficient to obtain the image coding information corresponding to the first transform coefficient.

[0188] As an example, when processing each two-dimensional subband coefficient array, each transform coefficient in each two-dimensional subband coefficient array is processed in the coefficient scanning order. The processed coefficients in the preset context area are selected as contexts, and the context information is input into the probability estimation network to estimate the probability distribution of the current coefficient, thereby obtaining the probability distribution of the current coefficient, and the current coefficient is arithmetic-coded using the probability distribution to obtain the coding information of the current coefficient. The preset context area may include the spatial neighborhood and / or subband neighborhood of the coefficient.

[0189] Specifically, the method for estimating the probability distribution of the current coefficient includes: the probability estimation network directly estimates the probability of the value within the value range according to the dynamic value range of the coefficient, and obtains a probability vector, wherein the probability vector includes the probability value of each value within the value range, wherein the sum of all probability values ​​is 1. For another example, the method for estimating the probability distribution of the current coefficient includes: using a probability distribution model to obtain a probability vector. For example, a single Gaussian model (Gaussiansingle model, GSM) or a Gaussian mixture model (Gaussian mixture model, GMM) is used for modeling, and the model parameters are estimated using a probability estimation network, and the model parameters are substituted into the probability distribution model to obtain a probability vector.

[0190] An example is given below: after obtaining the encoded coefficients in the three-dimensional neighborhood of the first transform coefficient as context information, the context information is input into the probability estimation network, the probability distribution of the first transform coefficient is obtained by probabilistically estimating the first transform coefficient, and the current first transform coefficient is arithmetically encoded using the probability distribution to obtain the encoding information of the first transform coefficient. The probability distribution estimation method used in the probability estimation network includes a probability distribution estimation method based on a deep learning network, such as a recurrent neural network (RNN) and a pixel convolutional neural network (PixelCNN).

[0191] In some embodiments of the present application, the specific process of entropy encoding the aforementioned first transform coefficient by the encoding end is described, and the encoding process of other transform coefficients in the first two-dimensional sub-band coefficient array or the encoding process of other transform coefficients included in a frequency sub-band can be implemented according to the entropy encoding process of the first transform coefficient. For example, the entropy encoding of the first transform coefficient according to the context area corresponding to the first transform coefficient in the aforementioned steps C4, D5, E6, F3, and G3 includes:

[0192] I1. When the first transform coefficient is a plurality of transform coefficients, context information of the plurality of transform coefficients is obtained from context areas corresponding to the plurality of transform coefficients, wherein the context information of the plurality of transform coefficients includes: encoded transform coefficients in the context areas corresponding to the plurality of transform coefficients;

[0193] I2. Estimating the multiple transform coefficients in parallel according to the context information of the multiple transform coefficients to obtain probability distribution information of the multiple transform coefficients;

[0194] I3. Perform arithmetic coding on multiple transform coefficients in parallel according to their probability distribution information.

[0195] Among them, the application scenario of step I1 to step H3 is different from the application scenario of the aforementioned step I1 to step I3 in that step I1 to step I3 is for the case where the first transform coefficient is a plurality of transform coefficients, at which time the probability distribution estimation and parallel arithmetic coding can be performed in parallel. Among them, the encoding end can obtain a plurality of transform coefficients, and the encoding end performs entropy coding on the plurality of transform coefficients in parallel according to the context information of the plurality of transform coefficients. Through parallel entropy coding, the image coding efficiency can be improved. An example is given below, when the encoding end obtains the probability distribution of the coefficient, it can also perform probability estimation on the probability distribution of the plurality of transform coefficients in parallel, that is, obtain the probability distribution of the plurality of transform coefficients at one time, and then perform arithmetic coding on the plurality of transform coefficients in parallel.

[0196] In some embodiments of the present application, the context area corresponding to the multiple transform coefficients includes at least one of the following: a spatial neighborhood of the multiple transform coefficients, a sub-band neighborhood of the multiple transform coefficients, and a three-dimensional neighborhood of the multiple transform coefficients.

[0197] Among them, the context area corresponding to multiple transform coefficients has multiple implementation methods, for example, it can include the spatial neighborhood of multiple transform coefficients, or the subband neighborhood of multiple transform coefficients, or the three-dimensional neighborhood of multiple transform coefficients. Among them, the spatial neighborhood of multiple transform coefficients refers to the adjacent area in the same horizontal space or vertical space as the multiple transform coefficients, the subband neighborhood of multiple transform coefficients refers to the adjacent area in the same subband as the multiple transform coefficients, and the three-dimensional neighborhood refers to the adjacent area in the same three-dimensional space as the first transform coefficient. The spatial neighborhood of multiple transform coefficients, the subband neighborhood of multiple transform coefficients, or the three-dimensional neighborhood of multiple transform coefficients can include encoded transform coefficients, so that the encoding end can obtain the context information of multiple transform coefficients.

[0198] In some embodiments of the present application, step 302 determines the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order, including:

[0199] J1. Obtain at least two two-dimensional subband coefficient arrays from at least two transform coefficient blocks, wherein one two-dimensional subband coefficient array includes transform coefficients corresponding to the same frequency points in at least two transform coefficient blocks, and different two-dimensional subband coefficient arrays include transform coefficients corresponding to different frequency points;

[0200] J2. Determine at least two sub-band images corresponding to at least two two-dimensional sub-band coefficient arrays according to a preset frequency point scanning order.

[0201] Step 303 encodes each transform coefficient corresponding to a frequency point according to a preset coefficient scanning order, including:

[0202] J3. Encode the transform coefficients in at least two sub-band images according to a preset coefficient scanning order.

[0203] The encoding end may arrange at least two transform coefficient blocks according to different frequency points, thereby obtaining at least two two-dimensional sub-band coefficient arrays. The same two-dimensional sub-band coefficient array includes transform coefficients in the transform coefficient blocks corresponding to the same frequency points, and different two-dimensional sub-band coefficient arrays have different frequency points, that is, the encoding end may attribute the transform coefficients in at least two transform coefficient blocks corresponding to the same frequency point to one two-dimensional sub-band coefficient array, so as many frequency points as there are, two-dimensional sub-band coefficient arrays corresponding to multiple frequency points may be obtained.

[0204] After the encoding end obtains at least two two-dimensional sub-band coefficient arrays, the encoding end adopts a method of lossless encoding of the sub-band image, using the at least two two-dimensional sub-band coefficient arrays as at least two sub-band images, and performing image lossless encoding on the transform coefficients in the at least two sub-band images according to a preset frequency point scanning order. The encoding end can encode the two-dimensional sub-band coefficient array as a sub-band image during T-shirt lossless encoding, thereby simplifying the implementation process of lossless encoding.

[0205] From the foregoing description of the image processing method, it can be seen that in the embodiment of the present application, at least two transform coefficient blocks can be obtained, so that for the transform coefficients corresponding to the same frequency point in at least two transform coefficient blocks, it is only necessary to encode them in the frequency point scanning order to obtain the image coding information. Therefore, multiple transform coefficients corresponding to the same frequency point can be encoded at the same time, thereby improving the image compression efficiency, reducing the storage cost of the image coding information, and ultimately saving storage resources.

[0206] In the embodiment of the present application, after the encoding end can generate the image coding information, the encoding end can send it to the decoding end through the image transmission channel, for example, the encoding end sends a compressed code stream, and the compressed code stream includes the image coding information. The image coding information is obtained by the encoding end after encoding at least two transform coefficient blocks, and the decoding end needs to use the image processing method corresponding to the encoding end for decoding. Specifically, the decoding end performs subsequent steps 311 to 313.

[0207] 311. Obtain image coding information.

[0208] In the embodiment of the present application, the decoding end first obtains the image coding information from the encoding end. For example, the decoding end receives a compressed code stream from the encoding end, and then parses the compressed code stream to obtain the image coding information.

[0209] 312. Determine a transform coefficient to be decoded corresponding to a frequency point in the image coding information according to a preset frequency point scanning order.

[0210] 313. Decode the to-be-decoded transform coefficients corresponding to a frequency point according to a preset coefficient scanning order to obtain at least two transform coefficient blocks.

[0211] Each of the at least two transform coefficient blocks includes: at least two transform coefficients, and each transform coefficient in a transform coefficient block corresponds to a different frequency point.

[0212] In an embodiment of the present application, after the decoding end obtains the image coding information, the transform coefficients to be decoded corresponding to the same frequency point in the image coding information are decoded, wherein the transform coefficients to be decoded can also be referred to as transform coefficients to be decoded. For example, the decoding methods that can be used include entropy decoding, Huffman decoding, etc. After all the transform coefficients corresponding to a certain frequency point are obtained, the next frequency point is determined according to a preset frequency point scanning order, and then multiple transform coefficients corresponding to the next frequency point are obtained by decoding, and all frequency points are obtained in sequence according to the frequency point scanning order. After the transform coefficients corresponding to all frequency points are obtained by decoding, the decoding end can generate at least two transform coefficient blocks, and each transform coefficient block in the at least two transform coefficient blocks includes: at least two transform coefficients, wherein each transform coefficient in a transform coefficient block corresponds to a different frequency point. For the description of the transform coefficient block, the frequency point scanning order and the coefficient scanning order, please refer to the example in the aforementioned encoding end.

[0213] In some embodiments of the present application, when the encoding end performs the encoding shown in steps C1 to C4, the decoding end performs a corresponding decoding method. Specifically, step 312 determines the transform coefficient to be decoded corresponding to a frequency point in the image coding information according to a preset frequency point scanning order, including:

[0214] K1. Determine a first two-dimensional subband coefficient array to be decoded in the image coding information according to a preset frequency point scanning order, wherein the first two-dimensional subband coefficient array to be decoded includes: a transform coefficient to be decoded corresponding to a frequency point in the image coding information.

[0215] Step 313 decodes the to-be-decoded transform coefficient corresponding to a frequency point according to a preset coefficient scanning order, including:

[0216] K2, determining a first transform coefficient to be decoded in a first two-dimensional subband coefficient array to be decoded according to a preset coefficient scanning order;

[0217] K3. Perform entropy decoding on the first transform coefficient to be decoded according to the context area corresponding to the first transform coefficient to be decoded.

[0218] Specifically, the decoding end adopts an image decoding method corresponding to the encoding end. If the encoding end executes the encoding method shown in steps C1 to C4, the image encoding information received by the decoding end may include at least two two-dimensional subband coefficient arrays to be decoded. The frequency point scanning order is the scanning order of the two-dimensional subband coefficient arrays to be decoded. According to the preset frequency point scanning order, a first two-dimensional subband coefficient array to be decoded is obtained from at least two two-dimensional subband coefficient arrays to be decoded, and the first two-dimensional subband coefficient array to be decoded includes a plurality of transform coefficients to be decoded.

[0219] In some embodiments of the present application, when the encoding end performs the encoding shown in steps D1 to D5, the decoding end performs a corresponding decoding method. Specifically, step 312 determines the transform coefficient to be decoded corresponding to a frequency point in the image coding information according to a preset frequency point scanning order, including:

[0220] L1. Determine the three-dimensional subband coefficient array to be decoded in the image coding information according to the preset frequency point scanning order, wherein the three-dimensional subband coefficient array to be decoded includes: the transform coefficients to be decoded corresponding to all the frequency points in the image coding information;

[0221] L2. Determine a first two-dimensional subband coefficient array to be decoded in the three-dimensional subband coefficient array to be decoded according to a preset three-dimensional subband coefficient scanning order, wherein the first two-dimensional subband coefficient array to be decoded includes: a transform coefficient to be decoded corresponding to a frequency point in the image coding information.

[0222] Step 313 decodes the to-be-decoded transform coefficient corresponding to a frequency point according to a preset coefficient scanning order, including:

[0223] L3. Determine a first transform coefficient to be decoded in a first two-dimensional subband coefficient array to be decoded according to a preset coefficient scanning order;

[0224] L4. Perform entropy decoding on the first transform coefficient to be decoded according to the context area corresponding to the first transform coefficient to be decoded.

[0225] Specifically, the decoding end adopts an image decoding method corresponding to the encoding end, and the image encoding information received by the decoding end may include a three-dimensional subband coefficient array to be decoded. The decoding end can obtain at least two two-dimensional subband coefficient arrays from the three-dimensional subband coefficient array to be decoded according to a preset three-dimensional subband coefficient scanning order. Among them, the same two-dimensional subband coefficient array includes transform coefficients in at least two transform coefficient blocks corresponding to the same frequency point, and different two-dimensional subband coefficient arrays have different frequency points.

[0226] The three-dimensional subband coefficient array includes multiple two-dimensional subband coefficient arrays. The decoding end first determines a two-dimensional subband coefficient array from the three-dimensional subband coefficient array according to a preset three-dimensional subband coefficient scanning order, for example, determines a first two-dimensional subband coefficient array, and the first two-dimensional subband coefficient array is a two-dimensional subband coefficient array in the three-dimensional subband coefficient array. The three-dimensional subband coefficient scanning order is the order in which the three-dimensional subband coefficient array is scanned.

[0227] The decoding end adopts an image decoding method corresponding to the encoding end. When the encoding end adopts the method of generating a three-dimensional subband coefficient array, the decoding end performs lossless decoding on the second two-dimensional subband coefficient array in a coefficient scanning order preset by the decoding end to obtain the at least two transform coefficient blocks. For example, lossless decoding may include entropy decoding.

[0228] In some embodiments of the present application, the three-dimensional sub-band coefficient scanning order includes:

[0229] A preset order of changing the horizontal position index; or,

[0230] The change order of the preset vertical position index; or,

[0231] The preset frequency index changes in order.

[0232] Please refer to the description of the scanning order of the three-dimensional sub-band coefficients in the above embodiment for details.

[0233] In some embodiments of the present application, when the encoding end performs the encoding shown in steps G1 to G3, the decoding end performs the corresponding decoding method. Specifically, step 312 determines the transform coefficient to be decoded corresponding to a frequency point in the image coding information according to the preset frequency point scanning order, including:

[0234] M1. Determine the transform coefficients to be decoded included in a frequency sub-band in the image coding information according to a preset frequency point scanning order, wherein the transform coefficients to be decoded included in a frequency sub-band correspond to the same frequency point.

[0235] Step 313 decodes the to-be-decoded transform coefficient corresponding to a frequency point according to a preset coefficient scanning order, including:

[0236] M2. Determine a first transform coefficient to be decoded in a frequency subband according to a preset coefficient scanning order;

[0237] M3. Perform entropy decoding on the first transform coefficient to be decoded according to a context area corresponding to the first transform coefficient to be decoded.

[0238] The decoding end adopts the image decoding method corresponding to the encoding end. The encoding end can encode the two-dimensional subband coefficient array as a frequency subband during encoding, and the image encoding information received by the decoding end may include the transform coefficients to be decoded corresponding to multiple frequency subbands. The decoding end obtains the transform coefficients to be decoded corresponding to a frequency subband from the image encoding information according to the frequency point scanning order.

[0239] The decoding end may adopt a method of not generating a subband coefficient array, for example, not generating a two-dimensional subband coefficient array, nor generating a three-dimensional subband coefficient array. The decoding end first determines a first frequency subband, and the transform coefficients in at least two transform coefficient blocks corresponding to the first frequency subband correspond to the same frequency point. The decoding end performs lossless decoding on the image coding information corresponding to the first frequency subband according to a preset coefficient scanning order, and performs lossless decoding on the image coding information corresponding to multiple frequency subbands respectively in this way of sampling to obtain multiple transform coefficients.

[0240] Specifically, the decoding end obtains image coding information corresponding to multiple frequency subbands, and the decoding end adopts a method of lossless decoding of the frequency subbands, using at least two two-dimensional subband coefficient arrays as at least two frequency subbands, and losslessly decoding the image coding information corresponding to the at least two frequency subbands according to a preset coefficient scanning order. During lossless decoding, the decoding end can decode the two-dimensional subband coefficient array as a frequency subband, thereby simplifying the implementation process of lossless decoding.

[0241] In some embodiments of the present application, entropy decoding the first transform coefficient to be decoded according to the context area corresponding to the first transform coefficient to be decoded in the aforementioned steps K3, L4 and M3 includes:

[0242] N1. Acquire context information of the first transform coefficient to be decoded from a context area corresponding to the first transform coefficient to be decoded, wherein the context area corresponding to the first transform coefficient to be decoded includes: a decoded transform coefficient in the context area corresponding to the first transform coefficient to be decoded;

[0243] N2. Estimating the first transform coefficient to be decoded according to the context information of the first transform coefficient to be decoded to obtain probability distribution information of the first transform coefficient to be decoded;

[0244] N3. Performing arithmetical decoding on the first transform coefficient to be decoded according to the probability distribution information of the first transform coefficient to obtain a decoded first transform coefficient, wherein the decoded first transform coefficient belongs to at least two transform coefficient blocks.

[0245] Among them, the lossless decoding based on the decoding end can be entropy decoding. The following is an explanation of the entropy decoding process of the first transform coefficient. The decoding process of other transform coefficients in the first two-dimensional sub-band coefficient array can be implemented according to the entropy decoding process of the first transform coefficient.

[0246] In an embodiment of the present application, after the decoding end obtains the context information of the first transform coefficient, the decoding end performs entropy decoding on the first transform coefficient according to the context information of the first transform coefficient to obtain the decoded first transform coefficient. The entropy decoding performed by the decoding end may include probability estimation and arithmetic decoding. Specifically, the decoding end performs probability estimation on the first transform coefficient according to the context information of the first transform coefficient to obtain the probability distribution of the first transform coefficient, and then the decoding end performs arithmetic decoding on the first transform coefficient according to the probability distribution of the first transform coefficient to obtain the image decoding information corresponding to the first transform coefficient.

[0247] An example is given below. When entropy decoding is performed on the coded information after entropy coding, each symbol is decoded using the same processing order as the encoding end. When processing each symbol, the decoded symbol in the preset context area of ​​the symbol to be decoded is input into the probability estimation network as the context, the probability distribution of the current symbol is estimated, and the current symbol is decoded using the probability distribution to obtain the transform coefficient of the current symbol. After decoding each symbol, a two-dimensional subband coefficient array is obtained.

[0248] The lossless decoding performed by the decoding end may be entropy decoding. The entropy decoding process of the first transform coefficient is used to illustrate. The decoding process of other transform coefficients in the first two-dimensional sub-band coefficient array may be implemented according to the entropy decoding process of the first transform coefficient.

[0249] In an embodiment of the present application, after the decoding end obtains the context information of the first transform coefficient, the decoding end performs entropy decoding on the first transform coefficient according to the context information of the first transform coefficient to obtain the decoded first transform coefficient. The entropy decoding performed by the decoding end may include probability estimation and arithmetic decoding. Specifically, the decoding end performs probability estimation on the first transform coefficient according to the context information of the first transform coefficient to obtain the probability distribution of the first transform coefficient, and then the decoding end performs arithmetic decoding on the first transform coefficient according to the probability distribution of the first transform coefficient to obtain the image decoding information corresponding to the first transform coefficient.

[0250] An example is given below. When entropy decoding is performed on the coded information after entropy coding, each symbol is decoded using the same processing order as the encoding end. When processing each symbol, the decoded symbol in the preset context area of ​​the symbol to be decoded is input into the probability estimation network as the context, the probability distribution of the current symbol is estimated, and the current symbol is decoded using the probability distribution to obtain the transform coefficient of the current symbol. After decoding each symbol, a two-dimensional subband coefficient array is obtained.

[0251] In some embodiments of the present application, the context area corresponding to the first transform coefficient to be decoded includes at least one of the following: a spatial neighborhood of the first transform coefficient to be decoded, a subband neighborhood of the first transform coefficient to be decoded, and a three-dimensional neighborhood of the first transform coefficient to be decoded.

[0252] In some embodiments of the present application, entropy decoding the first transform coefficient to be decoded according to the context area corresponding to the first transform coefficient to be decoded in the aforementioned steps K3, L4 and M3 includes:

[0253] P1. When the first transform coefficient to be decoded is a plurality of transform coefficients to be decoded, context information of the plurality of transform coefficients to be decoded is obtained from context areas corresponding to the plurality of transform coefficients to be decoded, wherein the plurality of context information to be decoded includes: decoded transform coefficients in the context areas corresponding to the plurality of transform coefficients to be decoded;

[0254] P2. Estimating multiple transform coefficients to be decoded in parallel according to context information of the multiple transform coefficients to be decoded to obtain probability distribution information of a first transform coefficient to be decoded;

[0255] P3. Performing arithmetical decoding on multiple transform coefficients to be decoded in parallel according to probability distribution information of the multiple transform coefficients to be decoded to obtain decoded first transform coefficients, wherein the decoded first transform coefficients belong to at least two transform coefficient blocks.

[0256] The decoding end adopts an image decoding method corresponding to the encoding end. The encoding end can perform encoding in parallel during encoding, and the image decoding efficiency can be improved by parallel decoding at the decoding end.

[0257] In some embodiments of the present application, the context area corresponding to the multiple transform coefficients to be decoded includes at least one of the following: a spatial neighborhood of the multiple transform coefficients to be decoded, a subband neighborhood of the multiple transform coefficients to be decoded, and a three-dimensional neighborhood of the multiple transform coefficients to be decoded.

[0258] In some embodiments of the present application, when the encoding end performs the encoding shown in steps G1 to G3, the decoding end performs a corresponding decoding method. Specifically, step 312 determines the transform coefficient to be decoded corresponding to a frequency point in the image coding information according to a preset frequency point scanning order, including:

[0259] Q1. Determine a first sub-band image in the image coding information according to a preset frequency point scanning order, where the first sub-band image includes: a transform coefficient to be decoded corresponding to a frequency point in the image coding information.

[0260] Step 313 decodes the to-be-decoded transform coefficient corresponding to a frequency point according to a preset coefficient scanning order, including:

[0261] Q2. Decode the coefficients to be decoded of the first sub-band image according to a preset coefficient scanning order to obtain at least two transform coefficient blocks.

[0262] Specifically, the decoding end adopts an image decoding method corresponding to the encoding end. The encoding end may encode the two-dimensional subband coefficient array as a subband image during encoding, and the image encoding information received by the decoding end may include the transform coefficients to be decoded corresponding to the multiple subband images. The decoding end obtains the transform coefficients to be decoded corresponding to the first subband image from the image encoding information according to the frequency point scanning order.

[0263] The decoding end adopts lossless decoding for the sub-band image, and after obtaining multiple transform coefficients, the decoding end adopts a method of lossless decoding for the sub-band image, using at least two two-dimensional sub-band coefficient arrays as at least two sub-band images, and losslessly decoding the image coding information corresponding to the at least two sub-band images according to a preset coefficient scanning order. The decoding end can decode the two-dimensional sub-band coefficient array as a sub-band image during lossless decoding, thereby simplifying the implementation process of lossless decoding.

[0264] In some embodiments of the present application, after step 313 obtains at least two transform coefficient blocks, the method provided by the embodiment of the present application further includes:

[0265] R1. performing entropy coding on at least two transform coefficient blocks to obtain transform coefficient coding information;

[0266] R2, generating at least two image blocks according to the transform coefficient coding information;

[0267] R3. Restore an original image region according to at least two image blocks, where the original image region includes: an original pixel image region or a residual image region.

[0268] The decoding end adopts an image decoding method corresponding to the encoding end. If the encoding end obtains a transform coefficient block by entropy decoding the transform coefficient coding information and encodes the transform coefficient block, the decoding end can entropy encode at least two transform coefficient blocks to obtain the transform coefficient coding information. In the embodiment of the present application, at least two image blocks are generated according to the transform coefficient coding information, and then the original image area is restored according to the at least two image blocks, so that the image decompression process can be completed.

[0269] In some embodiments of the present application, after step 313 obtains at least two transform coefficient blocks, the method provided by the embodiment of the present application further includes:

[0270] S1. Dequantizing and detransforming at least two transform coefficient blocks to obtain at least two image blocks;

[0271] S2. Restoring an original image region according to at least two image blocks, where the original image region includes: an original pixel image region or a residual image region.

[0272] In an embodiment of the present application, after generating at least two transform coefficient blocks, the decoder generates at least two image blocks according to the at least two transform coefficient blocks. The decoder can process the at least two transform coefficient blocks by inverse transformation and inverse quantization, thereby generating at least two image blocks.

[0273] The transform coefficients in at least two transform coefficient blocks corresponding to the same frequency point are inversely transformed and inversely quantized to obtain at least two image blocks; or the transform coefficients in at least two transform coefficient blocks corresponding to the same frequency point are entropy encoded to obtain at least two image blocks.

[0274] In the embodiment of the present application, after the decoding end generates at least two image blocks, the decoding end restores the original image according to the at least two image blocks, and the original image can be obtained by splicing multiple image blocks.

[0275] From the foregoing description of the image processing method, it can be seen that in the embodiment of the present application, by decoding the transform coefficients to be decoded corresponding to the same frequency point in the image coding information according to the preset frequency point scanning order and the frequency point scanning order, at least two transform coefficient blocks can be obtained. Since each transform coefficient in a transform coefficient block corresponds to a different frequency point, different transform coefficients corresponding to the same frequency point can be obtained for the decoding of the image coding information, thereby improving the image decompression efficiency, reducing the storage cost of the image coding information, and ultimately saving storage resources.

[0276] In order to better understand and implement the above-mentioned solutions in the embodiments of the present application, the following examples are given for specific explanation by using corresponding application scenarios.

[0277] The embodiment of the present application proposes a block transformation and coefficient coding scheme in video image compression. The embodiment of the present application improves image coding performance, ultimately saves storage resources, and improves the coding efficiency of video images. For example, the compressed file generated by entropy coding consumes a large amount of storage resources on the server side. The embodiment of the present application can be applied to the efficient lossless transcoding process of entropy coded images. The embodiment of the present application can also be applied to the server side to perform lossless transcoding on the compressed file generated by entropy coding, improve image coding performance, and ultimately save storage resources.

[0278] Next, the image processing method provided by the embodiment of the present application is described in detail through five different embodiments. In each embodiment, the image encoding process is first introduced from the encoding end. After the encoding end generates the image encoding code stream, the encoding end sends it to the decoding end. After the decoding end receives the image encoding code stream, the decoding end decodes the image encoding code stream to restore the original image.

[0279] Embodiment 1

[0280] This embodiment relates to an image coding scheme, first obtaining a transform coefficient block (referred to as a coefficient block), and then performing lossless coding on the coefficient block to obtain coded image data. This embodiment provides a whole coding process, which may include, for example, transforming and quantizing an image region, and then performing lossless coding on the quantized coefficients to obtain coded image data. Figure 4 As shown in the figure, the image encoding process mainly includes:

[0281] Step 401: Obtain a transform coefficient block.

[0282] Method 1: Divide the image area to be processed into image blocks of preset sizes, and transform and quantize each image block to obtain a transform coefficient block, which can specifically be a transform coefficient array C (bx, by, fx, fy).

[0283] First, the image to be processed or the image region is divided into sub-image blocks of the same preset size. The size of the sub-image blocks of the preset size may be 4x4, 8x8, 16x16, 32x32, 64x64, 128x128, 256x256, etc.

[0284] like Figure 5 FIG. 1 is a schematic diagram of obtaining a transform coefficient block provided by an embodiment of the present application. A DCT transform is performed on a sub-image block of a preset size to obtain a transform coefficient block. Optionally, the transform coefficient block is quantized to obtain a quantized coefficient array C (bx, by, fx, fy). Each transform coefficient in the coefficient array can be Indicates. Where (bx, by) represents the horizontal and vertical position index of the coefficient block, bx = 0, ..., M-1, by = 0, ..., N-1, Wherein, M and N are the width and height of the image in units of sub-image blocks, respectively, W and H represent the width and height of the current image, respectively, and subW and subH represent the width and height of the sub-image block in the current image, respectively. (fx, fy) represents the frequency index of each frequency point in the coefficient block relative to the frequency point of the upper left vertex, where fx = 0, .., subW-1, fy = 0, .., subH-1. In the embodiment of the present application, the size of the sub-image block of the preset size is taken as 8x8, that is, the width and height of the sub-image block are both 8, that is, the values ​​of subW and subH are both 8, which will not be repeated later.

[0285] For example Figure 5 As shown, (bx=0, by=0) represents the first coefficient block, (bx=0, by=1) represents the second coefficient block in the first row, and so on. (fx=0, fy=1) represents the frequency index of the frequency point with coordinates (0,1) relative to the upper left vertex in the coefficient block. It represents the coefficient value represented by the position (fx=3, fy=2) within the coefficient block (bx=1, by=0).

[0286] Method 2: Entropy decoding is performed on the entropy-encoded image data to obtain the transformation coefficients (referred to as coefficients for short) of the 8x8 blocks in the image.

[0287] The Huffman decoding method is used to entropy decode the transform coefficient coding information of all sub-image blocks in the JPEG file to obtain one or more transform coefficient blocks of sub-image blocks. Each transform coefficient of the coefficient block is recorded as Where (bx, by) represents the horizontal and vertical position index of the coefficient block, bx = 0, ..., M-1, by = 0, ..., N-1, Wherein, M and N are respectively the width and height of the image in units of sub-image blocks, W and H respectively represent the width and height of the current image, and subW and subH respectively represent the width and height of the sub-image block in the current image. (fx, fy) represents the frequency index of each frequency point in the coefficient block relative to the frequency point of the upper left vertex, where fx=0,..,subW-1, fy=0,..,subH-1. In the embodiment of the present application, the width and height of all sub-image blocks in the JPEG file are both 8, that is, the values ​​of subW and subH are both 8, which will not be repeated later. It is not limited that the technical solution described in the embodiment of the present application can also be applied to the case where the values ​​of subW and subH are not 8, for example, subW is not equal to subH.

[0288] For example, Figure 5 As shown, (bx=0, by=0) represents the first coefficient block, (bx=0, by=1) represents the second coefficient block in the first row, and so on. (fx=0, fy=1) represents the frequency index of the frequency point with coordinates (0,1) relative to the upper left vertex in the coefficient block. It represents the coefficient value represented by the position (fx=3, fy=2) within the coefficient block (bx=0, by=1).

[0289] Step 402: Obtain the transform coefficient value of a specified frequency point in an 8x8 transform coefficient block (referred to as 8x8 coefficient block) in the image to form a two-dimensional subband coefficient array of the frequency point.

[0290] Specifically, Figure 6 FIG. 1 is a schematic diagram of obtaining a two-dimensional subband coefficient array provided by an embodiment of the present application. Obtain the transform coefficient value of a specified frequency point (fx, fy) in an 8x8 transform coefficient block in an image, and obtain the two-dimensional subband coefficient array C of the frequency point. fx,fy (bx,by).

[0291] For example, the two-dimensional subband coefficient array obtained by obtaining the transform coefficient values ​​of the frequency points (fx=0, fy=0) in all 8x8 transform coefficient blocks in the image is as follows: Figure 6 As shown in the first figure, similarly, the two-dimensional subband coefficient arrays of the remaining frequency points are obtained. An 8x8 transform coefficient block has a total of 64 frequency points, so 64 two-dimensional subband coefficient arrays of the current coded image can be obtained.

[0292] Step 403: process each two-dimensional subband coefficient array according to a preset order one, and when processing each transform coefficient in each two-dimensional subband coefficient array, process each transform coefficient according to a preset order two, and when processing each transform coefficient, select the processed coefficients in the preset spatial neighborhood and subband neighborhood as context estimates to obtain the probability distribution of the current coefficient, and use the probability distribution to perform arithmetic coding on the current coefficient to obtain the coding information of the current coefficient.

[0293] It should be noted that processing each two-dimensional subband coefficient array according to preset order one means determining each two-dimensional subband coefficient array according to preset order one, and processing each transform coefficient according to preset order two means encoding each transform coefficient according to preset order two.

[0294] Step 403.1: Process each two-dimensional subband coefficient array according to a preset order.

[0295] like Figure 7 As shown, it is a schematic diagram of the transform coefficient block provided by the embodiment of the present application. Set the frequency index of the two-dimensional subband coefficient array of the specified frequency point (fx, fy) to fs, and each index value corresponds to a two-dimensional subband coefficient array. First, obtain any 8x8 transform coefficient block in the image, and scan the transform coefficients of the specified frequency point (fx, fy) in the coefficient block in a preset order. Then, the index fs of the two-dimensional subband coefficient array of the specified frequency point (fx, fy) is the same as the scanning order of the frequency point in the coefficient block, and the two-dimensional subband coefficient array C corresponding to fs is processed in the order of the size of fs. fx,fy (bx, by). Taking the coefficient block (bx=0, by=0) as an example, if the preset order is Z-shaped scanning, then fs=0 corresponds to the two-dimensional subband coefficient array of the frequency point (fx=0, fy=0), fs=1 corresponds to the two-dimensional subband coefficient array of the frequency point (fx=1, fy=0), and fs=2 corresponds to the two-dimensional subband coefficient array of the frequency point (fx=0, fy=1). The two-dimensional subband coefficient arrays of the corresponding frequency points are processed in the order of the size of fs. Wherein, fs is an integer, and its value range is 0,…,subW×subH-1.

[0296] Step 403.2: Process each transform coefficient in each two-dimensional sub-band coefficient array.

[0297] When processing each two-dimensional subband coefficient array, each transform coefficient in each two-dimensional subband coefficient array is processed according to preset order two. The processed coefficients in the preset context area are selected as contexts, and the context information is input into the probability estimation network to estimate the probability distribution of the current coefficient, thereby obtaining the probability distribution of the current coefficient, and the current coefficient is arithmetically encoded using the probability distribution to obtain the encoding information of the current coefficient. The preset context area (as explained below) may include the spatial neighborhood and / or subband neighborhood of the coefficient, or the three-dimensional neighborhood defined in subsequent embodiments. It should be noted that when obtaining the probability distribution of the coefficients, the probability distribution of the coefficients may also be estimated in parallel, that is, the probability distribution of multiple coefficients is obtained at one time, and then each transform coefficient is losslessly encoded according to preset order two, or multiple coefficients are losslessly encoded in parallel.

[0298] Specifically, the method for estimating the probability distribution of the current coefficient is as follows: Method 1: The probability estimation network directly estimates the probability of the value within the value range according to the dynamic value range of the coefficient to obtain a probability vector, which includes the probability value of each value in the value range, where the sum of all probability values ​​is 1. Method 2: You can also use a probability distribution model to obtain a probability vector. For example, use a single Gaussian model or a mixed Gaussian model for modeling, use a probability estimation network to estimate the model parameters, substitute the model parameters into the probability distribution model, and obtain a probability vector.

[0299] First, after obtaining the coefficient in the preset context area of ​​the coefficient as the context, the context information is input into the probability estimation network, the coefficient is probability estimated to obtain the probability distribution of the coefficient, and the current coefficient is losslessly encoded using the probability distribution to obtain the encoding information of the coefficient. The probability distribution estimation method used in the probability estimation network may include a probability distribution estimation method based on a deep learning network, such as a recurrent neural network and a convolutional neural network.

[0300] Among them, if the coordinates of the coefficient are (bx0, by0, fs0), if any coordinate position (bx1, by1, fs1) ​​in the preset context area satisfies bx0-M1≤bx1≤bx0+N1, and by0-M2≤by1≤by0+N2, and fs1=fs0, then the coefficient at the position (bx1, by1, fs1) ​​is within the subband neighborhood of the coefficient.

[0301] If any coordinate position (bx1, by1, fs1) ​​in the preset context area satisfies bx1=bx0, and by0-M2≤by1≤by0+N2, and fs0-N3≤fs1≤fs0, or satisfies bx0-M1≤bx1≤bx0+N1, and by1=by0, and fs0-N3≤fs1≤fs0, then the coefficient of the position (bx1, by1, fs1) ​​is within the spatial neighborhood of the coefficient. Wherein, M1, M2, N1, N2 and N3 are integers greater than or equal to 0, and the values ​​can be equal or different, and are not limited here. For example, the values ​​can be 1, 2, 3, 4 or 5.

[0302] The preset order 1 and the preset order 2 may be processing orders obtained by zig-zag scanning, or other scanning methods, such as row-by-row scanning, column-by-column scanning, diagonal scanning, horizontal reverse scanning, and vertical reverse scanning, etc. Row-by-row scanning refers to continuous scanning from the first row to the last row of an image, and column-by-column scanning refers to continuous scanning from the first column to the last column of an image.

[0303] Step 404: Obtain a transcoded output file.

[0304] The encoded information after entropy encoding is output to obtain a compressed file.

[0305] like Figure 8 FIG. 1 is a schematic diagram of an image decoding process provided by an embodiment of the present application. It mainly includes:

[0306] Step 801: Obtain the coded information after entropy coding, and perform entropy decoding on the coded information to obtain a coefficient array.

[0307] When entropy decoding is performed on the coded information after entropy coding, each codeword is decoded using the same processing order as the encoding end. That is, each encoded two-dimensional subband coefficient array is first processed according to the preset order one, and when processing each encoded two-dimensional subband coefficient array, each encoded coefficient in each two-dimensional subband coefficient array is processed according to the preset order two. In this embodiment, the encoded coefficient can be understood as a codeword. When processing each codeword, the decoded codeword in the preset context area of ​​the codeword to be decoded is input as the context into the probability estimation network, the probability distribution of the current codeword is estimated, and the current codeword is decoded using the probability distribution to obtain the transform coefficient of the current codeword. After decoding each codeword, an 8x8 transform coefficient block or coefficient array C (bx, by, fx, fy) of multiple images is obtained.

[0308] Step 802: Obtain decoded image data.

[0309] Method 1: Corresponding to the decoding of method 1 in step 401: performing inverse transformation and inverse quantization on the coefficient array C (bx, by, fx, fy) to obtain decoded image data.

[0310] Method 2: Corresponding to the decoding of method 2 in step 401: entropy encoding is performed on the 8x8 transform coefficient block to obtain entropy encoded image data.

[0311] In the encoding scheme provided in the first embodiment, the input image block is divided into image blocks of preset size, and then transformed and quantized to obtain a transform coefficient array. Each transform coefficient in the transform coefficient array is processed in a preset order, and when arithmetic coding is performed on one of the coefficients, the processed coefficients in the neighborhood of the coefficient are used as context to estimate the probability distribution of the coefficient value. The embodiment of the present application can improve the compression rate of the image and reduce the storage resources on the server side.

[0312] Embodiment 2

[0313] This embodiment relates to an image coding scheme, first obtaining a transform coefficient block (referred to as a coefficient block), and then performing lossless coding on the coefficient block to obtain coded image data. This embodiment provides a whole coding process, which may include, for example, transforming and quantizing an image region, and then performing lossless coding on the quantized coefficients to obtain coded image data. Fig. 9 As shown in the figure, the image encoding process mainly includes:

[0314] Step 901: Obtain a transform coefficient block.

[0315] This step is the same as step 401 in the first embodiment.

[0316] Step 902: Obtain the transform coefficient value of a specified frequency point in an 8x8 transform coefficient block in the image to form a two-dimensional subband coefficient array of the frequency point.

[0317] This step is the same as step 402 in the first embodiment.

[0318] Step 903: Arrange the two-dimensional sub-band coefficient array in a preset order to obtain a three-dimensional sub-band coefficient array.

[0319] like Fig.10 As shown, the two-dimensional subband coefficient array C of the specified frequency point (fx, fy) is arranged in a preset order. fx,fy(bx, by), and set the frequency index of the sorted two-dimensional subband coefficient array to fs. First, obtain any 8x8 transform coefficient block in the image, and sort the transform coefficients of the specified frequency point (fx, fy) in the coefficient block according to the preset order. The sorting index of the two-dimensional subband coefficient array of the specified frequency point (fx, fy) is the same as its sorting index in the coefficient block, thereby determining the two-dimensional subband coefficient array C of the specified frequency point (fx, fy). fx,fy The frequency index fs of (bx, by). Taking the coefficient block (bx=0, by=0) as an example, if the preset order is zigzag scanning, then fs=0 corresponds to the two-dimensional subband coefficient array of the frequency point (fx=0, fy=0), fs=1 corresponds to the two-dimensional subband coefficient array of the frequency point (fx=1, fy=0), and fs=2 corresponds to the two-dimensional subband coefficient array of the frequency point (fx=0, fy=1). And the two-dimensional subband coefficient array C fx,fy (bx,by) together, we get Figure 8 The three-dimensional subband coefficient array C(bx,by,fs) is shown. The one-dimensional frequency index fs takes the value of an integer in 0,…,subW×subH-1. It can be understood that the frequency index fs is the index after sorting the two-dimensional subband coefficient array in a preset order.

[0320] The preset order one may be an arrangement order obtained by zig-zag scanning, or may be obtained by other scanning methods, such as row-by-row scanning, column-by-column scanning, and diagonal scanning, etc. Row-by-row scanning refers to scanning continuously from the first row to the last row of the image, and column-by-column scanning refers to scanning continuously from the first column to the last column of the image.

[0321] like Fig.11a , which is a schematic diagram of a three-dimensional subband coefficient array provided in an embodiment of the present application. The three-dimensional subband coefficient array C (bx, by, fs) may include data of three dimensions: bx, by and fs.

[0322] It should be noted that step 901 and step 902 of this embodiment are the same as step 401 and step 402 of embodiment 1. The difference is that embodiment 2 adds a step of combining the two-dimensional subband coefficient array into a three-dimensional subband coefficient array according to a preset order.

[0323] Step 904: Process each transform coefficient in the three-dimensional subband coefficient array. When processing each coefficient, use the processed coefficients in the three-dimensional neighborhood of the coefficient as context to estimate the probability distribution of the current coefficient, and use the probability distribution to arithmetically encode the current coefficient to obtain the encoding information of the current coefficient.

[0324] like Fig.11bAs shown, it is a schematic diagram of the three-dimensional transform coefficient provided by the embodiment of the present application. In the figure, a square filled with black represents the transform coefficient currently being processed, a square filled with horizontal lines represents an uncoded transform coefficient, and a white square represents a coded transform coefficient. The three-dimensional neighborhood of the transform coefficient currently being processed may include the coded transform coefficient.

[0325] Step 9041: Set the processing order of each transform coefficient in the three-dimensional subband coefficient array C(bx,by,fs).

[0326] Specifically, one of the following methods may be used to set the processing order of each transform coefficient, wherein method one is the optimal method.

[0327] Method 1: When processing each transform coefficient in the three-dimensional subband coefficient array C(bx,by,fs), the two-dimensional coefficient subband arrays of fs=0 to subW×subH-1 layers are processed in sequence. When processing the two-dimensional coefficient subband array of each layer, each transform coefficient in the two-dimensional coefficient subband array C(bx,by) is processed according to preset order 2.

[0328] Method 2: When processing each transform coefficient in the three-dimensional subband coefficient array C(bx,by,fs), the two-dimensional coefficient subband arrays from bx=0 to M-1 layers are processed in turn. When processing the two-dimensional coefficient subband array of each layer, each transform coefficient in the two-dimensional coefficient subband array C(by,fs) is processed according to preset order 2.

[0329] Method three: When processing each transform coefficient in the three-dimensional subband coefficient array C(bx,by,fs), the two-dimensional coefficient subband arrays of by=0 to N-1 layers are processed in turn. When processing the two-dimensional coefficient subband array of each layer, each transform coefficient in the two-dimensional coefficient subband array C(bx,fs) is processed according to the preset order 2.

[0330] It should be noted that the preset order 2 may use processing orders such as zig-zag scanning, row-by-row scanning, and column-by-column scanning, and its meaning is the same as that of the "preset order 1" in step 3, and will not be elaborated here.

[0331] Step 9042: Obtain the probability distribution of the coefficient.

[0332] When processing a coefficient, the processed coefficients in the three-dimensional neighborhood of the coefficient are used as contexts to estimate the probability distribution of the coefficient, and the current coefficient is arithmetically encoded using the probability distribution to obtain the encoding information of the current coefficient.

[0333] The coefficients in the three-dimensional neighborhood of the coefficient can be obtained according to the following method: if the coordinates of the coefficient are (bx0, by0, fs0), if the coordinate position (bx1, by1, fs1) ​​satisfies bx0-M1≤bx1≤bx0+N1, and by0-M2≤by1≤by0+N2, and fs0-N3≤fs1≤fs0, then the coefficients at the position (bx1, by1, fs1) ​​are in the three-dimensional neighborhood of the coefficient, where M1, M2, N1, N2 and N3 are integers greater than or equal to 0, and the values ​​can be equal or different, which are not limited here, for example, the values ​​can be 1, 2, 3, 4 or 5. Therefore, the processed coefficients in the three-dimensional neighborhood of the coefficient also include the processed coefficients in the two-dimensional neighborhood of the coefficient, which are not limited here. In particular, when fs1=fs0, when the selected context and the coefficient to be processed are in the same two-dimensional neighborhood, it is more conducive to parallel coding, that is, probability estimation of multiple coefficients is performed in parallel, and probability distribution of multiple coefficients is obtained at the same time, thereby improving coding efficiency.

[0334] It should be noted that when obtaining the probability distribution of the coefficients, the probability distribution of the coefficients can also be estimated in parallel, that is, the probability distribution of multiple coefficients is obtained at one time, and then each transform coefficient is losslessly encoded according to the preset order two, or multiple coefficients are losslessly encoded in parallel.

[0335] First, after obtaining the coefficients in the three-dimensional neighborhood of the coefficient as context, the context information is input into the probability estimation network, the coefficient is probability estimated to obtain the probability distribution of the coefficient, and the current coefficient is losslessly encoded using the probability distribution to obtain the encoding information of the coefficient. The probability distribution estimation methods used in the probability estimation network include probability distribution estimation methods based on deep learning networks, such as recurrent neural networks and pixel-by-pixel convolutional neural networks.

[0336] Specifically, the method for estimating the probability distribution of the coefficient is as follows: Method 1: The probability estimation network directly estimates the probability of the value within the value range according to the dynamic value range of the coefficient to obtain a probability vector, which includes the probability value of each value in the value range, where the sum of all probability values ​​is 1. Method 2: The probability vector can also be obtained using a probability distribution model. For example, a single Gaussian model or a mixed Gaussian model is used for modeling, and the model parameters are estimated using a probability estimation network, and the model parameters are substituted into the probability distribution model to obtain a probability vector.

[0337] Step 905: Obtain the transcoded output file.

[0338] The encoding information after entropy encoding is output to obtain a transcoded output file.

[0339] like Fig.12FIG. 1 is a schematic diagram of an image decoding process provided by an embodiment of the present application. It mainly includes:

[0340] Step 1201: Obtain the coded information after entropy coding, and perform entropy decoding on the coded information to obtain decoded transform coefficients.

[0341] When entropy decoding is performed on the coded information after entropy coding, each codeword is decoded using the same processing order as the encoding end. For example, the same processing order means first determining the coefficient to be decoded corresponding to a frequency point in the image coding information according to the frequency point scanning order, and then decoding the coefficient to be decoded corresponding to a frequency point according to the coefficient scanning order. When processing each codeword, the codeword to be decoded is input into the same probability estimation network as the encoding end, and the processed codewords in the three-dimensional neighborhood of the codeword are used as context estimation to obtain the probability distribution of the current codeword, and the probability distribution is used to decode the current codeword to obtain the transform coefficient of the current codeword. After decoding each codeword, a transform coefficient block or coefficient array C (bx, by, fx, fy) is obtained.

[0342] Step 1202: Obtain decoded image data.

[0343] Method 1: Corresponding to the decoding of method 1 in step 401: performing inverse transformation and inverse quantization on the coefficient array C (bx, by, fx, fy) to obtain decoded image data.

[0344] Method 2: Corresponding to the decoding of method 2 in step 401: entropy encoding is performed on the 8x8 transform coefficient block to obtain entropy encoded image data.

[0345] In the coding scheme provided in the second embodiment, the present embodiment combines the transform coefficients according to the frequency components to obtain a three-dimensional coefficient array C(bx, by, fs), and when performing arithmetic coding on one of the coefficients, the processed coefficients in the neighborhood of the coefficient are used as context to estimate the probability distribution of the coefficient value. The present application embodiment can improve the compression rate of the image and reduce the storage resources on the server side.

[0346] Embodiment 3

[0347] This embodiment relates to an image coding scheme, first obtaining a transform coefficient block (referred to as a coefficient block), and then performing lossless coding on the coefficient block to obtain coded image data. This embodiment provides a whole coding process, for example, which may include transforming and quantizing an image region, and then performing lossless coding on the quantized coefficients to obtain coded image data.

[0348] This embodiment uses the method in step 401 of the first embodiment to obtain the transform coefficient array C (bx, by, fx, fy), and there are two encoding methods:

[0349] Method 1: first sort each transform coefficient in the coefficient block according to a preset order, then obtain the coefficients of the same frequency point, combine them, and obtain a three-dimensional subband coefficient array (referred to as a three-dimensional coefficient array). Fig.13 As shown in the figure, the image encoding process mainly includes:

[0350] Step 1301, obtain the transform coefficient block.

[0351] This step is the same as step 401 in the first embodiment.

[0352] Step 1302: Arrange each transform coefficient in the 8x8 transform coefficient block in a preset order to obtain a sorted 8x8 transform coefficient block.

[0353] Step 1303: Obtain the transform coefficient value of a specified frequency point in the 8x8 transform coefficient block in the image, construct a two-dimensional subband coefficient array of the frequency point, and combine the two-dimensional subband coefficient arrays to obtain a three-dimensional subband coefficient array.

[0354] Step 1304: Process each transform coefficient in the three-dimensional subband coefficient array, and when processing each coefficient, use the processed coefficients in the three-dimensional neighborhood of the coefficient as context to estimate the probability distribution of the current coefficient, and use the probability distribution to perform arithmetic coding on the current coefficient to obtain the coding information of the current coefficient. Specifically, it is the same as step 904 in embodiment 2.

[0355] Method 2: The implementation scheme of this method is similar to that of Example 1, and does not need to be combined into a three-dimensional sub-band coefficient array.

[0356] First, each transform coefficient in the 8x8 transform coefficient block is scanned in a preset order to obtain a frequency index fs, and then the transform coefficient value of a specified frequency point in the 8x8 transform coefficient block is obtained to form a two-dimensional subband coefficient array of the frequency point.

[0357] Each two-dimensional subband coefficient array is processed according to fs. When processing each transform coefficient in each two-dimensional subband coefficient array, each transform coefficient in each two-dimensional subband coefficient array is processed according to preset order two. When processing each transform coefficient, the processed coefficients in the preset spatial neighborhood and subband neighborhood are selected as context estimates to obtain the probability distribution of the current coefficient, and the current coefficient is arithmetically encoded using the probability distribution to obtain the encoding information of the current coefficient.

[0358] In the encoding scheme provided in the third embodiment, the processing order of each transform coefficient in the third embodiment is different from the processing order of the first and second embodiments.

[0359] like Fig.12FIG. 1 is a schematic diagram of an image decoding process provided by an embodiment of the present application. It mainly includes:

[0360] Step 1201: Obtain the coded information after entropy coding, and perform entropy decoding on the coded information to obtain decoded transform coefficients.

[0361] When entropy decoding is performed on the coded information after entropy coding, each codeword is decoded using the same processing order as the encoding end. For example, the same processing order means first determining the coefficient to be decoded corresponding to a frequency point in the image coding information according to the frequency point scanning order, and then decoding the coefficient to be decoded corresponding to a frequency point according to the coefficient scanning order. When processing each codeword, the codeword to be decoded is input into the same probability estimation network as the encoding end, and the processed codewords in the three-dimensional neighborhood of the codeword are used as context estimation to obtain the probability distribution of the current codeword, and the probability distribution is used to decode the current codeword to obtain the transform coefficient of the current codeword. After decoding each codeword, a transform coefficient block or coefficient array C (bx, by, fx, fy) is obtained.

[0362] Step 1202: Obtain decoded image data.

[0363] Method 1: Corresponding to the decoding of method 1 in step 401: performing inverse transformation and inverse quantization on the coefficient array C (bx, by, fx, fy) to obtain decoded image data.

[0364] Method 2: Corresponding to the decoding of method 2 in step 401: entropy encoding is performed on the 8x8 transform coefficient block to obtain entropy encoded image data.

[0365] In the coding scheme provided in the third embodiment, the present embodiment combines the transform coefficients according to the frequency components to obtain a three-dimensional coefficient array C(bx, by, fs), and when performing arithmetic coding on one of the coefficients, the processed coefficients in the neighborhood of the coefficient are used as context to estimate the probability distribution of the coefficient value. The present application embodiment can improve the compression rate of the image and reduce the storage resources on the server side.

[0366] Embodiment 4

[0367] This embodiment relates to an image coding scheme, first obtaining a transform coefficient block (referred to as a coefficient block), and then performing lossless coding on the coefficient block to obtain coded image data. This embodiment provides a whole coding process, which may include, for example, transforming and quantizing an image region, and then performing lossless coding on the quantized coefficients to obtain coded image data. Fig.14 As shown in the figure, the image encoding process mainly includes:

[0368] Step 1401, obtain the transform coefficient block.

[0369] This step is the same as step 401 in the first embodiment.

[0370] Step 1402: Obtain the transform coefficient value of a specified frequency point in an 8x8 transform coefficient block in the image to form a two-dimensional subband coefficient array of the frequency point.

[0371] It should be noted that step 1401 and step 1402 of this embodiment are the same as step 401 and step 402 of embodiment 1. The difference is that in embodiment 4, during entropy coding, the two-dimensional sub-band coefficient array is coded as a sub-band image.

[0372] Step 1403: The two-dimensional sub-band coefficient array corresponding to the frequency point is taken as a sub-band image, and when processing one coefficient, an image lossless coding method is used for encoding.

[0373] The two-dimensional subband coefficient array C corresponding to the specified frequency point fx,fy (bx, by) is taken as a sub-band image, the sub-band image is processed in a preset order, and the sub-band image processing order index is set to fs. When processing a coefficient of the sub-band image, an existing lossless image processing method is used for encoding. For example, a lossless coding mode in standard schemes such as H.264 and H.265, such as an arithmetic coding method.

[0374] Among them, the method for obtaining the processing order index fs includes: first obtaining any 8x8 transform coefficient block in the image, and sorting the transform coefficients of the specified frequency point (fx, fy) in the coefficient block according to a preset order, then the processing order index fs of the two-dimensional subband coefficient array of the specified frequency point (fx, fy) is the same as its sorting index in the coefficient block.

[0375] The preset order has the same meaning as in the first embodiment and will not be described in detail here.

[0376] In the encoding scheme provided in the fourth embodiment, during entropy encoding, the two-dimensional sub-band coefficient array is encoded as a sub-band image, and the scheme is simpler to implement.

[0377] Embodiment 5

[0378] This embodiment relates to an image coding scheme, first obtaining a transform coefficient block (referred to as a coefficient block), and then performing lossless coding on the coefficient block to obtain coded image data. This embodiment provides a whole coding process, which may include, for example, transforming and quantizing an image region, and then performing lossless coding on the quantized coefficients to obtain coded image data. Fig.15 As shown in the figure, the image encoding process mainly includes:

[0379] Step 1500 (not shown in the figure), obtain the transform coefficient block.

[0380] This step is the same as step 401 in the first embodiment.

[0381] Step 1501: Divide the image area to be processed into image blocks of preset sizes, and transform and quantize each image block to obtain a transform coefficient array.

[0382] First, the image to be processed or the image region is divided into sub-image blocks of the same preset size, where the size of the sub-image blocks of the preset size may be 4x4, 8x8, 16x16, 32x32, 64x64, 128x128, 256x256, etc.

[0383] like Figure 5 As shown, a DCT transform is performed on a sub-image block of a preset size to obtain a transform coefficient block. Optionally, the transform coefficient block is quantized to obtain a quantized transform coefficient array C (bx, by, fx, fy). Each transform coefficient in the coefficient array can be used Indicates. Where (bx, by) represents the horizontal and vertical position index of the coefficient block, bx = 0, ..., M-1, by = 0, ..., N-1, Wherein, M and N are the width and height of the image in sub-blocks, respectively, W and H represent the width and height of the current image, respectively, and subW and subH represent the width and height of the sub-image block in the current image, respectively. (fx, fy) represents the frequency index of each frequency point in the coefficient block relative to the frequency point of the upper left vertex, where fx = 0, .., subW-1, fy = 0, .., subH-1. In the embodiment of the present application, the size of the sub-image block of the preset size is taken as 8x8, that is, the width and height of the sub-image block are both 8, that is, the values ​​of subW and subH are both 8, which will not be repeated later.

[0384] For example, Figure 5 As shown, (bx=0, by=0) represents the first coefficient block, (bx=0, by=1) represents the second coefficient block in the first row, and so on. (fx=0, fy=1) represents the frequency index of the frequency point with coordinates (0,1) relative to the upper left vertex in the coefficient block. It represents the coefficient value represented by the position (fx=3, fy=2) within the coefficient block (bx=1, by=0).

[0385] The difference between the fifth embodiment and the first to fourth embodiments is that, when processing each transform coefficient, there is no action of constructing a two-dimensional subband coefficient array, nor is there an action of constructing a three-dimensional coefficient array. The coefficient block obtained in step 1501 is directly processed according to preset order one and preset order two, which is simpler to implement and has lower complexity.

[0386] It is not limited that the aforementioned step 1501 can be replaced by the following step: entropy decoding the entropy-encoded image data to obtain the transformation coefficients of the 8x8 blocks in the image.

[0387] The Huffman decoding method is used to entropy decode the transform coefficient encoding information of all sub-image blocks in the JPEG file to obtain the transform coefficients of one or more sub-image blocks, referred to as coefficient blocks. Each transform coefficient of the coefficient block is recorded as Wherein, (bx, by) represents the horizontal and vertical position index of the coefficient block, bx=0,…,M-1, by=0,…,N-1, M=W / subW, N=H / subH, where M and N are the image width and height in sub-blocks, respectively, W and H represent the width and height of the current image, respectively, and subW and subH represent the width and height of the sub-image block in the current image, respectively. (fx, fy) represents the frequency index of each frequency point in the coefficient block relative to the frequency point of the upper left vertex, where fx=0,..,subW-1, fy=0,..,subH-1. In the embodiment of the present application, the width and height of all sub-image blocks in the JPEG file are both 8, that is, the values ​​of subW and subH are both 8, which will not be repeated later. It is not limited that the technical solution described in the embodiment of the present application can also be applied to the case where the values ​​of subW and subH are not 8, or even subW is not equal to subH.

[0388] For example, Figure 5 As shown, (bx=0, by=0) represents the first coefficient block, (bx=0, by=1) represents the second coefficient block in the first row, and so on. (fx=0, fy=1) represents the frequency index of the frequency point with coordinates (0,1) relative to the upper left vertex in the coefficient block. It represents the coefficient value represented by the position (fx=3, fy=2) within the coefficient block (bx=0, by=1).

[0389] Step 1502: Process each frequency subband of the coefficient block of the current image according to preset order 1, and process the transform coefficients in each frequency subband according to preset order 2. When processing each transform coefficient, select the processed coefficient in the preset context area as the context to estimate the probability distribution of the current coefficient, and use the probability distribution to perform arithmetic coding on the current coefficient to obtain the coding information of the current coefficient.

[0390] It is not limited that the aforementioned step 1502 can be replaced by the following step: processing a transform coefficient corresponding to a frequency point in a coefficient block of the current image according to preset order 1, and processing each transform coefficient corresponding to each frequency point according to preset order 2. When processing each transform coefficient, select a processed coefficient in a preset context area as a context to estimate the probability distribution of the current coefficient, and use the probability distribution to perform arithmetic coding on the current coefficient to obtain coding information of the current coefficient.

[0391] For example, the set C of coefficients at a specified frequency point (fx, fy) in a transform coefficient block with a frequency subband of 8x8 fx,fy (bx, by), in this embodiment, there are 64 frequency subbands. First, obtain any 8x8 transform coefficient block in the image, and scan the coefficients of the specified frequency point (fx, fy) in the coefficient block in a preset order. The index fs of the frequency subband of the specified frequency point (fx, fy) is the same as the scanning order of the frequency point in the coefficient block, and the frequency subband corresponding to fs is processed in the order of the size of fs. Wherein, fs is an integer, and the value range is 0,…,subW×subH-1.

[0392] Then, when processing each frequency subband, each transform coefficient in the frequency subband is processed according to the preset order 2. When processing each transform coefficient, the processed coefficient in the preset context area is selected as the context, and the context information is input into the probability estimation network, the probability distribution of the coefficient is estimated, so as to obtain the probability distribution of the current coefficient, and the current coefficient is arithmetically encoded using the probability distribution to obtain the encoding information of the current coefficient. The preset context area may include the spatial neighborhood and / or subband neighborhood of the coefficient, or the three-dimensional neighborhood defined in the second embodiment.

[0393] Specifically, the method for estimating the probability distribution of the coefficient is as follows: Method 1: The probability estimation network directly estimates the probability of the value within the value range according to the dynamic value range of the coefficient to obtain a probability vector, which includes the probability value of each value in the value range, where the sum of all probability values ​​is 1. Method 2: The probability vector can also be obtained using a probability distribution model. For example, a single Gaussian model or a mixed Gaussian model is used for modeling, and the model parameters are estimated using a probability estimation network, and the model parameters are substituted into the probability distribution model to obtain a probability vector.

[0394] First, after obtaining the coefficients in the three-dimensional neighborhood of the coefficient as context, the context information is input into the probability estimation network, the coefficient is probability estimated to obtain the probability distribution of the coefficient, and the current coefficient is losslessly encoded using the probability distribution to obtain the encoding information of the coefficient. The probability distribution estimation method used in the probability estimation network includes a probability distribution estimation method based on a deep learning network, such as a recurrent neural network and a convolutional neural network.

[0395] Among them, if the coordinates of the coefficient are (bx0, by0, fs0), if any coordinate position (bx1, by1, fs1) ​​in the preset context area satisfies bx0-M1≤bx1≤bx0+N1, and by0-M2≤by1≤by0+N2, and fs1=fs0, then the coefficient at the position (bx1, by1, fs1) ​​is within the subband neighborhood of the coefficient.

[0396] If any coordinate position (bx1, by1, fs1) ​​in the preset context area satisfies bx1=bx0, and by0-M2≤by1≤by0+N2, and fs0-N3≤fs1≤fs0, or satisfies bx0-M1≤bx1≤bx0+N1, and by1=by0, and fs0-N3≤fs1≤fs0, then the coefficient of the position (bx1, by1, fs1) ​​is within the spatial neighborhood of the coefficient; where M1, M2, N1, N2 and N3 are integers greater than or equal to 0, and their values ​​may be equal or unequal, and are not limited here, for example, 1, 2, 3, 4 or 5.

[0397] The preset order 1 and the preset order 2 may be processing orders obtained by zig-zag scanning, or other scanning methods, such as row-by-row scanning, column-by-column scanning, diagonal scanning, horizontal reverse scanning, and vertical reverse scanning, etc. The row-by-row scanning refers to continuous scanning from the first row to the last row of the image, and the column-by-column scanning refers to continuous scanning from the first column to the last column of the image.

[0398] Step 1503: Obtain the transcoded output file.

[0399] The encoding information after entropy encoding is output to obtain a transcoded output file.

[0400] like Fig.16 FIG. 1 is a schematic diagram of an image decoding process provided by an embodiment of the present application. It mainly includes:

[0401] Step 1601: Obtain the coded information after entropy coding, and perform entropy decoding on each code element in the coded information to obtain the decoded transformation coefficient.

[0402] The coded information after entropy coding is entropy decoded, and each codeword is decoded using the same processing order as the encoder. First, each frequency subband is obtained according to the index of the frequency subband or the preset order 1, and then the codewords in each frequency subband are processed according to the preset order 2. When processing each codeword, the decoded codewords in the preset context area of ​​the codeword are used as context input to the probability estimation network to obtain the probability distribution of the current codeword, and the probability distribution is used to decode the current codeword to obtain the transform coefficient of the current codeword. Each codeword output by the encoder is entropy decoded in turn to obtain the transform coefficient of each codeword, thereby obtaining the coefficient block of the image.

[0403] Step 1602: Obtain decoded image data.

[0404] Method 1: Corresponding to the decoding of method 1 in step 401: performing inverse transformation and inverse quantization on the coefficient block to obtain decoded image data.

[0405] Method 2: Corresponding to the decoding of method 2 in step 401: entropy encoding is performed on the 8x8 transform coefficient block to obtain entropy encoded image data.

[0406] In the coding scheme provided in the fifth embodiment, there is no need to generate a two-dimensional sub-band coefficient array, but the frequency sub-band is directly encoded, so the scheme is simpler to implement.

[0407] The embodiment of the present application relates to block transform and coefficient coding in video image compression. The transform coefficients are combined at specified frequency points to obtain a three-dimensional coefficient array C (bx, by, fs), and the coefficient sub-image corresponding to each frequency index fs is losslessly encoded. The transform coefficients are combined at specified frequency points to obtain a three-dimensional coefficient array C (bx, by, fs), and when arithmetic coding is performed on one of the coefficients, the processed coefficients of the three-dimensional neighborhood of the coefficient are used as context to estimate the probability distribution of the coefficient value. Each two-dimensional subband coefficient array C is processed in a preset order. fx,fy (bx, by), when processing each transform coefficient in each two-dimensional subband coefficient array, process each transform coefficient in each two-dimensional subband coefficient array according to the preset order 2, and use the processed coefficients in the neighborhood of the coefficient as context to estimate the probability distribution of the coefficient value. Get the two-dimensional subband coefficient array C of the specified frequency point fx,fy (bx, by). The two-dimensional subband coefficient array corresponding to the frequency point is regarded as a subband image. When processing each subband image, each subband image is processed according to the preset order 1. When processing each transform coefficient of the subband image, the image lossless coding method is used for encoding. Each frequency subband of the current image is processed according to the preset order 1, and the transform coefficient in each frequency subband is processed according to the preset order 2. The frequency subband is a set C of coefficients of the specified frequency point (fx, fy) in the 8x8 transform coefficient block. fx,fy(bx,by).

[0408] It can be seen from the above examples that in the embodiment of the present application, the image area to be processed is divided into image blocks of preset sizes, and each image block is transformed and quantized to obtain a transformation coefficient array C (bx, by, fx, fy). Then, each of the coefficients is entropy encoded in a preset order, and the preset order will give priority to the coefficients of all coefficient blocks corresponding to each frequency index. When entropy encoding each coefficient, the processed coefficients in the adjacent area are selected as the context to estimate the probability distribution of the current coefficient according to the spatial proximity relationship of the coefficients and the proximity relationship of the frequency components of the coefficients, and the subsequent encoding operations are performed on the current coefficient based on the probability distribution. The embodiment of the present application can further reduce the size of the compressed file and reduce the storage cost of the video image file on the server side.

[0409] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but technicians in this field should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, technicians in this field should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0410] In order to better implement the above-mentioned solution of the embodiment of the present application, relevant devices for implementing the above-mentioned solution are also provided below.

[0411] See also Fig.17 As shown, an image encoding device 1700 provided in an embodiment of the present application may include: a transceiver module 1701 and a processing module 1702, wherein:

[0412] A transceiver module, configured to obtain at least two transform coefficient blocks, each of the at least two transform coefficient blocks comprising: at least two transform coefficients, wherein each transform coefficient in one transform coefficient block corresponds to a different frequency point;

[0413] A processing module is used to determine the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order; and encode the transform coefficients corresponding to the one frequency point according to the preset coefficient scanning order to obtain image coding information. In an embodiment of the present application, a frequency point scanning order can be pre-set or configured, and a frequency point is determined based on the frequency point scanning order. The frequency point can be any frequency point, and each frequency point corresponds to a transform coefficient in each transform coefficient block in at least two transform coefficient blocks, that is, one frequency point corresponds to multiple transform coefficients, and these multiple transform coefficients come from different transform coefficient blocks. The coefficient scanning order can be pre-set or configured, and the transform coefficients corresponding to a frequency point are encoded based on the coefficient scanning order, and finally the image coding information can be obtained. In the embodiment of the present application, for the transform coefficients corresponding to the same frequency point in at least two transform coefficient blocks, image coding information can be obtained by only encoding them according to the coefficient scanning order. Therefore, encoding can be achieved for multiple transform coefficients corresponding to the same frequency point, thereby improving image compression efficiency. In the embodiment of the present application, the image coding information is obtained by encoding the transform coefficients corresponding to the same frequency point in at least two transform coefficient blocks according to the coefficient scanning order. The file size of the encoded image coding information is reduced, the storage cost of the image coding information is reduced, and storage resources are ultimately saved.

[0414] In one possible implementation, the processing module is used to obtain at least two two-dimensional subband coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional subband coefficient array includes transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks, and different two-dimensional subband coefficient arrays include transform coefficients corresponding to different frequency points; determine a first two-dimensional subband coefficient array in the at least two two-dimensional subband coefficient arrays according to a preset frequency point scanning order; determine a first transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order; and perform entropy encoding on the first transform coefficient according to a context area corresponding to the first transform coefficient.

[0415] In one possible implementation, the processing module is used to obtain at least two two-dimensional subband coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional subband coefficient array includes transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks, and different two-dimensional subband coefficient arrays include transform coefficients corresponding to different frequency points; the at least two two-dimensional subband coefficient arrays are combined according to a preset frequency point scanning order to obtain a three-dimensional subband coefficient array; a first two-dimensional subband coefficient array in the three-dimensional subband coefficient array is determined according to a preset three-dimensional subband coefficient scanning order; a first transform coefficient in the first two-dimensional subband coefficient array is determined according to a preset coefficient scanning order; and the first transform coefficient is entropy encoded according to a context area corresponding to the first transform coefficient.

[0416] In one possible implementation, the processing module is used to sort the at least two transform coefficient blocks according to a preset frequency point scanning order to obtain at least two sorted transform coefficient blocks; obtain multiple two-dimensional sub-band coefficient arrays corresponding to different frequency points from the at least two sorted transform coefficient blocks; combine the multiple two-dimensional sub-band coefficient arrays corresponding to different frequency points to obtain a three-dimensional sub-band coefficient array; determine a first two-dimensional sub-band coefficient array from the three-dimensional sub-band coefficient array according to a preset three-dimensional sub-band coefficient scanning order; determine a first transform coefficient in the first two-dimensional sub-band coefficient array according to a preset coefficient scanning order; and entropy encode the first transform coefficient according to a context area corresponding to the first transform coefficient.

[0417] In a possible implementation, the three-dimensional sub-band coefficient scanning order includes: a preset horizontal position index change order; or a preset vertical position index change order; or a preset frequency index change order.

[0418] In one possible implementation, the processing module is used to determine a first two-dimensional sub-band coefficient array consisting of transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks according to a preset frequency point scanning order; determine a first transform coefficient in the first two-dimensional sub-band coefficient array according to a preset coefficient scanning order; and perform entropy encoding on the first transform coefficient according to a context area corresponding to the first transform coefficient.

[0419] In a possible implementation, the processing module is used to determine the transform coefficients in the at least two transform coefficient blocks included in a frequency sub-band according to a preset frequency point scanning order, and the transform coefficients in the at least two transform coefficient blocks included in the one frequency sub-band correspond to the same frequency point; determine the first transform coefficient included in the one frequency sub-band according to a preset coefficient scanning order; and perform entropy encoding on the first transform coefficient according to a context area corresponding to the first transform coefficient.

[0420] In a possible implementation, the processing module is used to obtain context information of the first transform coefficient from the context area corresponding to the first transform coefficient, wherein the context information of the first transform coefficient includes: an encoded transform coefficient in the context area corresponding to the first transform coefficient; estimating the first transform coefficient according to the context information of the first transform coefficient to obtain probability distribution information of the first transform coefficient; and arithmetically encoding the first transform coefficient according to the probability distribution information of the first transform coefficient.

[0421] In a possible implementation manner, the context area corresponding to the first transform coefficient includes at least one of the following: a spatial neighborhood of the first transform coefficient, a subband neighborhood of the first transform coefficient, and a three-dimensional neighborhood of the first transform coefficient.

[0422] In one possible implementation, the processing module is used to obtain context information of the multiple transform coefficients from the context area corresponding to the multiple transform coefficients when the first transform coefficient is multiple transform coefficients, wherein the context information of the multiple transform coefficients includes: encoded transform coefficients in the context area corresponding to the multiple transform coefficients; estimating the multiple transform coefficients in parallel according to the context information of the multiple transform coefficients to obtain probability distribution information of the multiple transform coefficients; and arithmetically encoding the multiple transform coefficients in parallel according to the probability distribution information of the multiple transform coefficients.

[0423] In a possible implementation, the context area corresponding to the multiple transform coefficients includes at least one of the following: a spatial neighborhood of the multiple transform coefficients, a subband neighborhood of the multiple transform coefficients, and a three-dimensional neighborhood of the multiple transform coefficients.

[0424] In a possible implementation, the processing module is used to obtain at least two two-dimensional sub-band coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional sub-band coefficient array includes transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks, and different two-dimensional sub-band coefficient arrays include transform coefficients corresponding to different frequency points; determine at least two sub-band images corresponding to the at least two two-dimensional sub-band coefficient arrays according to a preset frequency point scanning order; and encode the transform coefficients in the at least two sub-band images according to a preset coefficient scanning order.

[0425] In a possible implementation, the transceiver module is used to obtain transform coefficient coding information of at least two image blocks; and perform entropy decoding on the transform coefficient coding information to obtain the at least two transform coefficient blocks.

[0426] In a possible implementation, the transceiver module is used to divide an original image area into at least two image blocks, where the original image area includes: a pixel image area or a residual image area; and transform and quantize the at least two image blocks to obtain the at least two transform coefficient blocks.

[0427] In the third aspect of the present application, the constituent modules of the image processing device may also execute the steps described in the aforementioned first aspect and various possible implementations, see the aforementioned description of the first aspect and various possible implementations for details.

[0428] See also Fig.18 As shown, an image decoding device 1800 provided in an embodiment of the present application may include: a transceiver module 1801 and a processing module 1802, wherein:

[0429] A transceiver module, used for obtaining image coding information;

[0430] A processing module is used to determine the transform coefficient to be decoded corresponding to a frequency point in the image coding information according to a preset frequency point scanning order; decode the transform coefficient to be decoded corresponding to the one frequency point according to a preset coefficient scanning order to obtain at least two transform coefficient blocks, wherein each transform coefficient block in the at least two transform coefficient blocks includes: at least two transform coefficients, and each transform coefficient in a transform coefficient block corresponds to a different frequency point.

[0431] In a possible implementation, the processing module is used to determine a first two-dimensional subband coefficient array to be decoded in the image coding information according to a preset frequency point scanning order, wherein the first two-dimensional subband coefficient array to be decoded includes: a transform coefficient to be decoded corresponding to a frequency point in the image coding information; determining the first transform coefficient to be decoded in the first two-dimensional subband coefficient array to be decoded according to a preset coefficient scanning order; and performing entropy decoding on the first transform coefficient to be decoded according to a context area corresponding to the first transform coefficient to be decoded.

[0432] In a possible implementation, the processing module is used to determine a three-dimensional subband coefficient array to be decoded in the image coding information according to a preset frequency point scanning order, the three-dimensional subband coefficient array to be decoded includes: transform coefficients to be decoded corresponding to all frequency points in the image coding information; determine a first two-dimensional subband coefficient array to be decoded in the three-dimensional subband coefficient array to be decoded according to a preset three-dimensional subband coefficient scanning order, the first two-dimensional subband coefficient array to be decoded includes: a transform coefficient to be decoded corresponding to a frequency point in the image coding information; determine a first transform coefficient to be decoded in the first two-dimensional subband coefficient array to be decoded according to a preset coefficient scanning order; and perform entropy decoding on the first transform coefficient to be decoded according to a context area corresponding to the first transform coefficient to be decoded.

[0433] In a possible implementation, the three-dimensional sub-band coefficient scanning order includes: a preset horizontal position index change order; or a preset vertical position index change order; or a preset frequency index change order.

[0434] In a possible implementation, the processing module is used to determine the transform coefficients to be decoded included in a frequency subband in the image coding information according to a preset frequency point scanning order, and the transform coefficients to be decoded included in the frequency subband correspond to the same frequency point; determine the first transform coefficient to be decoded in the frequency subband according to a preset coefficient scanning order; and perform entropy decoding on the first transform coefficient to be decoded according to a context area corresponding to the first transform coefficient to be decoded.

[0435] In a possible implementation, the processing module is used to obtain context information of the first transform coefficient to be decoded from the context area corresponding to the first transform coefficient to be decoded, wherein the context area corresponding to the first transform coefficient to be decoded includes: a decoded transform coefficient in the context area corresponding to the first transform coefficient to be decoded; estimating the first transform coefficient to be decoded according to the context information of the first transform coefficient to be decoded to obtain probability distribution information of the first transform coefficient to be decoded; arithmetically decoding the first transform coefficient to be decoded according to the probability distribution information of the first transform coefficient to be decoded to obtain a decoded first transform coefficient, and the decoded first transform coefficient belongs to the at least two transform coefficient blocks.

[0436] In a possible implementation, the context area corresponding to the first transform coefficient to be decoded includes at least one of the following: a spatial neighborhood of the first transform coefficient to be decoded, a subband neighborhood of the first transform coefficient to be decoded, and a three-dimensional neighborhood of the first transform coefficient to be decoded.

[0437] In one possible implementation, the processing module is used to obtain context information of the multiple transform coefficients to be decoded from the context area corresponding to the multiple transform coefficients to be decoded when the first transform coefficient to be decoded is a plurality of transform coefficients to be decoded, wherein the multiple context information to be decoded include: decoded transform coefficients in the context area corresponding to the multiple transform coefficients to be decoded; estimating the multiple transform coefficients to be decoded in parallel according to the context information of the multiple transform coefficients to be decoded to obtain probability distribution information of the first transform coefficient to be decoded; and arithmetically decoding the multiple transform coefficients to be decoded in parallel according to the probability distribution information of the multiple transform coefficients to be decoded to obtain a decoded first transform coefficient, and the decoded first transform coefficient belongs to the at least two transform coefficient blocks.

[0438] In a possible implementation, the context area corresponding to the multiple transform coefficients to be decoded includes at least one of the following: a spatial neighborhood of the multiple transform coefficients to be decoded, a subband neighborhood of the multiple transform coefficients to be decoded, and a three-dimensional neighborhood of the multiple transform coefficients to be decoded.

[0439] In a possible implementation, the processing module is used to determine the first sub-band image in the image coding information according to a preset frequency point scanning order, the first sub-band image including: the transform coefficient to be decoded corresponding to a frequency point in the image coding information; the coefficient to be decoded of the first sub-band image is decoded according to a preset coefficient scanning order to obtain the at least two transform coefficient blocks.

[0440] In a possible implementation, the processing module is further used to, after obtaining at least two transform coefficient blocks, perform entropy encoding on the at least two transform coefficient blocks to obtain transform coefficient encoding information; generate at least two image blocks according to the transform coefficient encoding information; and restore the original image area according to the at least two image blocks, wherein the original image area includes: a pixel image area or a residual image area.

[0441] In a possible implementation, the processing module is further used to, after obtaining at least two transform coefficient blocks, perform inverse quantization and inverse transformation on the at least two transform coefficient blocks to obtain at least two image blocks; and restore the original image area based on the at least two image blocks, wherein the original image area includes: a pixel image area or a residual image area.

[0442] It should be noted that the information interaction, execution process, etc. between the modules / units of the above-mentioned device are based on the same concept as the method embodiment of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown above in the present application, and will not be repeated here.

[0443] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a program, and the program executes some or all of the steps recorded in the above method embodiment.

[0444] Next, another image encoding device provided by the embodiment of the present application is introduced. Fig.19 As shown, the image encoding device 1900 includes:

[0445] A receiver 1901, a transmitter 1902, a processor 1903 and a memory 1904 (wherein the number of the processor 1903 in the image encoding device 1900 may be one or more, Fig.19 In some embodiments of the present application, the receiver 1901, the transmitter 1902, the processor 1903 and the memory 1904 may be connected via a bus or other means, wherein: Fig.19 The example of connecting through bus is taken in the following.

[0446] The memory 1904 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1903. A portion of the memory 1904 may also include a non-volatile random access memory (NVRAM). The memory 1904 stores an operating system and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic services and processing hardware-based tasks.

[0447] The processor 1903 controls the operation of the image coding device, and the processor 1903 may also be referred to as a central processing unit (CPU). In a specific application, the various components of the image coding device are coupled together through a bus system, wherein the bus system may include a power bus, a control bus, and a status signal bus in addition to a data bus. However, for the sake of clarity, various buses are referred to as bus systems in the figure.

[0448] The method disclosed in the above embodiment of the present application can be applied to the processor 1903, or implemented by the processor 1903. The processor 1903 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 1903. The above processor 1903 can be a general processor, a digital signal processor (digital signal processing, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in this field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1904, and the processor 1903 reads the information in the memory 1904 and completes the steps of the above method in combination with its hardware.

[0449] Receiver 1901 can be used to receive input digital or character information, and generate signal input related to the relevant settings and function control of the image encoding device. Transmitter 1902 may include a display device such as a display screen. Transmitter 1902 can be used to output digital or character information through an external interface.

[0450] In the embodiment of the present application, the processor 1903 is used to execute the above-mentioned embodiment. Figure 3 The image processing method performed by the image encoding device is shown.

[0451] Next, another image decoding device provided by the embodiment of the present application is introduced. Fig. 20 As shown, the image decoding apparatus 2000 comprises:

[0452] The receiver 2001, the transmitter 2002, the processor 2003 and the memory 2004 (the number of the processor 2003 in the image decoding device 2000 may be one or more, Fig. 20 In some embodiments of the present application, the receiver 2001, the transmitter 2002, the processor 2003 and the memory 2004 may be connected via a bus or other means, wherein: Fig. 20 The example of connecting through bus is taken in the following.

[0453] The memory 2004 may include a read-only memory and a random access memory, and provides instructions and data to the processor 2003. A portion of the memory 2004 may also include an NVRAM. The memory 2004 stores an operating system and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic services and processing hardware-based tasks.

[0454] The processor 2003 controls the operation of the image decoding device, and the processor 2003 may also be referred to as a CPU. In a specific application, the various components of the image decoding device are coupled together through a bus system, wherein the bus system may include a power bus, a control bus, and a status signal bus in addition to a data bus. However, for the sake of clarity, various buses are referred to as bus systems in the figure.

[0455] The method disclosed in the above embodiment of the present application can be applied to the processor 2003, or implemented by the processor 2003. The processor 2003 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 2003. The above processor 2003 can be a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in this field. The storage medium is located in the memory 2004, and the processor 2003 reads the information in the memory 2004 and completes the steps of the above method in combination with its hardware.

[0456] In the embodiment of the present application, the processor 2003 is used to execute the above embodiment Figure 3 The image processing method shown is performed by the image decoding device.

[0457] In another possible design, when the image processing device or the image processing device is a chip in a terminal, the chip includes: a processing unit and a communication unit, the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit, etc. The processing unit may execute computer-executable instructions stored in the storage unit so that the chip in the terminal executes the wireless communication method of any one of the above-mentioned first aspects. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc., and the storage unit may also be a storage unit in the terminal located outside the chip, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0458] The processor mentioned in any of the above places may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned first aspect or second aspect method.

[0459] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0460] Through the description of the above implementation mode, the technicians in the related field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0461] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0462] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. An image processing method, characterized in that: include: Acquire at least two transform coefficient blocks, each of the at least two transform coefficient blocks comprising: at least two transform coefficients, wherein each transform coefficient in one transform coefficient block corresponds to a different frequency point; Determine the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order; Simultaneously encoding each transform coefficient corresponding to the one frequency point according to a preset coefficient scanning order to obtain image coding information; The step of simultaneously encoding each transform coefficient corresponding to the one frequency point according to a preset coefficient scanning order includes: Determine a first transform coefficient from a first two-dimensional subband coefficient array in at least two two-dimensional subband coefficient arrays according to a preset coefficient scanning order; perform entropy coding on the first transform coefficient based on a context area corresponding to the first transform coefficient, wherein one two-dimensional subband coefficient array includes transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks, and transform coefficients included in different two-dimensional subband coefficient arrays correspond to different frequency points; or, A first transform coefficient included in a frequency subband is determined according to a preset coefficient scanning order, and the first transform coefficient is entropy encoded according to a context area corresponding to the first transform coefficient, wherein the transform coefficients in the at least two transform coefficient blocks included in the one frequency subband correspond to the same frequency point.

2. The method according to claim 1, characterized in that The step of determining the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order includes: Acquire at least two two-dimensional subband coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional subband coefficient array includes transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks, and different two-dimensional subband coefficient arrays include transform coefficients corresponding to different frequency points; A first two-dimensional sub-band coefficient array in the at least two two-dimensional sub-band coefficient arrays is determined according to a preset frequency point scanning order.

3. The method according to claim 1, characterized in that The step of determining the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order includes: Acquire at least two two-dimensional subband coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional subband coefficient array includes transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks, and different two-dimensional subband coefficient arrays include transform coefficients corresponding to different frequency points; Combining the at least two two-dimensional sub-band coefficient arrays according to a preset frequency point scanning order to obtain a three-dimensional sub-band coefficient array; A first two-dimensional sub-band coefficient array in the three-dimensional sub-band coefficient array is determined according to a preset three-dimensional sub-band coefficient scanning order.

4. The method according to claim 1, characterized in that The step of determining the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order includes: Sorting the at least two transform coefficient blocks according to a preset frequency point scanning order to obtain at least two sorted transform coefficient blocks; Acquire a plurality of two-dimensional sub-band coefficient arrays corresponding to different frequency points from the at least two sorted transform coefficient blocks; Combining the multiple two-dimensional sub-band coefficient arrays corresponding to the different frequency points to obtain a three-dimensional sub-band coefficient array; A first two-dimensional sub-band coefficient array is determined from the three-dimensional sub-band coefficient array according to a preset three-dimensional sub-band coefficient scanning order.

5. The method according to claim 3 or 4, characterized in that: The three-dimensional sub-band coefficient scanning order includes: A preset order of changing the horizontal position index; or, The change order of the preset vertical position index; or, The preset frequency index changes in order.

6. The method according to claim 1, characterized in that The step of determining the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order includes: A first two-dimensional subband coefficient array consisting of transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks is determined according to a preset frequency point scanning order.

7. The method according to any one of claims 2 to 4, characterized in that The entropy encoding the first transform coefficient according to the context area corresponding to the first transform coefficient includes: When the first transform coefficient is a plurality of transform coefficients, context information of the plurality of transform coefficients is acquired from context areas corresponding to the plurality of transform coefficients, wherein the context information of the plurality of transform coefficients comprises: encoded transform coefficients in the context areas corresponding to the plurality of transform coefficients; estimating the plurality of transform coefficients in parallel according to context information of the plurality of transform coefficients to obtain probability distribution information of the plurality of transform coefficients; The multiple transform coefficients are arithmetically encoded in parallel according to probability distribution information of the multiple transform coefficients.

8. The method according to claim 7, characterized in that The context area corresponding to the multiple transform coefficients includes at least one of the following: a spatial neighborhood of the multiple transform coefficients, a subband neighborhood of the multiple transform coefficients, and a three-dimensional neighborhood of the multiple transform coefficients.

9. The method according to any one of claims 1 to 4, characterized in that The obtaining of at least two transform coefficient blocks comprises: Obtaining transform coefficient coding information of at least two image blocks; The transform coefficient encoding information is entropy decoded to obtain the at least two transform coefficient blocks.

10. The method according to any one of claims 1 to 4, characterized in that The obtaining of at least two transform coefficient blocks comprises: Dividing an original image area into at least two image blocks, the original image area comprising: a pixel image area or a residual image area; The at least two image blocks are transformed and quantized to obtain the at least two transform coefficient blocks.

11. An image processing method, characterized in that: include: Get image coding information; Determine a transform coefficient to be decoded corresponding to a frequency point in the image coding information according to a preset frequency point scanning order; The to-be-decoded transform coefficients corresponding to the one frequency point are simultaneously decoded according to a preset coefficient scanning order to obtain at least two transform coefficient blocks, wherein each of the at least two transform coefficient blocks comprises: at least two transform coefficients, and each transform coefficient in one transform coefficient block corresponds to a different frequency point; The step of simultaneously decoding the to-be-decoded transform coefficients corresponding to the one frequency point in a preset coefficient scanning order includes: Determine, according to a preset coefficient scanning order, a first transform coefficient to be decoded in a first two-dimensional subband coefficient array to be decoded in at least two two-dimensional subband coefficient arrays to be decoded; The first transform coefficient to be decoded is entropy decoded according to the context area corresponding to the first transform coefficient to be decoded, a two-dimensional sub-band coefficient array to be decoded includes transform coefficients corresponding to the same frequency point in the at least two transform coefficient blocks, and the transform coefficients included in different two-dimensional sub-band coefficient arrays to be decoded correspond to different frequency points; or, the first transform coefficient to be decoded in a frequency sub-band is determined according to a preset coefficient scanning order, and the first transform coefficient to be decoded is entropy decoded according to the context area corresponding to the first transform coefficient to be decoded, and the transform coefficients in the at least two transform coefficient blocks included in the one frequency sub-band correspond to the same frequency point.

12. The method according to claim 11, characterized in that The step of determining the to-be-decoded transform coefficient corresponding to a frequency point in the image coding information according to a preset frequency point scanning order includes: A first two-dimensional subband coefficient array to be decoded in the image coding information is determined according to a preset frequency point scanning order, wherein the first two-dimensional subband coefficient array to be decoded includes: a transform coefficient to be decoded corresponding to a frequency point in the image coding information.

13. The method according to claim 11, characterized in that The step of determining the to-be-decoded transform coefficient corresponding to a frequency point in the image coding information according to a preset frequency point scanning order includes: Determine the three-dimensional subband coefficient array to be decoded in the image coding information according to a preset frequency point scanning order, wherein the three-dimensional subband coefficient array to be decoded includes: the transform coefficients to be decoded corresponding to all frequency points in the image coding information; A first two-dimensional subband coefficient array to be decoded in the three-dimensional subband coefficient array to be decoded is determined according to a preset three-dimensional subband coefficient scanning order, wherein the first two-dimensional subband coefficient array to be decoded includes: a transform coefficient to be decoded corresponding to a frequency point in the image coding information.

14. The method according to claim 13, characterized in that The three-dimensional sub-band coefficient scanning order includes: A preset order of changing the horizontal position index; or, The change order of the preset vertical position index; or, The preset frequency index changes in order.

15. The method according to any one of claims 12 to 14, characterized in that The entropy decoding of the first transform coefficient to be decoded according to the context area corresponding to the first transform coefficient to be decoded includes: When the first transform coefficient to be decoded is a plurality of transform coefficients to be decoded, context information of the plurality of transform coefficients to be decoded is obtained from context areas corresponding to the plurality of transform coefficients to be decoded, wherein the context information of the plurality of transform coefficients to be decoded includes: decoded transform coefficients in the context areas corresponding to the plurality of transform coefficients to be decoded; estimating the plurality of transform coefficients to be decoded in parallel according to context information of the plurality of transform coefficients to be decoded, so as to obtain probability distribution information of the first transform coefficient to be decoded; The multiple transform coefficients to be decoded are arithmetically decoded in parallel according to the probability distribution information of the multiple transform coefficients to be decoded to obtain decoded first transform coefficients, and the decoded first transform coefficients belong to the at least two transform coefficient blocks.

16. The method according to claim 15, characterized in that The context area corresponding to the multiple transform coefficients to be decoded includes at least one of the following: a spatial neighborhood of the multiple transform coefficients to be decoded, a subband neighborhood of the multiple transform coefficients to be decoded, and a three-dimensional neighborhood of the multiple transform coefficients to be decoded.

17. The method according to claim 11, characterized in that The step of determining the to-be-decoded transform coefficient corresponding to a frequency point in the image coding information according to a preset frequency point scanning order includes: The first sub-band image in the image coding information is determined according to a preset frequency point scanning order, wherein the first sub-band image includes: a transform coefficient to be decoded corresponding to a frequency point in the image coding information.

18. The method according to any one of claims 11 to 14, characterized in that After obtaining at least two transform coefficient blocks, the method further includes: Dequantizing and detransforming the at least two transform coefficient blocks to obtain at least two image blocks; An original image region is restored according to the at least two image blocks, where the original image region includes: a pixel image region or a residual image region.

19. An image processing device, characterized in that: The image processing device comprises: a processor and a memory, wherein the processor is configured to be coupled to the memory, read and execute instructions in the memory, and execute the method according to any one of claims 1 to 10 or claims 11 to 18.

20. A computer-readable storage medium comprising instructions, which, when executed on a computer, causes the computer to perform the method according to any one of claims 1 to 10 or claims 11 to 18.

Images